1

Pertemuan 17HOPFIELD NETWORK

Matakuliah : H0434/Jaringan Syaraf Tiruan

Tahun : 2005

Versi : 1

2

Learning Outcomes

Pada akhir pertemuan ini, diharapkan mahasiswa

akan mampu :

• Menjelaskan konsep dari Jaringan Hopfield

3

Outline Materi

• Hopfield Model.

• Lyapnov Function.

4

Hopfield Model

5

Equations of Operation

Cdni t( )

dt------------- T i j aj t( )

j 1=

S

ni t( )

Ri----------– Ii+=

ni - input voltage to the ith amplifierai - output voltage of the ith amplifierC - amplifier input capacitanceIi - fixed input current to the ith amplifier

T i j1Ri j---------= 1

Ri-----

1---

1Ri j---------

j 1=

S

+= ni f1–ai = ai f ni =

6

Network Format

RiCdni t( )

dt------------- RiT i j a j t( )

j 1=

S

ni t( )– RiI i+=

RiC= w i j RiT i j= bi RiI i=

Define:

d ni t( )

dt------------- ni t( )– wi j aj t( )

j 1=

S

bi+ +=

dn t( )dt

------------ n t( )– Wa t( ) b+ +=

a t( ) f n t( ) =

Vector Form:

7

Hopfield Network

8

Lyapunov Function

V a 12---aTWa– f

1–u ud

0

ai

i 1=

S

bTa–+=

9

Individual Derivatives

tdd 1

2---aTWa–

1

2--- aTWa

Tdadt------– Wa Tda

dt------– aTWda

dt------–= = =

tdd

f1–u ud

0

ai

aidd

f1–u ud

0

ai

td

daif

1–ai

td

daini td

dai= = =

ddt----- f

1–u ud

0

ai

i 1=

S

nTdadt------=

tdd bTa– bTa

Tdadt------– bTda

dt------–= =

Third Term:

Second Term:

First Term:

10

Complete Lyapunov Derivative

tddV a –

dn t( )dt

------------Tdadt------ –

td

dni

td

dai

i 1=

S

–td

dni

td

dai

i 1=

S

= = =

–aiddf

1–ai

td

dai

2

i 1=

S

=

tddV a a

TWdadt------– n

Tdadt------ b

Tdadt------–+ a

TW– n

TbT

–+ dadt------= =

aTW– n

TbT

–+ –dn t( )dt

------------T

=

From the system equations we know:

So the derivative can be written:

tddV a 0If thenaid

df

1–ai 0

11

Invariant Sets

Z a : dV a dt 0= a in the closure of G =

tddV a –

aiddf

1–ai

td

dai

2

i 1=

S

=

This will be zero only if the neuron outputs are not changing:dadt------ 0=

Therefore, the system energy is not changing only at the equilibrium points of the circuit. Thus, all points in Z arepotential attractors:

L Z=

12

Example

a f n 2---tan

1– n2

--------- == n

2------ tan

2---a =

R1 2 R2 1 1= =

T1 2 T2 1 1= =W 0 1

1 0=

RiC 1= =

1.4=

I1 I2 0= = b 00

=

13

Example Lyapunov Function

V a 12---aTWa– f

1–u ud

0

ai

i 1=

S

bTa–+=

12---aTWa–

12--- a1 a2

0 11 0

a1

a2

– a1a2–= =

f1–u ud

0

a i

2------

2---u tan ud

0

a i

2------ log

2---u cos

2---–a i

0

4

2--------- log

2---ai cos–= = =

V a a1a2– 4

1.42------------- log

2---a1 cos

log 2---a2 cos

+–=

14

Example Network Equations

dndt------- n– Wf n + n– Wa+= =

dn1 dt a2 n1–=

dn2 dt a1 n2–=

a12---tan

1– 1.42

-----------n1 =

a22---tan

1– 1.42

-----------n2 =

15

Lyapunov Function and Trajectory

-1 -0.5 0 0.5 1-1

-0.5

0

0.5

1

-1

-0.5

0

0.5

1

-1

-0.5

0

0.5

1

0

1

2

a1

a2

a2 a1

V(a)

16

Time Response

0 2 4 6 8 10-1

-0.5

0

0.5

1

0 2 4 6 8 10

0

0.5

1

1.5

2

t t

a1

a2

V(a)

17

Convergence to a Saddle Point

-1 -0.5 0 0.5 1-1

-0.5

0

0.5

1

a1

a2

18

Hopfield Attractors

dadt------ 0=

Va1

Va2

V ...

aSV

T

0= =

The potential attractors of the Hopfield network satisfy:

How are these points related to the minima of V(a)? Theminima must satisfy:

Where the Lyapunov function is given by:

V a 12---aTWa– f

1–u ud

0

ai

i 1=

S

bTa–+=

19

Hopfield Attractors

Using previous results, we can show that:

V a W– a n b–+ –dn t( )dt

------------= =

The ith element of the gradient is therefore:

aiV a –

td

dni –tdd

f1–ai ( ) –

aidd

f1–ai

td

da i= = =

aiddf

1–ai 0

d a t( )dt

------------ 0= V a 0=

Since the transfer function and its inverse are monotonicincreasing:

All points for which will also satisfy

Therefore all attractors will be stationary points of V(a).

20

Effect of Gain

a f n 2--- tan

1– n2

--------- = =

-5 -2.5 0 2.5 5-1

-0.5

0

0.5

1

1.4=

0.14=

14=

n

a

21

Lyapunov Function

V a 12---aTWa– f

1–u ud

0

ai

i 1=

S

bTa–+= f1–u 2

------

u2

------ tan=

-1 -0.5 0 0.5 1

0

0.5

1

1.5

1.4=

0.14=

14=

a

f1–u ud

0

ai

2------

2---

ai2

-------- cos

log4

2---------ai2

-------- coslog–= =

4

2---------a2

-------- coslog–

22

High Gain Lyapunov Function

V a 12---aTWa– b

Ta–

12---aTAa d

Ta c+ += =

V a 2 A W–= = d b–= c 0=

where

V a 12---aTWa– bTa–=

As the Lyapunov function reduces to:

The high gain Lyapunov function is quadratic:

23

Example

V a 2 W– 0 1–1– 0

= = V a 2 I– – 1–1– –

21– 1+ 1– = = =

1 1–= 2 1=z11

1= z2

1

1–=

-1 -0.5 0 0.5 1-1

-0.5

0

0.5

1

-1

-0.5

0

0.5

1

-1

-0.5

0

0.5

1-1

-0.5

0

0.5

1

a1

a2

a2 a1

V(a)