Hopfield Networks on the SoC FPAA
The Hopfield network, inspired by the interconnected structure of neural systems, can "remember" patterns through a Hebbian learning rule and attractor dynamics (Hopfield, PNAS 1982) . These dynamics find local minimum over an energy surface essentially performing gradient descent, a technique that can be applied to many computationally difficult problems such as the NP-Hard class.
Hopfield networks can be created from analog circuits. The Field-Programmable Analog Array (FPAA) is especially amenable to these structures, having been shown to efficiently map Hopfield networks and solve diverse NP-hard problems(Mathews, ISCAS 2023; Mathews, TCAS 2024; Mathews, TVLSI 2024; Mathews, ICRC 2024) This page will cover how to program a Hopfield network onto the SoC FPAA.
FPAA Implementation of a Hopfield Network
A Hopfield network is made of \(N\) interconnected neurons. The network dynamics minimize a quadratic energy function,
\begin{equation}
H = -\frac{1}{2}\sum_{i}^N\sum_j^N W_{ij}x_ix_j - \sum_{k}^N I_kx_i ,
\label{eqn:HopfieldEnergy}
\end{equation}
where \(W_{ij}\) describes the weighted connections between different neurons, \(x_i\) represents the output state of neuron \(i\), and \(I_k\) is an external bias.
A neurons output,\(x_i\), is described by a sigmoid-like function (\(S_{i}\)) that converges to
\begin{equation}
x_{i} = \begin{cases}
1, & \sum_{j} x_{ji} \geq T\\
0, & \sum_{j} x_{ji} < T
\end{cases}
\label{Hopfield}
,
\end{equation}
which can be broken into two main elements: a signed, weighted summation and a sigmoidal thresholding function. These two elements can be replicated using analog circuitry and then interconnected to form an analog Hopfield network.
We use an OTA to perform the sigmoid function, and the routing FGs inside the FPAA for the signed-weighted summation. The schematic can be seen in the above figure. On the 3.0 SoC FPAA, four Hopfield neurons can fit in a single CAB and take 13 external inputs, for a maxmimum network size of 16 x 16.
Programming the SoC FPAA
The following information assumes you have familiarity with creating macrocabs through the SOC FPAA tools.
The Vector-Matrix-Multiplier of the Hopfield Network uses a large amount of Floating-Gates (FG). This triggers a known bug in the Scilab tools, which can be rectified through the steps listed in Fixing_Scilab_Bugs.pdf .
An Example 4-Neuron Hopfield Network
We can follow the steps with an example Macrocab.
- Step 1: Change the number on lines 636 and 642 from 100 -> 500 in rasp30/sci2blif/macrocab_gen_fcn.sce
- Step 2: Restart Scilab. Then, use the Macrocab GUI to add Hopfield4.xcos to the tools
- Step 3: Replace Hopfield4.sce in rasp30/sci2blif/sci2blif_added_blocks/Hopfield4.sce with the modified Hopfield4.sce
- Step 4: Replace Hopfield4.sci in rasp30/xcos_blocks/Hopfield4.sci with the modified Hopfield4.sci
- Step 5: Restart Scilab. The Hopfield Network should now be ready to use
Some usage notes:
- Ibiasp/Ibiasn correspond to the leakage weights used to bias zero input towards a high neuron output
- IbiasG is the FG pFET ItoV converter used in 2/4 neurons.
- SigmoidBias controls the current on all TAs
- The weights are given as a 8x6 matrix
An example weight matrix for solving the Max-Cut problem of a 4-Neuron, 4 Edge Graph is shown below. Input 1 forces an initial condtion of 0000, and the network converges after the condition is released.
|
Input 1 |
Input 2 |
Neuron 1 Feedback |
Neuron 2 Feedback |
Neuron 3 Feedback |
Neuron 4 Feedback |
| Neuron 1 Positive |
1.000D-12 |
10.00D-9 |
1.000D-12 |
1.000D-12 |
1.000D-12 |
1.000D-12 |
| Neuron 1 Negative |
10.00D-6 |
1.000D-12 |
1.000D-12 |
10.00D-9 |
10.00D-9 |
10.00D-9 |
| Neuron 2 Positive |
1.000D-12 |
10.00D-9 |
1.000D-12 |
1.000D-12 |
1.000D-12 |
1.000D-12 |
| Neuron 2 Negative |
10.00D-6 |
1.000D-12 |
10.00D-9 |
1.000D-12 |
1.000D-12 |
10.00D-9 |
| Neuron 3 Positive |
1.000D-12 |
5.00D-9 |
1.000D-12 |
1.000D-12 |
1.000D-12 |
1.000D-12 |
| Neuron 3 Negative |
10.00D-6 |
1.000D-12 |
10.00D-9 |
1.000D-12 |
1.000D-12 |
1.000D-12 |
| Neuron 4 Positive |
1.000D-12 |
5.00D-9 |
1.000D-12 |
1.000D-12 |
1.000D-12 |
1.000D-12 |
| Neuron 4 Negative |
10.00D-6 |
1.000D-12 |
10.00D-9 |
10.00D-9 |
1.000D-12 |
1.000D-12 |
For convenience, I have also created a rasp30 folder with the Hopfield network pre-installed (rasp30_V5.zip). The Hopfield4 block will already be in the Palette browser, but there are also examples for on-chip and off-chip usage (under the examples tab in the Blue GUI). The off-chip uses the shift register, which can be queried by running the command "GPIO(1,#)" in the Scilab command window, where "#" indicates which input you want to see.
|