Implementing NEF Neural Networks on Embedded FPGAs

International Conference on Field-Programmable Technology (FPT), 2018

Benjamin Morcos, Terrence C Stewart, Chris Eliasmith, Nachiket Kapre

Abstract

Low-power, high-speed neural networks are critical for providing deployable embedded AI applications at the edge. We describe an FPGA implementation of Neural Engineering Framework (NEF) networks with online learning that outperforms mobile GPU implementations by an order of magnitude or more. Specifically, we provide an embedded Python-capable PYNQ FPGA implementation supported with a High-Level Synthesis (HLS) workflow that allows sub-millisecond implementation of adaptive neural networks with low-latency, direct I/O access to the physical world. We tune the precision of the different intermediate variables in the code to achieve competitive absolute accuracy against slower and larger floating-point reference designs. The online learning component of the neural network exploits immediate feedback to adjust the network weights to best support a given arithmetic precision. As the space of possible design configurations of such networks is vast and is subject to a target accuracy constraint, we use the Hyperopt hyper-parameter tuning tool instead of manual search to find Pareto optimal designs. Specifically, we are able to generate the optimized designs in under 500 iterations of Vivado HLS before running the complete Vivado place-and-route phase on that subset. For neural network populations of 64–4096 neurons and 1–8 representational dimensions our optimized FPGA implementation generated by Hyperopt has a speedup of 10–484× over a competing cuBLAS implementation on the Jetson TX1 GPU while using 2.4–9.5× less power. Our speedups are a result of HLS-specific reformulation (15× improvement), precision adaptation (4× improvement), and low-latency direct I/O access (1000× improvement).

Full text links

 PDF

 DOI

Supplementary information

 Presentation

Conference Proceedings

Booktitle
International Conference on Field-Programmable Technology (FPT)
Eventdate
2018-12-11/2018-12-14
Location
Naha, Okinawa, Japan
Publisher
IEEE
Pages
25-32
Doi
10.1109/FPT.2018.00015

Cite

Plain text

BibTeX