FPGA based customised acceleration means moving the hot loop of an application out of software and into a hardware circuit built for that one job on a field-programmable gate array. Because the circuit is shaped around the algorithm, with its own parallel processing elements, on-chip memories and number formats, it can beat a general-purpose processor on throughput, latency and energy per operation for the right workloads. The trade-off is engineering effort: the gains only arrive when the workload suits streaming hardware and the team can design for it.
In this post I introduce the concept of FPGA based customised acceleration and explain why it is a powerful technique for improving the performance and efficiency of many applications, and where its limits are.
Key takeaways
- An FPGA accelerator wins by matching the hardware to the algorithm: deep pipelines, many parallel processing elements and data kept in on-chip memory.
- It suits streaming, fixed-function and low-latency work such as packet processing, video, compression, cryptography and low-precision inference.
- FPGAs are usually more energy-efficient than CPUs for these tasks, but they are not zero-power when idle: static power is significant.
- Partial reconfiguration lets a running FPGA swap accelerator functions, which is the basis of adaptable and fault-tolerant designs.
- The hard parts are development effort, clock speeds well below CPUs and GPUs, and getting data to the FPGA fast enough.
How customised acceleration works
One of the main advantages of FPGA based acceleration is that it can exploit the parallelism and locality of the application, reducing the communication and synchronisation overheads that are common in general-purpose processors. A CPU spends much of its energy fetching and decoding instructions and moving data through a cache hierarchy. An FPGA circuit has no instruction stream: the datapath is the program.
For example, FPGAs can implement multiple processing elements (PEs) that operate on different data streams simultaneously, or use dedicated memory blocks close to the PEs, avoiding the latency and bandwidth limits of external memory. FPGAs can also perform fine-grained optimisations, such as customising the data types, arithmetic operations and control logic to match the precision and accuracy the application really needs.
The main techniques
| Technique | What it means on an FPGA | Typical benefit |
|---|---|---|
| Pipelining | Split the computation into stages so a new input enters every clock | One result per cycle regardless of algorithm depth |
| Spatial parallelism | Replicate PEs across the fabric and DSP blocks | Throughput scales with the number of PEs that fit |
| Data locality | Keep working data in block RAM, UltraRAM or on-package HBM | Less external memory traffic and lower latency |
| Custom precision | Use 8-bit, 4-bit, fixed-point or odd widths instead of 32-bit float | More operations per DSP block and per watt |
| Custom I/O | Process network or sensor data directly as it arrives | Microsecond-level, deterministic latency |
Worked example: why a 300 MHz FPGA can beat a 3 GHz CPU (illustrative)
Take a filter that needs 64 multiply-accumulate (MAC) operations per output sample. Suppose a CPU core sustains 16 MACs per cycle with vector instructions at 3 GHz, which is 48 GMAC/s at best, and less once memory stalls are counted. An FPGA design with 1,024 MAC units running at 300 MHz delivers 1,024 x 0.3e9 = about 307 GMAC/s, or about 4.8 billion output samples per second, at a fraction of the clock speed. The numbers are illustrative; the point is that the FPGA wins through width and pipelining, not frequency.
Energy efficiency and reconfigurability
Another benefit of FPGA based acceleration is energy efficiency. Because the circuit does only the work the algorithm needs, energy per operation is often much lower than on a CPU. A correction to the original version of this post: FPGAs do not consume power only when active and switch off when idle. They draw significant static (leakage) power whenever they are powered and configured. Designers reduce dynamic power with clock gating and clock enables, and some devices support power-down of unused blocks, but an idle FPGA is not at zero power.
FPGAs can reconfigure themselves to adapt to changing workloads or conditions. With partial reconfiguration, one region of the device can be loaded with a new accelerator while the rest keeps running. Systems can also scale clock frequency, and on some platforms supply voltage, to balance performance and power, or switch between configurations. For reliability, FPGAs can detect and correct errors in the hardware: configuration memory scrubbing finds and repairs single event upsets, and ECC protects on-chip RAM, which matters in space, automotive and data centre use.
Where it has been applied
FPGA based customised acceleration has been applied in image and video processing, machine learning inference, cryptography, bioinformatics such as genome alignment, network processing, financial trading and database or storage offload.
Some well-known examples:
- Microsoft Project Catapult: FPGAs in Microsoft data centre servers, used to accelerate Bing search ranking and Azure cloud services such as accelerated networking.
- Amazon AWS EC2 F1 instances: cloud access to FPGAs (Xilinx UltraScale+ devices) so customers can deploy their own accelerators. AWS later introduced F2 instances with AMD FPGAs that include HBM.
- Intel Stratix 10 MX FPGA: integrates high-bandwidth memory (HBM2) in the package for faster data transfer and processing. Intel's FPGA business now operates as Altera.
- Xilinx Zynq UltraScale+ MPSoC: combines Arm processors and FPGA fabric for heterogeneous computing. Xilinx is now part of AMD.
- Google TPU (Tensor Processing Unit): included as a contrast. The TPU is a custom ASIC, not an FPGA. It applies the same principles of a dedicated, highly parallel datapath (a systolic array of MAC units) and reduced precision, but it is fixed in silicon rather than reconfigurable.
FPGA, GPU or ASIC
| Factor | FPGA | GPU | ASIC |
|---|---|---|---|
| Flexibility after deployment | High, reprogrammable | High, software | None |
| Latency | Very low and deterministic | Higher, batch-oriented | Very low |
| Peak throughput for dense maths | Medium | Very high | Highest |
| Development effort | High (RTL or HLS) | Lower (CUDA, frameworks) | Highest, plus mask cost |
| Best fit | Streaming, custom I/O, evolving algorithms, moderate volume | Large-batch training and inference | Stable algorithms at very high volume |
Common mistakes
- Accelerating a kernel that is only a small share of total run time, so overall speed-up stays small (Amdahl's law).
- Ignoring the cost of moving data over PCIe or the network to and from the FPGA.
- Porting CPU code to high-level synthesis without restructuring it for pipelining and on-chip memory.
- Underestimating timing closure and verification effort at high fabric utilisation.
FAQ
Do I need to write RTL to build an FPGA accelerator?
Not always. High-level synthesis tools such as AMD Vitis HLS and Intel HLS Compiler, and OpenCL or SYCL flows, compile C or C++ into hardware. RTL still gives the most control for demanding designs.
Are FPGAs used for AI?
Yes, mainly for low-latency or low-power inference with custom precision, and in network and edge systems. Large-scale training is dominated by GPUs and custom ASICs.
What is the Open FPGA Stack in the picture?
It is Intel's open-source collection of hardware and software building blocks for FPGA accelerator platforms, intended to give developers a common starting point for board support, drivers and accelerator interfaces.
Summary
FPGA based customised acceleration can offer significant improvements in performance and efficiency for the right applications. By using the reconfigurability and programmability of FPGAs, developers can build solutions tailored to their specific needs, provided they choose workloads that suit streaming hardware and plan for the design effort.
This topic can be elaborated in more detail based on reader interest; tell me in the comments which part you would like covered next.
Your comments will be moderated before it appears here.