Projects

BERT Encoder on AMD Versal Hard-NoC

Transformer encoder datapath on AMD Versal with 4-head self-attention over parallel AXI-Stream + DMA pipelines — 1.5 GB/s NoC bandwidth at 300 MHz measured in RTL simulation.

GitHub →

BERT Encoder on AMD Versal Hard-NoC

FPGA simulation — results measured in RTL simulation, not on silicon.

Overview

Architected a transformer encoder datapath on AMD Versal with 4-head self-attention over parallel AXI-Stream + DMA pipelines, leveraging the Versal Hard-NoC for DDR–PL data movement. The design emphasizes scalable data movement over raw compute.

Architecture

  • 4-head self-attention implemented over parallel AXI-Stream + DMA pipelines
  • 1.5 GB/s aggregate NoC bandwidth at 300 MHz, measured in RTL simulation
  • Cut memory bandwidth 4x with INT8 quantization of activations and weights
  • Parameterized tokens, embedding size, and head count
  • Enforced constraints for systolic tiling and AXI burst alignment

Key Results

MetricValue
NoC Bandwidth1.5 GB/s
Clock Frequency300 MHz
Attention Heads4 (configurable)
Memory BW Reduction4x (INT8)

Tools

SystemVerilog, Tcl, AMD Versal, Hard-NoC, AXI4, AXI-Stream, DMA, Vivado