Paying for Coherency: Snoop-Filter Sizing and the AMBA Choice for SoC Architects

Hardware coherency is usually presented as a yes or no question about the whole fabric. In practice each master on a SoC gets its own answer, because a CPU cluster, a DMA engine and a neural accelerator share data in very different ways.

This post is a decision guide for architects. It puts rough numbers on what hardware coherency costs in storage, traffic and latency, sets out when IO or software coherency is enough, and shows how AMBA CHI, ACE-Lite and AXI map to each choice.

There are four options, not two

Research from Columbia University sorts accelerator coherence into four modes. The table uses that split. The AMBA column is our own mapping, drawn from Arm's documents.

Mode	Who keeps data consistent	Typical AMBA attachment	Where it tends to fit
Non-coherent DMA	Software flushes and invalidates around each hand-off. Requests bypass the caches	AXI, or Non-shareable transactions on ACE-Lite	Large buffers and long bursts
LLC-coherent DMA	Hardware against the last-level cache. Software still flushes the CPUs' private caches	A bridge from a plain interface. We found no dedicated AMBA interface for this mode	Data sets that fit in the last-level cache
Coherent DMA, also called IO coherency	Hardware. No flushes needed, and the accelerator has no private cache to snoop	ACE-Lite or ACE5-Lite, reaching an I/O coherent request node on a CHI mesh	Small, frequent shared buffers such as descriptors and packet headers
Fully coherent	Hardware, in the same way as for a CPU, including the device's own cache	CHI as a fully coherent request node (RN-F), or full ACE	CPU clusters and devices that share fine-grained data with them

Only the last row adds a cache that the fabric must track and snoop, so it is the row to keep short.

What hardware coherency actually costs

Storage

Something has to record which caches hold which lines. The cheapest place is the tag array of an inclusive shared cache. Martin, Hill and Sorin estimate that a 16-core system adds 16 bits to each 64-byte block, about 3%, and argue that hierarchy keeps the cost low at far higher core counts.

That result depends on inclusion. A non-inclusive shared cache needs a separate directory holding its own tags, and the same paper gives the overhead as (R × D) / (1 + S2/S1). Here S1 is total private cache, S2 is shared cache, R is directory entries per private block and D is entry size relative to a block and its tag. With R of 2 and 64-bit entries, the paper reports 2.5% when the shared cache is eight times the private total, rising to 11% when the two are equal. So as the shared cache shrinks relative to the private caches, the directory takes a larger share.

Arm's own sizing guidance points the same way. The CMN-650 manual calls a total snoop filter size of twice the exclusive cache of all fully coherent request nodes optimal. It gives the example of 64 MB for 32 MB of cache. The older CCI-550 manual recommends a configured size of 0.75 to 1 times that cache and then says the filter stores twice as many tags as configured, which by our reading comes to 1.5 to 2 times in tags. These vendor recommendations roughly agree.

Traffic

Broadcast snooping scales badly. Arm's ACE white paper puts traffic in proportion to N × (N - 1) for N coherent masters. A directory brings the best case down to order 2N, if data is shared by only two masters, and gives no help in the worst case, where everything is shared by everyone.

With exact tracking, the overhead per miss is small and does not depend on core count. Martin, Hill and Sorin calculate 96 bytes per miss with coherence against 80 without for clean blocks, a 20% overhead, and 160 against 152 for dirty blocks, or 5%. Their no-coherence baseline is hypothetical, because it assumes software knows when data is stale at no cost.

Latency

The same paper finds that coherence adds no latency to private cache hits or off-chip misses, and nothing appreciable to shared cache hits on blocks that no other core holds. The expensive case is the indirect miss, where the home node must contact another cache. That costs 1.5 to 2 times a non-coherent miss, depending on cache and interconnect latency. A separate directory also has its own lookup, and Arm's white paper notes the added latency of consulting it before each shared transaction.

A worked example: sizing the filter

Here is a snoop filter for a mid-sized cluster. Every input is an illustrative assumption, not data from any product, apart from the twice-capacity ratio taken from the CMN-650 manual and the message sizes from Martin, Hill and Sorin.

Assumption	Value
Fully coherent requesters	32, each with a 1 MiB private cache
Line size and physical address	64 bytes, 48 bits
Filter organisation	16-way set associative, 3 state bits per entry, SECDED protection

1. Lines to track: 32 MiB / 64 bytes = 524,288.
2. Entries at twice capacity: 1,048,576.
3. Sets: 1,048,576 / 16 = 65,536, which takes 16 index bits, whether those bits select a slice or a set.
4. Tag: 48 - 6 offset bits - 16 index bits = 26 bits.
5. Payload: 26 tag + 3 state + 32 sharer bits = 61 bits. SECDED on 61 bits adds 8, so an entry is 69 bits.
6. Storage: 1,048,576 × 69 bits = 9,043,968 bytes, or 8.625 MiB.

Two ways to cut this are a smaller filter and coarser sharer tracking. The CMN-650 supports the second with a clustered mode in which one vector bit stands for two, four or eight request nodes, and the home node snoops the whole cluster on a shared line.

Option	Entries	Bits per entry	Storage	Share of 32 MiB private data
A: twice capacity, exact sharers	1,048,576	69	8.625 MiB	27.0%
B: twice capacity, clusters of four (8 vector bits)	1,048,576	44	5.5 MiB	17.2%
C: equal capacity, exact sharers	524,288	70	4.375 MiB	13.7%

Those percentages look far larger than the paper's 2.5% because they compare against private data alone. If you put the 69-bit entry into the paper's formula, it gives 2.7% of total cache storage with a shared cache eight times the private total and 12.3% with equal sizes. State your denominator when you quote one, and convert bits to area with your own compiled SRAM data.

Option C looks like the cheap win, but it risks more conflict evictions. When a filter entry is evicted, the lines it tracked have to be pulled out of the caches. The CCI-550 manual calls this back-invalidation and expects it to be rare at the recommended size. A core can lose a line it was still using, for reasons unrelated to sharing.

Now the traffic side. Assume each requester sends 20 million misses per second into the coherent domain, or 640 million in total, all to clean blocks, and that 10% need one peer snooped. Control messages are 8 bytes and the base miss is 80 bytes. We count one snoop and one response per target, and leave out forwarded data.

Scheme	Bytes per miss	Fabric load	Overhead	Snoops per cache, per second
No hardware coherence (baseline)	80	51.2 GB/s	none	none
Filter, exact sharers	80 + 16 + 1.6 = 97.6	62.5 GB/s	22%	2 million
Filter, clusters of four	80 + 16 + 6.4 = 102.4	65.5 GB/s	28%	8 million
Broadcast, no filter	80 + 496 = 576	368.6 GB/s	620%	620 million

The 16 bytes are the eviction notice and its acknowledgement, which a filter needs to stay accurate. The 496 bytes are 31 snoops and 31 responses at 8 bytes each. On these assumptions, clustering saves 3.125 MiB of storage for 4.8 more bytes per miss. Dropping the filter altogether is a far worse trade. Change the 10% to match your workload, because heavier sharing raises both filtered rows, and the clustered row most of all.

When IO or software coherency is enough

Software coherency has real costs, though the published figures are old and come from a vendor. A 2013 Arm blog post says its benchmarking found a networking application processing every packet header might spend more than a third of CPU cycles on cache maintenance. Arm's ACE white paper measured flushes on the order of 100 µs on a dual-core Cortex-A9 with a 256 KB L2, and extrapolated to perhaps 5 to 6% of CPU time when flushing a 4 MB L2 every frame at 60 frames per second. It calls that a rough order of magnitude. Stale-data bugs are the other cost. They are hard to reproduce, and Arm's paper says they eat into the time-to-market window.

IO coherency removes both for devices without caches of shared data. The fabric snoops CPU caches on the device's behalf. On a snoop hit, the white paper notes, the device also avoids a memory read.

None of this makes coherence the right answer for large data. Two FPGA studies on the open-source ESP platform show why, and their trends travel better than their numbers:

• Giri, Mantovani and Carloni (2018) found LLC-coherent accelerators ranging from 0.5 to 4 times the speed of non-coherent ones. They did best when the data fitted in the last-level cache. Beyond that, evictions generally pulled them below non-coherent DMA, and the fully coherent model won only on the smallest data sets.
• Zuckerman and colleagues (2021) compared all four modes on roughly 16 KB, 256 KB and 4 MB workloads, with a 1 MB last-level cache, and found no fixed winner. One autoencoder went from slowest in non-coherent mode at the small size to at least three times faster than the rest at the large size.
• Contention changes the picture again. With 12 accelerators running against one, that study saw execution time grow up to 2.4 times in non-coherent mode and 8 times in coherent DMA, so test with the number you will really run together.

If a job's buffers are large relative to the shared cache, arrive in long bursts and change hands at clear boundaries, then non-coherent DMA with software maintenance is a sound choice. The flush cost becomes small next to the job, and the caches stay free for the CPUs. If hand-offs are frequent and small, the flush overhead dominates and IO coherency should pay for itself.

How AMBA maps to the choice

CHI is a protocol and does not dictate a network shape. Arm's guide says it defines the components of a network but not the topology, and lists ring, mesh and crossbar as options. For the building blocks underneath, see our introduction to network-on-chip IP.

Interface	Coherency role	Limits
AXI	None on its own. Transactions carry no coherency meaning	Consistency with CPU caches is left to software or to a bridge
ACE-Lite, ACE5-Lite, and the DVM variants	AXI plus extra signals on the read and write address channels. Can mark transactions as shareable, issue barriers and broadcast cache maintenance. The DVM variants also carry distributed virtual memory messages	No snoop channels, so the master cannot hold hardware coherent data
ACE, ACE5	Full two-way coherency for cached masters	Snoop channels and an interconnect able to serve them
CHI, RN-F	Coherent caches that accept and respond to snoops. CHI adds support for snoop filters and directory-based systems	Each one adds to the tracking load at the home nodes

Fully coherent home nodes (HN-F) order requests to coherent memory and issue snoops. Non-coherent home nodes (HN-I) order requests to the I/O subsystem. In the CMN-650, RN-I bridges connect ACE-Lite masters, and RN-D bridges connect ACE-Lite-with-DVM masters. Both act only for masters without hardware coherent caches, and neither can be snooped.

That suggests a hybrid. Keep CPUs and true cache-holding devices on the coherent mesh, bring IO-coherent masters in through request-node bridges, and put the rest on a non-coherent tier. The CMN-650 manual describes adding hierarchy with an interconnect such as the NIC-450 to reach more devices.

Plan DVM with these ports. A device with its own address translation has to hear about page table changes, which is why an accelerator with no data cache may still need the DVM variant. Our guide to virtual memory and the TLB covers why those cached translations go stale.

A decision sequence

Work through these for each master:

1. Does it hold modified shared data in its own cache? If another master must see that data without software help, it needs a fully coherent port, either CHI or ACE. If not, drop to the next question.
2. How large are its buffers against the last-level cache? Much larger with long bursts favours non-coherent DMA. Small favours IO or LLC coherence.
3. How often does data change hands with the CPU? Frequent, small hand-offs favour IO coherency. Measure flush time on your real driver rather than estimating it.
4. Does it translate addresses? If so, plan for DVM or an equivalent invalidation path.
5. Can the driver choose per job? Both FPGA studies favour run-time selection, and the later paper reports a 38% average speedup and 66% fewer off-chip accesses against design-time approaches, on its own prototypes. A second path costs wiring and verification, so weigh that against the gain.
6. Then size the filter. Count RN-F capacity only, repeat the worked example, and choose the sharer encoding against real sharing data. After bring-up, watch the performance counters: the CCI-550 exposes back-invalidations and stalls from filter sets with all ways in use, and its manual notes the RAM might be too small in that case.

This is a trade-off call like any other, and our post on balancing performance, power and area sets out a way of framing one. Coherency decisions set the ports, the verification plan and the driver model, so they belong early. Our SoC lifecycle on The Semiconductor Hub shows where architecture decisions sit in a programme.

Sources

• Martin, Hill and Sorin, "Why On-Chip Cache Coherence Is Here to Stay", Communications of the ACM, vol. 55, no. 7, July 2012.
• Giri, Mantovani and Carloni, "NoC-Based Support of Heterogeneous Cache-Coherence Models for Accelerators", NOCS 2018.
• Zuckerman, Giri, Kwon, Mantovani and Carloni, "Cohmeleon: Learning-Based Orchestration of Accelerator Coherence in Heterogeneous SoCs", MICRO 2021, arXiv:2109.06382, September 2021.
• Arm, "AMBA AXI and ACE Protocol Specification", IHI 0022H.c, January 2021.
• Arm, "Learn the architecture: Introducing AMBA CHI", 102407 version 1.2, 23 June 2025.
• Arm, "Neoverse CMN-650 Coherent Mesh Network Technical Reference Manual", r2p0 issue 04, 11 August 2021 (vendor material).
• Arm, "CoreLink CCI-550 Cache Coherent Interconnect Technical Reference Manual", r1p0, 100282_0100_01_en, copyright 2015 to 2018 (vendor material).
• Stevens, "Introduction to AMBA 4 ACE and big.LITTLE Processing Technology", Arm white paper, 6 June 2011, updated 29 July 2013 (vendor material).
• Parris, "Extended System Coherency: Part 1, Cache Coherency Fundamentals", Arm Community blog, 3 December 2013 (vendor material).

The worked example uses illustrative numbers. Vendor figures are the companies' own claims and recommendations, and the accelerator results come from academic FPGA prototypes rather than production silicon.