In Memory Compute

Originally published on LinkedIn

𝗖𝗼𝗺𝗽𝘂𝘁𝗶𝗻𝗴 𝗯𝗿𝗼𝗸𝗲 𝘁𝗵𝗲 𝗼𝗻𝗲 𝗿𝘂𝗹𝗲 𝗶𝘁 𝘄𝗮𝘀 𝗯𝘂𝗶𝗹𝘁 𝗼𝗻. 𝗧𝗵𝗲 𝗿𝘂𝗹𝗲 𝗳𝗿𝗼𝗺 𝟭𝟵𝟰𝟱.
Von Neumann's Memory holds the data, the processor does the math, a bus shuttles between them.

Eighty years of every chip you have ever used obeyed that line.
In-memory compute deletes it.

The math now happens inside the memory array.
I've watched this land in an architecture prposal. 
Someone puts up the energy breakdown for an edge inference workload.
The room goes quiet, because the compute was never the problem.

Here is the number that shouldn't be possible:
A 32-bit integer add costs about 0.1 picojoules.
Fetching those same 32 bits from DRAM costs about 640.
Roughly 6,000x more energy to MOVE the number than to USE it. (Horowitz, ISSCC from 2014 and it has only got worse since.)

You cannot fix that with a better core. 
You cannot fix it with a smaller node. 
Wires don't scale the way transistors do which is why the memory wall kept widening while Moore's Law was still delivering.

So the industry stopped racing toward the finish line.
It moved the finish line.
→ Old way is weights in DRAM 
→ fetch to SRAM 
→ fetch to registers 
→ multiply 
→ write back 
→ repeat, millions of times.

→ New way is weights stay in the array 
→ drive the wordlines 
→ let the bitline do the multiply-accumulate 
→ read out the result.

Ohm's law does the multiply. 
Kirchhoff's law does the accumulate.
No fetch. 
No bus. 
No round trip.
And that one change rewrites the entire stack.

Wordline drivers become the input layer. 
The bitline becomes the accumulator. 
Sense amps become the arithmetic readout. 
ADCs and DACs become the new critical path. 

The compiler becomes the thing that decides whether any of it works at all.

Five layers. 
Not one of them is a CPU.
But here's the real story, and it has nothing to do with picojoules.
In-memory compute quietly changes who owns the architecture.
For fifty years memory was a commodity you bought, and logic was where you competed.

IMC inverts that. 
If the multiply happens on the bitline, then your memory technology, your device variability, your ADC design and your calibration scheme ARE your architecture.

Memory vendors become compute vendors.
Mixed-signal teams become the critical path.
And the moat moves out of the RTL and into the mapping compiler because a crossbar nobody can program is just an expensive resistor.

The array is the Trojan horse. 
The software stack is the takeover.
Stop moving data to the math. 
Move the math to the data.

Nobody has won this yet. 
Variability, ADC overhead, limited precision, write endurance, verification, and a tool flow the industry has not built all still open.

So, your call: which layer takes it? 
A mature SRAM IMC
High-density DRAM IMC
or the ReRAM crossbar?

And the more interesting question. Whoever wins the array, who wins the compiler?