NVIDIA Interview Question: Power Saving in a Branching Pipeline

This NVIDIA power reduction interview question extends a two-stage pipeline to four stages: stage A sends each valid parcel either to stages B and C (35% of the time) or to stage D (65%). The answer is to gate each stage with its own valid bit, so a stage switches only in the cycles when it has valid data for it. With the activity factors and capacitances given, power drops from 50.5 to about 4.5 units, a saving of about 91% before overhead and about 88% with a simple estimate of the clock-gating overhead.

Key takeaways

• Relative dynamic power for each stage is activity factor x capacitance, because supply voltage and clock frequency do not change.
• The environment sends valid data in only 1 cycle in 5, so a stage gated by the valid bit is active 20% of the time.
• Routing information adds a second saving: B and C are needed for only 7% of cycles (0.2 x 0.35) and D for 13% (0.2 x 0.65).
• Original power 50.5, gated power 4.505: a 91.1% saving, or 88.3% after a simple overhead estimate. Gating on the incoming valid bit alone would save only 80%.

The question

Two stages are added to the pipeline discussed in the earlier post to form the circuit shown below.

1.

2. Stage A decides to send the input either to stages B-C or to stage D.
3. 35% of data from stage A is sent to stages B-C while 65% of data from stage A is sent to stage D.
4. All information given in the earlier post is applicable to this pipeline as well.

Extend the power reduction scheme that you developed in the earlier post and apply it to the new pipeline. Calculate how much power will be saved when applying this scheme to the new circuit.

The table in note 1, in text form:

	Stage A	Stage B	Stage C	Stage D
Activity factor	0.4	0.3	0.45	0.5
Capacitance	10	40	60	15

Recap of the earlier two-stage question

The earlier NVIDIA power reduction question had stages B (activity 0.3, capacitance 40) and C (activity 0.45, capacitance 60). A gate has capacitance 1 and a flip-flop 2. A valid bit travels with the data, and the environment sends valid data once every 5 cycles. The scheme must not change the clock, the technology, the interface signals or the behaviour for valid data, but it may add latency.

Scheme: use the valid bit as the clock enable of each stage's registers (in practice, an integrated clock gating cell per stage). In cycles with invalid data the registers do not load, so neither the registers nor the logic after them switch. The valid bit itself keeps flowing through its own always-clocked flop. If the enable has to be ready before the clock edge that would load the data, the valid bit can be registered one cycle earlier, which is why the question allows extra latency.

Assumptions for the numbers: relative power = sum of (activity x capacitance), since voltage and frequency are fixed; the listed activity factors describe the original circuit, in which every stage switches every cycle whether the data is valid or not; a gated stage switches only in the cycles when it loads valid data.

Two-stage circuit	Original	Gated by valid (20% of cycles)
Stage B: 0.3 x 40	12	2.4
Stage C: 0.45 x 60	27	5.4
Total	39	7.8 (80% saving before overhead)

Worked answer for the four-stage pipeline

Step 1: power of the original circuit

Without gating, all four stages switch every cycle. Stage A drives one data bus to both B and D, so both branches toggle for every parcel, wanted or not.

P(original) = 0.4 x 10 + 0.3 x 40 + 0.45 x 60 + 0.5 x 15 = 4 + 12 + 27 + 7.5 = 50.5

Step 2: extend the scheme

• Stage A is enabled by the incoming valid: active in 1 cycle of 5, a fraction of 0.2.
• Stage A already computes the routing decision. It drives valid1 high only for parcels going to B and C, and valid2 only for parcels going to D.
• Stage B is enabled by valid1, stage C by valid1 delayed one stage (the valid bit B passes on), and stage D by valid2.
• B and C are then active for 0.2 x 0.35 = 0.07 of cycles; D for 0.2 x 0.65 = 0.13 of cycles. The shared data bus no longer wakes the branch that is not selected.

Step 3: power with the scheme

Stage	Original (activity x capacitance)	Fraction of cycles active	With gating
A	4	0.20	0.800
B	12	0.07	0.840
C	27	0.07	1.890
D	7.5	0.13	0.975
Total	50.5		4.505

Saving = (50.5 minus 4.505) / 50.5 = 91.1% before overhead.

Step 4: include the overhead

Each gated stage needs one clock gating cell, modelled here as one latch (counted like a flip-flop, capacitance 2) plus one AND gate (capacitance 1): 3 units per stage, 4 stages.

Overhead model	Overhead	Total power	Saving
No overhead (ideal)	0	4.505	91.1%
Gating cells switch only when their stage is enabled (3 x 0.2 + 3 x 0.07 + 3 x 0.07 + 3 x 0.13)	1.41	5.915	88.3%
Pessimistic: gating cells switch every cycle (4 x 3)	12	16.505	67.3%

A realistic answer is close to the middle row: about 88% less power than the original circuit. The same overhead model applied to the two-stage circuit gives 7.8 + 1.2 = 9.0, a saving of about 77%.

Why the routing information matters

If the scheme from the earlier post were applied without using valid1 and valid2 (all four stages enabled by the incoming valid bit), each stage would be active 20% of the time: 0.2 x 50.5 = 10.1, an 80% saving. Using the routing decision lowers this to 4.505, less than half. That extension is the point of the question.

Verilog sketch of the scheme

A register with a load enable is the portable way to describe this; synthesis tools map the enable to an integrated clock gating cell when clock gating is turned on. The + 8'd1 stands in for each stage's real logic.

module stage_a (
  input  wire       clk,
  input  wire       rst,
  input  wire       valid,
  input  wire [7:0] data,
  input  wire       to_bc,     // routing decision for this parcel
  output reg        valid1,    // parcel for stages B and C
  output reg        valid2,    // parcel for stage D
  output reg  [7:0] data_out   // shared bus to B and D
);
  always @(posedge clk)
    if (rst) begin
      valid1 <= 1'b0;
      valid2 <= 1'b0;
    end else begin
      valid1 <= valid &  to_bc;
      valid2 <= valid & ~to_bc;
    end

  // loads only on valid data, so the bus is quiet otherwise
  always @(posedge clk)
    if (valid) data_out <= data + 8'd1;
endmodule

module gated_stage (
  input  wire       clk,
  input  wire       rst,
  input  wire       in_valid,
  input  wire [7:0] in_data,
  output reg        out_valid,
  output reg  [7:0] out_data
);
  always @(posedge clk)
    if (rst) out_valid <= 1'b0;
    else     out_valid <= in_valid;

  always @(posedge clk)
    if (in_valid) out_data <= in_data + 8'd1;
endmodule

Instantiate gated_stage as B (fed by valid1), as C (fed by the out_valid of B) and as D (fed by valid2). Stage D ignores the shared bus whenever valid2 is low, and stage B whenever valid1 is low.

How interviewers extend this question

• "What if the split were 50/50?" B and C active 0.1, D active 0.1: total 0.8 + 3.9 + 0.75 = 5.45, a 89.2% saving before overhead.
• "Is gating the data with AND gates (operand isolation) as good?" It stops the logic toggling but not the register clocks, and it costs one gate per data bit, so clock gating usually wins for wide buses.
• "Where is the glitch risk?" A plain AND of clock and enable can glitch; the latch in the clock gating cell prevents it. The enable must change only while the clock is inactive, a point covered in advantages and disadvantages of gated clocks.
• "What about leakage?" Clock gating saves only dynamic power. Power gating idle stages would save leakage too, at the cost of wake-up time and state retention.

Mistakes to avoid

• Gating every stage with the incoming valid bit and missing the 35/65 split.
• Gating the valid-bit flops themselves, which would lose the protocol.
• Forgetting that stage C's enable is the valid bit delayed through stage B, not valid1 directly.
• Ignoring overhead altogether; interviewers expect at least a statement of what it is.

A short definition of the technique is in the Semiconductor Hub glossary entry on clock gating.

Related NVIDIA interview questions

• NVIDIA true or false questions on simulation, timing, power and test
• NVIDIA pipeline question: total time for 100 input parcels
• NVIDIA power reduction question for a two-stage image pipeline

Frequently asked questions

How do you calculate the power saving from clock gating?

Sum activity x capacitance for each block before and after, scaling each gated block by the fraction of cycles it is enabled, then add the gating cells as overhead. The percentage saving is (original minus new) / original.

Why use the valid bit for clock gating?

The valid bit already marks exactly the cycles in which a stage does useful work, so using it as the enable changes nothing for valid data and needs no new interface signals.

Does clock gating change the function of the pipeline?

Not for valid data. Registers hold stale values during invalid cycles, which is harmless because the valid bit marks them as invalid.

Why does a branching pipeline save more than a straight one?

Each branch handles only part of the traffic, so with per-branch valid bits each branch is idle for longer.