Thermal density at the true system boundary

§1 · THE BOUNDARY PROBLEM

The boundary problem

When a vendor quotes watts per litre for an accelerated machine, the litres almost always mean the chassis, the box you can put your hand on. The choice is understandable, since the chassis is the one volume a vendor can measure without knowing anything about the building the machine will eventually stand in. It nonetheless leaves out a large part of the thermal system, and specifically the part that only exists because the machine does.

Consider a 5U server rated at 4 kW with direct liquid cooling. It needs a coolant distribution unit; the CDU needs a facility loop; in a warm climate the loop will usually need a chiller as well. None of this appears in the 5U figure, and yet none of it would be in the building if the server were not.◆

The result is that two machines end up compared on numbers that measure different things. In what follows we propose a boundary that treats every design the same way, apply it to nine systems ranging from a desk-side node to a 100 kW rack, and look at what changes.

§2 · DEFINITION

Definition

We use true system volume: the total occupied volume of every component required to move heat from the die to the outside environment, under the deployment the vendor specifies.◆

Included:

• the compute chassis
• heat exchangers and radiators
• pumps and pump skids, including redundancy where the design specifies it
• coolant distribution units
• chillers, where the design requires them
• distribution piping and valves within the deployment envelope

We exclude building fabric, floor void, and any plant shared with loads beyond the system under test. If anything, leaving shared plant out flatters the conventional designs rather than ours.¹

def thermal_density(design_watts, true_volume_litres):
    # W/L over the whole heat path, not the chassis alone.
    return design_watts / true_volume_litres

thermal_density(3_000,  68.9)   # SuperNode, sealed   E9
thermal_density(4_000, 659.0)   # 5U + CDU + chiller  E3
thermal_density(2_500,  83.2)   # 5U, air cooled      E7

# 43.5, 6.1, 30.0. Published figures round to one
# decimal, so the register reads 43.6 for the first.

Two properties of the definition matter in practice: it is taken at the design point rather than at a short-lived peak, and it gives no credit for moving volume out of the frame.

§3 · METHOD AND SOURCES

Method and sources

Rack and server figures are taken from published vendor documentation and from ASHRAE reference data for the corresponding class of deployment.[1] Where a vendor publishes a range, the design point is used rather than the maximum.

Our own figures come from customer proof of concept deployments, from CFD simulation of the enclosure, and from prototype testing. They are prototype figures; we label them as such in the register and return to the point under limitations.

§4 · RESULTS

Results

Across the table the spread is a factor of 17.6, and nothing in the comparison changes the silicon: the accelerators in the dense rows are the same class as those in the sparse ones.◆◆

It helps to plot the two terms separately, because the difference is not coming from power. The systems here draw between 2.5 and 20 kW, a spread of eight, while their volumes run from 68.9 to 1,199 litres, a spread of more than seventeen.

In the figure the rack deployments cluster near the top of the volume axis at moderate power, while the think nodes sit at the bottom at comparable or higher power. Essentially the whole of the result lives in the denominator.

§5 · SENSITIVITY: WHAT THE COMPARISON RESTS ON

Sensitivity: what the comparison rests on

Any headline ratio here is a comparison between two particular rows, so it matters which two are being compared. Against the 5U server with its CDU and facility loop, a SuperNode occupies a seventh of the volume and delivers 12.7 times the density; against a bare air-cooled 5U server, with no liquid path to plumb anywhere, the advantage narrows to roughly 1.5 times.

§6 · WHY THE LIQUID-COOLED RACK LOSES GROUND

Why the liquid-cooled rack loses ground

The row that repays attention is the 50 kW liquid-cooled rack. Measured at the chassis it looks dramatically denser than the 10 kW air-cooled rack; measured at the true system boundary it comes out slightly less dense, 6.0 W/L against 8.3.◆◆

We do not believe this is an artefact of our definition. A CDU and a chiller occupy real volume and compute nothing, so a density improvement that pushes the rejection stage into a plant room has relocated volume rather than removed it, and only a boundary drawn at the chassis can make the relocation look like progress.

A sealed enclosure avoids the problem in two ways, and it is worth being clear that neither is a matter of thermodynamic efficiency. First, the rejection stage already sits inside the counted volume, so there is no facility loop, CDU, raised floor or CRAC to add on afterwards. Second, air-cooled components need clearance between them for air to pass, a physical requirement that puts a floor under the volume of any air-cooled design regardless of how tightly its components are integrated, whereas direct liquid contact lets them sit against each other.

The compute got denser. The system did not.

§7 · SUSTAINED AGAINST PEAK

Sustained against peak

One further point about Table 1 deserves more weight than it usually gets: every figure in it is taken at the design point rather than at a peak.

A density quoted at a power the machine cannot hold is really a statement about a few seconds of operation. What sets the capacity of a production estate is the power a machine sustains indefinitely, and that is a thermal property before it is an electrical one. Under sustained full load in a think enclosure we measure accelerator die temperature 15 to 20 C above ambient, with no chilled water anywhere in the deployment.◆

The effect reaches further up the stack than cooling. An orchestrator scheduling against a nameplate figure the thermal environment cannot deliver is planning around capacity that does not exist, a mechanism we take up in a companion note on idle accelerator time.

§8 · THE FLOOR IT DOES NOT NEED

The floor it does not need

Volume is the denominator this note has used, because it is the one that can be compared across designs without knowing anything about the room. It is not the denominator anyone is billed for. Estates are leased, built and depreciated by the square metre, and a machine that occupies less floor for the same delivered power is cheaper to house whether or not anyone writes that down.

The relationship needs no new measurement. Floor area is volume over height, so for a system built to a height h in metres, areal density follows from the volumetric figure directly.

The second line is the one that matters commercially. The annual space cost of a delivered watt is inversely proportional to volumetric density: halve the volume a system needs for the same power and you halve the rent that power carries, at any height and at any local rate. We do not publish a figure for C. It is specific to a city, a building and a lease, and a number we invented would be worse than none. Put your own in.

Because the relationship is proportional, the ratios in Table 1 carry across unchanged. At equal build height, the 12.7 between a sealed think node and a datacentre rack is also a factor of 12.7 in the floor area each delivered watt occupies, and therefore in what that floor costs per year.◆

There is a tension in the second denominator that the first does not have, and it should be said rather than left for a reader to find. A higher areal density is cheaper in rent and harder to cool, because the heat per square metre of floor is what a room has to remove. That is precisely the trade the true boundary is meant to make visible: a machine that carries its own heat path can be dense on the floor without making the floor the problem, and a machine that pushes its heat into the room cannot.

§9 · A PROPOSED REPORTING CONVENTION

A proposed reporting convention

The comparison in this note was only possible because we recomputed nine systems ourselves against a boundary we defined. That should not be necessary, and it is not a reasonable thing to ask of a buyer. We therefore propose a reporting convention, and we are adopting it for our own figures first.

Any vendor of accelerated compute, liquid cooled or otherwise, publishing a density figure should publish these five alongside it:

• The boundary itself. A one-line statement of what is inside it and what is not. Everything else here is uninterpretable without it, and it is the item most often missing.
• P_sus, sustained system power in watts. The power the machine holds indefinitely at the stated inlet condition, not a peak and not a nameplate.
• V_true, true system volume in litres. Every component that moves heat from die to environment and exists only because the machine does.
• A_occ, occupied floor area in square metres. The envelope footprint plus the service clearance the design actually requires, plus the footprint of any dedicated plant.
• h, build height in metres. The height the system is deployed to, so a reader can convert between the two densities and check the arithmetic.

From those five, both densities follow, and a reader can compute the two numbers they actually need: the volume a watt occupies and the rent it carries. Note the deliberate asymmetry between the third item and the fourth. Service clearance is excluded from volume, because it is not uniform across designs and including it would import a judgement that cannot be applied consistently. It is included in area, because a lease does not care whether the aisle is uniform.

We are not a standards body and this is not a standard. It is a convention, it costs a vendor five numbers they already know, and it would make the comparison in section 4 unnecessary. Every figure this note publishes is reported against it, and we would like to be held to that.

§10 · LIMITATIONS

Limitations

• Volumes are of the deployed envelope. Service clearance is excluded from them, because it is a real constraint that is not uniform across designs and including it would introduce a judgement we cannot apply consistently. The proposed convention in section 9 asks for it separately, in the area term, for exactly that reason.
• Rack class figures are reference data rather than measurements of a specific unit. A particular deployment varies with the CDU and chiller selected.
• think figures are from proof of concept deployments, CFD and prototype hardware. Production units will differ.
• W/L is a packaging measure and says nothing about efficiency. A machine can be dense and wasteful. It is not a coefficient of performance and should not be read as one.
• The exclusion of shared plant is a simplification. Where plant is dedicated rather than shared, the conventional figures in Table 1 would fall further.
• The areal relationship in section 8 is an identity, not a measurement. It follows from the definitions and it is exact for the envelope. It says nothing about whether a given design can actually be built to a given height, which is a structural and serviceability question we have not addressed.

§11 · WHAT THIS DOES NOT SETTLE

What this does not settle

Thermal density determines whether a machine needs a building around it. Whether the compute inside is actually being used is a different question with different causes, and we address it separately.

§12 · CONCLUSION

Conclusion

The chassis convention is not dishonest. It answers the question a vendor can answer without knowing which building the machine will stand in. What this note establishes is that it is the wrong question for anyone who has to find room, power and cooling for the result, and that the two questions do not have the same answer or even the same ordering.

So we would rather leave a proposal than a table. Any vendor of accelerated compute, liquid cooled or otherwise, publishing a density figure should publish five things with it: the boundary, stated in one line; sustained system power at that boundary; true system volume; occupied floor area including the service clearance the design actually needs; and the build height. Five numbers a vendor already knows. From them, both densities follow, every machine becomes comparable to every other, and nobody has to do what section 4 of this note had to do.

The reason to want the fourth and fifth of those is commercial rather than thermal, and it is the part of this subject that usually goes unsaid. Floor area is not free and it is not a rounding error. It is leased, built and depreciated by the square metre, and section 8 shows that the annual cost of the floor a delivered watt occupies is inversely proportional to volumetric density. Compressing compute into less volume is therefore not only a cooling result. At a fixed build height it is a direct and proportional reduction in the rent that compute carries for as long as it runs, at any local rate. Density is a property of the estate’s cost base, not only of the machine.

It does not follow that liquid cooling is a mistake, or that the ordering here holds at every power level, and section 10 sets out what would change it. What does follow is that the machines which win at this boundary are the ones designed against it, which is the case we are making rather than a result we discovered.

We should be plain that we are not neutral. A sealed liquid-cooled node whose design target is to run in an office or an edge location with no facility chilled water and no electrical retrofit is the thing we build, and it is among the four filings we have pending. The boundary defined in section 2 is the one that design is answerable to, and a convention we propose is a convention we benefit from. That is an argument for reading the method in section 3 and disagreeing with it if it is wrong, not for ignoring the question. The alternative on offer is a number with its plant left off the page.