Glossary

Open AI Infrastructure Glossary

110 terms from the rack to the grid, in plain English.

The language of AI infrastructure, from 800 VDC and cold plates to interconnection queues and cost per token. Written by the Digital Infrastructure World 2027 editorial team for engineers and for the finance, procurement and business teams who work with them. Terms link to the track that covers them.

1 MW rack

Also: megawatt rackRack

A single equipment rack drawing about one megawatt, roughly ten times today's densest AI racks. The industry is redesigning rack power and cooling to make it possible.

Google has said machine learning will need more than 500 kW per IT rack before 2030. NVIDIA has said it is moving to 800 VDC to support 1 MW racks starting in 2027, and Google, Meta and Microsoft are working at OCP on a +/-400 VDC power rack called Mt Diablo.

Related: 800 VDC, Mt Diablo, Sidecar, Cold plate

51.2T and 102.4T switches

Also: switch ASIC, TomahawkCluster

The two latest generations of Ethernet switch chip, moving 51.2 or 102.4 terabits of data per second through a single chip.

Broadcom announced the 51.2 Tbps Tomahawk 5 in August 2022 and began shipping the 102.4 Tbps Tomahawk 6 in June 2025, with 200G SerDes and a co-packaged optics version planned. Each doubling lets a network connect more GPUs with fewer switch layers.

Related: SerDes, CPO, Scale-out, Clos network

800 VDC

Also: 800V HVDCRack

Distributing power to AI racks as 800 volts direct current instead of AC or 54-volt DC, so the same copper can carry far more power.

NVIDIA described the architecture in May 2025 and said full-scale production would coincide with its Kyber rack systems in 2027. It says 800 V busways carry 85% more power through the same conductor size and cut copper by 45% compared with 415 VAC distribution.

800G and 1.6T optics

Also: optical transceiversCluster

Pluggable optical transceivers that send 800 gigabits or 1.6 terabits per second over fiber, linking switches and servers in AI clusters.

AI clusters use far more optics per GPU than ordinary cloud networks, so transceivers are a large share of network cost and power. 1.6T modules pair with 200G-per-lane switch chips.

Accelerator

Also: XPUGeneral

A chip designed to run AI math much faster than a general-purpose CPU. GPUs are the most common type; custom chips from cloud providers are another.

Engineers often write XPU to cover GPUs, TPUs and other custom AI chips in one word. Accelerators drive most of the power, cooling and networking demands in an AI data center.

Related: GPU, TDP, HBM, OAM

AI factory

General

A data center built mainly to train and run AI models, where the output being measured is AI work, usually counted in tokens.

The term treats the facility as a production line: power goes in, tokens come out. It is used most by NVIDIA and its partners, and it usually implies dense liquid-cooled racks, a dedicated high-speed network and power planned at the campus level.

Air cooling

Data Center

Removing server heat with fans and chilled room air, the traditional data center approach.

Many existing facilities were designed for racks drawing well under 20 kW. Air alone cannot handle most AI accelerator racks, so new halls combine air for general equipment with liquid for accelerators.

Back-end network

Also: GPU fabric, compute fabricCluster

The dedicated network that connects accelerators to each other for AI training and inference, separate from the network serving users and storage.

The back-end carries the heavy GPU-to-GPU traffic and is built for high bandwidth and low loss, using InfiniBand or Ethernet with RoCE or Ultra Ethernet. The front-end network handles user traffic, storage and management.

Battery backup unit

Also: BBURack

A battery pack that sits inside the rack and keeps servers running for a short time if the incoming power drops.

In Open Rack designs, BBU shelves sit alongside the power shelf on the rack's DC busbar, which reduces reliance on a large central UPS. Their job is to bridge brief outages or hand over to generators, not to run the rack for long periods.

Battery energy storage system

Also: BESS, grid batteryEnergy

Large banks of batteries, usually lithium-ion, that store electricity and release it when needed, sized in megawatts and megawatt-hours.

Data centers use BESS to smooth AI load swings, provide backup, shift grid draw away from peak hours or bridge power until a grid connection arrives. In the US interconnection queue, storage alone made up about 890 GW at the end of 2024, according to LBNL.

Behind-the-meter

Also: BTM, on-site generationEnergy

Power generation or storage located on the customer's side of the utility meter, supplying a data center directly rather than through the public grid.

Examples are on-site gas turbines, engines, fuel cells or batteries. It can get a site running sooner than a grid connection, but brings fuel, permitting and reliability risk to the owner.

BoW

Also: Bunch of WiresRack

Bunch of Wires: an open OCP specification for connecting chiplets inside one package, so chiplets from different makers can work together.

BoW came out of OCP's Open Domain-Specific Architecture work. OCP has published both a physical-layer spec and a transaction and link layer spec for BoW interfaces.

Related: Chiplet, HBM, Open Compute Project

Bridge power

Also: interim powerEnergy

Temporary on-site generation that runs a data center until its permanent grid connection is ready.

Bridge power is usually gas turbines, reciprocating engines, fuel cells or batteries. It lets operators open a site years earlier than the utility can deliver, then switch to backup or grid support duty later.

Busbar

Also: bus barRack

A solid metal bar, usually copper, that carries power up the back of a rack so each server can plug straight into it.

Open Rack uses a pair of busbars at the rear fed with about 48 V DC from a power shelf. At AI power levels low voltage needs very large busbars: NVIDIA says a 1 MW rack on 54 VDC could need up to 200 kg of copper busbar.

Related: Open Rack v3, Power shelf, 800 VDC

Capacity factor

Energy

How much electricity a power plant actually produces in a year compared with what it would produce running flat out all year.

EIA data for 2024 show US capacity factors of 90.8% for nuclear, 34.3% for wind and 23.2% for solar PV. A data center running near full load all year needs supply that matches that profile, often through a mix of sources and storage.

Related: PPA, SMR, Firm power

Capacity market

Energy

A market where grid operators pay power plants to be available at future peak times, not only for the energy they produce.

PJM runs one of the main US capacity markets. Fast-growing data center demand has become a major factor in capacity prices, which flow into everyone's electricity bills.

Related: PJM, ISO and RTO, Firm power

Checkpointing

Cluster

Saving a snapshot of a training job at intervals so it can restart from that point after a failure, instead of starting over.

In a run on thousands of accelerators, some hardware will fail during the job, so checkpoint frequency and speed directly affect goodput. Writing checkpoints puts heavy bursts of load on storage and the network.

Related: Goodput, Training, MFU

Chiller

Data Center

A large refrigeration machine that produces cold water for a building's cooling system.

Chillers are often the largest power draw after the IT load. Liquid-cooled AI systems can run on warmer water, which lets more sites rely on dry coolers and run chillers less.

Related: Facility water system, PUE

Chiplet

Rack

A small chip die designed to be combined with others in one package, instead of building one large chip. It improves yields and lets makers mix parts.

Modern AI accelerators combine compute chiplets with stacked HBM memory on a shared base. Open die-to-die standards such as OCP's Bunch of Wires and UCIe aim to let chiplets from different suppliers connect.

Related: BoW, HBM, Accelerator

Clos network

Also: fat tree, leaf-spineCluster

A network layout with layers of switches where every lower switch connects to every upper switch, giving many equal paths between any two servers.

Most data center and AI back-end networks are Clos designs, often called leaf-spine or fat-tree. Higher-radix switch chips mean fewer layers, which cuts cost, power and latency.

Co-located generation

Also: co-locationEnergy

Placing a data center at or next to an existing power plant, often nuclear, so it can take power directly from that plant.

Co-location can avoid long transmission waits, but grid operators and regulators have questioned who pays for grid services such sites still rely on. These arrangements have been the subject of FERC proceedings since 2024.

Related: Behind-the-meter, FERC, PJM

Cold plate

Also: direct-to-chipRack

A metal plate mounted directly on a hot chip with liquid flowing through it, carrying heat away far better than air.

Cold plates are the most common form of direct liquid cooling for AI racks today. OCP defines the technology cooling system as everything from the cold plate to the CDU, including tubing, manifolds and quick disconnects.

Collective

Also: collective communication, all-reduceCluster

A communication step where many accelerators exchange or combine data at once, such as averaging results across all GPUs in training.

Common collectives are all-reduce, all-gather and all-to-all. The slowest link sets the pace for the whole collective, so network tail latency matters more than average speed.

Colocation

Also: coloGeneral

Renting space, power and cooling in a data center owned by someone else, rather than building your own.

The colocation provider runs the building, power and cooling; the tenant owns and runs the IT equipment. Contracts are usually priced by power (kW or MW) as much as by floor space, and AI tenants increasingly ask for liquid cooling to be in place at handover.

Combined cycle

Also: CCGT, NGCCEnergy

A gas power plant that uses a gas turbine and then captures its exhaust heat to drive a steam turbine, producing more power from the same fuel.

Combined-cycle plants are the most efficient large fossil plants and the backbone of US power supply. They take longer to build than simple-cycle turbines or engines.

Commissioning

Also: CxData Center

Testing a new data center's power, cooling and controls under realistic load before handover, to prove everything works as designed.

It is often described in five levels, from factory testing of equipment through to integrated systems testing, where the whole site is run with load banks and failures are simulated. Liquid cooling adds flushing, pressure and coolant quality checks.

Congestion control

Cluster

Methods that stop network switches and links from being overloaded, so data keeps moving without loss or long delays.

AI traffic sends huge synchronized bursts that can overwhelm standard Ethernet. Ultra Ethernet, RoCE deployments and InfiniBand each use their own mechanisms to manage this.

Coolant distribution unit

Also: CDURack

A unit with pumps, a heat exchanger and controls that moves coolant to the racks and keeps it separate from the building's water loop.

CDUs sit in the row or inside the rack and isolate the clean rack coolant loop from facility water. Google reports about 99.999% fleet-wide availability for its in-row CDU design since 2020 and said it would contribute its fifth generation to OCP.

Cost per token

Cluster

The all-in cost to generate one token of AI output, used to compare hardware, software and data center choices on a common basis.

It folds together hardware cost, power, cooling, utilization and software efficiency. It is usually quoted per million tokens.

CPO

Also: co-packaged opticsCluster

Co-packaged optics: putting the optical components in the same package as the switch chip, instead of in separate plug-in modules.

Shortening the electrical path cuts power per bit. Broadcom built CPO versions of Tomahawk 4 and 5 and plans one for Tomahawk 6. The trade-off is serviceability: a failed laser cannot simply be swapped like a pluggable module.

Data center load growth

Also: large load growthEnergy

The rapid rise in electricity demand from data centers, driven largely by AI, that utilities and grid operators now plan around.

A Lawrence Berkeley National Laboratory report for DOE estimated US data centers used 176 TWh in 2023, about 4.4% of US electricity, and projected 6.7% to 12% by 2028. The range reflects uncertainty in AI chip shipments and efficiency.

DC-MHS

Rack

Datacenter Modular Hardware System: an OCP family of specs that standardizes server building blocks so motherboards and modules from different vendors fit common chassis.

DC-MHS defines items such as host processor module sizes and standard modules for management and I/O. AMD says its Helios rack integrates DC-MHS alongside UALink and Ultra Ethernet.

DCIM

Data Center

Data center infrastructure management: software that monitors and manages a facility's power, cooling, space and assets in one place.

DCIM tools collect readings from power meters, cooling plant and sensors to track capacity and spot problems. Liquid cooling adds new things to watch, such as coolant flow, pressure and leak detection.

Related: PUE, Commissioning, White space

Demand response

Also: DREnergy

Programs that pay large electricity users to cut or shift their use when the grid is under stress.

Data centers can join by running on batteries or generators, or by slowing non-urgent computing. Participation may come with faster grid access or lower charges.

Direct liquid cooling

Also: DLC, liquid coolingRack

Cooling where liquid is brought right to the hottest chips, usually through cold plates, instead of relying on fans and room air.

AI racks now routinely exceed what air can remove, so most new AI deployments use direct liquid cooling for the accelerators while fans still handle smaller parts. It needs a CDU and a technology cooling loop in the building.

DPU

Also: SmartNIC, SuperNICCluster

Data processing unit: a programmable network card with its own processors that handles networking, storage and security tasks so the main CPUs and GPUs do not have to.

DPUs and SuperNICs sit at the server's network edge and can enforce isolation between tenants, which matters for GPU clouds. Examples include NVIDIA BlueField and AMD Pensando.

ERCOT

Energy

Electric Reliability Council of Texas: the operator of the grid serving most of Texas, a major destination for new data centers.

ERCOT serves about 90% of Texas electric load and more than 26 million customers. Its grid is largely separate from the rest of the US, and it is overseen by the Public Utility Commission of Texas.

ESUN

Rack

Ethernet for Scale-Up Networking: an OCP workstream to make standard Ethernet work for the tightly linked accelerator groups inside AI racks.

OCP launched ESUN at its October 2025 Global Summit with founding members including AMD, Arista, Arm, Broadcom, Cisco, HPE Juniper, Marvell, Meta, Microsoft and NVIDIA. It covers the switch side, such as framing, lossless transfer and error handling, and coordinates with the Ultra Ethernet Consortium and IEEE 802.3.

Related: Scale-up, UALink, NVLink, Ultra Ethernet

Facility water

Also: primary loopData Center

The water the building supplies to cooling equipment, as opposed to the coolant that flows through the servers.

Many AI rack vendors specify the facility water temperature and flow their systems need. Warmer allowed temperatures cut chiller use and make heat reuse easier.

Facility water system

Also: FWSData Center

The building's main water loop that carries heat from the CDUs and cooling units out to chillers, cooling towers or dry coolers.

FWS water does not reach the chips: the CDU hands heat from the rack's technology cooling system to the facility loop. Supply temperature and water quality for the FWS are key design choices for liquid-cooled halls.

FERC

Also: Federal Energy Regulatory CommissionEnergy

Federal Energy Regulatory Commission: the US agency that regulates interstate electricity transmission and wholesale power markets.

FERC sets the rules for connecting generators to the grid and oversees regional grid operators. Its Order No. 2023, issued July 28, 2023, moved interconnection to cluster studies with stricter readiness deposits.

Firm power

Also: baseload, dispatchable powerEnergy

Electricity that is available whenever it is needed, regardless of weather, as opposed to variable sources like wind and solar.

Nuclear, gas, geothermal and hydro with storage are typical firm sources. AI data centers run near full load around the clock, so buyers often pair variable contracts with firm supply.

Related: Capacity factor, PPA, SMR

Front-end network

Cluster

The general-purpose network that connects AI servers to users, storage and the rest of the data center.

It is usually standard Ethernet and is kept separate from the back-end network that carries GPU-to-GPU traffic. Training data and checkpoints often travel over it.

Fuel cell

Also: SOFCEnergy

A device that makes electricity from a chemical reaction, usually with natural gas or hydrogen, without burning the fuel.

Solid oxide fuel cells running on natural gas are being deployed for on-site data center power. They are modular and quiet with low local air emissions, but cost more per kW than engines or turbines.

Gas turbine

Also: combustion turbine, CTEnergy

A jet-engine-style generator that burns natural gas to spin a turbine and make electricity. It can be used alone or in a combined-cycle plant.

Simple-cycle turbines start fast and suit peaking, backup and bridge power. Order backlogs for large turbines grew sharply with AI demand, extending delivery times.

Generator

Also: backup generator, gensetData Center

On-site engines, usually diesel, that start automatically to power the data center if the utility supply fails.

The UPS or rack batteries carry the load for the seconds it takes generators to start. Air permits often limit how many hours backup generators may run each year, which constrains using them for grid support.

Goodput

Cluster

The share of a cluster's time spent doing useful work that counts toward the result, after losses to failures, restarts and waiting.

Throughput counts everything processed; goodput excludes work that had to be redone. Faster checkpointing, quicker failure recovery and a reliable network all raise goodput.

Related: Checkpointing, MFU, Training

GPU

Also: graphics processing unitGeneral

Graphics processing unit. A chip with thousands of small cores that run in parallel, now the main engine for training and running AI models.

AI GPUs are paired with stacked high-bandwidth memory and linked to each other by dedicated scale-up links. A single modern data center GPU draws around a kilowatt or more, which is why AI racks need liquid cooling.

Related: Accelerator, HBM, NVLink, TDP

HBM

Also: high-bandwidth memory, HBM4Rack

High-bandwidth memory: memory chips stacked vertically and placed right beside the accelerator, giving far faster data access than ordinary server memory.

JEDEC published the HBM4 standard in April 2025 with a 2,048-bit interface, up to 2 TB/s of bandwidth and stacks of up to 16 dies holding up to 64 GB. HBM supply and cost are a major constraint on accelerator output.

Related: GPU, Chiplet, Accelerator

Heat reuse

Also: waste heat recoveryData Center

Capturing waste heat from a data center and using it elsewhere, for example to heat homes, offices or greenhouses.

Liquid cooling returns heat at higher temperatures than air, which makes reuse more practical. OCP runs a heat reuse workstream that publishes reference designs, economics and a global map of projects.

Hot aisle containment

Also: containmentData Center

Enclosing the hot exhaust side of server rows so hot and cold air do not mix, making air cooling more efficient.

Racks face each other in alternating hot and cold aisles; containment closes off one aisle with doors and roof panels. It remains standard for the air-cooled share of an AI hall.

Hyperscaler

General

One of the largest cloud and internet companies, which build and run their own data centers at very large scale.

The usual list is Amazon, Microsoft, Google and Meta, sometimes with Oracle and large Chinese providers. Hyperscalers founded or lead much of the open hardware work at the Open Compute Project.

Immersion cooling

Also: immersionData Center

Submerging servers in a tank of non-conductive liquid that absorbs heat directly from every component.

Single-phase immersion keeps the fluid liquid; two-phase lets it boil. It removes the need for server fans but changes how equipment is serviced, and OCP has a dedicated immersion sub-project.

Inference

General

Running a trained AI model to answer a request, such as producing a chatbot reply. It is the day-to-day use of a model, as opposed to building it.

Inference is measured in tokens per second and cost per token, and is sensitive to latency in a way training is not. As AI products scale, inference becomes a growing share of total AI compute.

InfiniBand

Also: IBCluster

A high-speed, low-latency networking standard long used in supercomputers and now widely used to connect GPUs in AI training clusters.

InfiniBand has lossless transport and RDMA built in. NVIDIA is the dominant supplier after acquiring Mellanox in 2020. Ethernet-based options such as RoCE and Ultra Ethernet compete with it for AI back-end networks.

Interconnection queue

Also: grid queue, queueEnergy

The waiting list of power plants and large loads asking to connect to the grid, each needing studies and upgrades before approval.

LBNL counted about 1,400 GW of generation and 890 GW of storage in US queues at the end of 2024. The median time from request to operation rose to over four years for projects built in 2018 to 2024, and only 13% of capacity requested from 2000 to 2019 was built by 2024.

ISO and RTO

Also: grid operator, RTOEnergy

Independent system operators and regional transmission organizations: the bodies that run the high-voltage grid and wholesale power markets across a region.

Major US examples are PJM, ERCOT, MISO, CAISO, SPP, NYISO and ISO New England. Most are regulated by FERC; ERCOT is regulated in Texas.

Related: PJM, ERCOT, FERC, Capacity market

Large load interconnection

Also: large loadEnergy

The process for connecting a very large electricity user, such as an AI data center, to the transmission grid.

Rules for large loads are being rewritten across US grids because data center requests now rival big power plants in size. New rules often add deposits, study timelines and requirements for loads to ride through grid faults.

Load flexibility

Also: flexible loadEnergy

A data center's ability to reduce or shift its power use on request, which can help it connect to a congested grid sooner.

Options include switching to on-site batteries or generation and pausing or slowing non-urgent AI work. Grid operators are testing whether flexible loads can be connected with fewer grid upgrades.

LPO

Also: linear pluggable optics, LPO MSACluster

Linear pluggable optics: optical modules that drop the power-hungry signal-processing chip and rely on the switch chip instead, saving power while staying pluggable.

Twelve companies formed the LPO multi-source agreement in March 2024 to write common specifications, and reported multi-vendor interoperability over 400G and 800G links later that year. LPO sits between standard pluggables and co-packaged optics.

Related: CPO, 800G and 1.6T optics, SerDes

Medium voltage

Also: MVData Center

Power at thousands of volts, between the high-voltage grid and the low voltages used by equipment. Large data centers take power from the utility at this level.

A common example is 13.8 kV AC, which NVIDIA's 800 VDC design converts at the data center perimeter. Bringing medium voltage closer to the IT load cuts losses and copper.

MFU

Also: model FLOPs utilizationCluster

Model FLOPs utilization: the share of an accelerator's peak math capacity a training job puts to useful work.

The metric was set out by Google in its 2022 PaLM paper. Real jobs run well below 100% because of communication, memory limits and stalls, so MFU is a common yardstick for how well a cluster and its software are tuned.

Related: Goodput, Training, Collective

Mt Diablo

Also: DiabloRack

An OCP rack design that moves power conversion into a separate rack beside the compute rack and feeds it with +/-400 volts DC.

Google, Meta and Microsoft are standardizing it at OCP, with a 0.5 draft released for feedback in May 2025. Google says the 400 V choice lets it reuse the electric vehicle supply chain, and that the sidecar approach improves end-to-end efficiency by about 3%.

MW of critical IT

Also: IT load, critical loadData Center

The power available to the IT equipment itself, excluding cooling and other overheads. It is the standard way to size and price data center capacity.

A site advertised at 100 MW critical IT draws more from the grid once cooling and losses are added, in proportion to its PUE. Leases and build costs are usually quoted per MW of critical IT.

Related: PUE, White space, Colocation

Neocloud

Also: GPU cloudGeneral

A cloud provider built specifically to rent out GPU capacity for AI, rather than a general-purpose cloud offering many services.

Neoclouds sell GPU time by the hour or on multi-year contracts, often to AI labs and enterprises. They usually lease capacity in colocation or purpose-built sites rather than owning a large estate of their own.

Related: Hyperscaler, Colocation, GPU, AI factory

NVL72

Also: GB200 NVL72Rack

NVIDIA's GB200 NVL72: a liquid-cooled rack that links 72 Blackwell GPUs and 36 Grace CPUs so they act like one large accelerator.

The 72 GPUs share one NVLink domain with 130 TB/s of GPU-to-GPU bandwidth, according to NVIDIA. Racks of this class draw well over 100 kW, which pushed liquid cooling into the mainstream.

NVLink

Rack

NVIDIA's own high-speed link that connects its GPUs to each other, much faster than standard networking, inside a server or rack.

Fifth-generation NVLink on Blackwell GPUs gives each GPU 1.8 TB/s of bandwidth. NVLink switches extend this across a whole rack, such as the 72 GPUs in an NVL72.

Related: Scale-up, NVL72, UALink, ESUN

OAM

Also: OCP Accelerator ModuleRack

OCP Accelerator Module: an open standard card format for AI accelerators so different vendors' chips fit the same server baseboard.

Baidu, Facebook and Microsoft defined the OAM specification in 2019 under OCP's Open Accelerator Infrastructure work. It sets the module size, connectors, power and cooling interface.

OCP NIC 3.0

Also: OCP 3.0 NICRack

An open standard form factor for server network cards, letting one card design fit servers from many manufacturers.

The OCP NIC 3.0 specification was first released in 2019. It defines small and large card sizes that slide in from the front or rear of a server, making cards easier to service than standard PCIe cards.

Open Compute Project

Also: OCPGeneral

A nonprofit foundation where hyperscalers and suppliers publish open hardware designs for servers, racks, networking, power and cooling, so anyone can build to them.

OCP was started by Facebook (now Meta) in 2011. Its specifications include Open Rack, the OCP NIC 3.0 network card and the OCP Accelerator Module, and it runs workstreams on AI rack power, liquid cooling and Ethernet for scale-up networking.

Open Rack v3

Also: ORv3, Open RackRack

The third version of OCP's open rack standard, with 21-inch-wide equipment bays and power delivered over a shared DC busbar.

ORv3 equipment is about 21 inches wide and measured in OpenU of 48 mm, taller than the standard 1.75-inch rack unit. Power shelves convert AC to about 48 to 50 V DC on the busbar; one vendor's ORv3 high-power shelf is rated at 33 kW from six 5.5 kW supplies.

Open Rack Wide

Also: ORWRack

A double-width open rack specification introduced by Meta at OCP for the power, cooling and service needs of large AI systems.

Meta introduced Open Rack Wide (ORW) at the OCP Global Summit in October 2025. AMD's Helios rack-scale AI system is built on it.

Packet spraying

Also: multipathingCluster

Sending the packets of one data flow across many network paths at once, instead of one, to use all available bandwidth.

Traditional Ethernet keeps a flow on one path, which can leave some links idle while others jam. Spraying needs endpoints that can handle packets arriving out of order, which Ultra Ethernet's transport is designed for.

PJM

Also: PJM InterconnectionEnergy

The regional grid operator covering all or parts of 13 states and the District of Columbia, including Northern Virginia, one of the densest concentrations of data centers anywhere.

PJM runs the region's wholesale electricity and capacity markets. Data center growth has become a major driver of its load forecasts and capacity prices.

Power shelf

Rack

A shelf of power supplies in the rack that converts incoming AC power to the DC voltage the servers use.

In Open Rack, the power shelf feeds the rear busbar. As rack power rises, designs are moving these shelves out of the IT rack into a separate sidecar power rack.

Powered shell

Data Center

A data center building with utility power connected but without the cooling, backup power and fit-out installed.

Tenants buy powered shells to install their own systems, which suits AI operators with specific liquid cooling and power designs. It is quicker to deliver than a fully fitted facility.

PPA

Also: power purchase agreement, VPPAEnergy

Power purchase agreement: a long-term contract to buy electricity from a specific generator at an agreed price, often for 10 to 20 years.

A physical PPA delivers power to the buyer; a virtual PPA is a financial contract that settles the price difference while the power goes to the grid. Data center operators use PPAs to fund new renewable, nuclear and other supply and to meet carbon goals.

Related: Firm power, Capacity factor, SMR

Prefab and modular

Also: modular data center, prefabricationData Center

Building data center power, cooling or whole halls from factory-built modules that are shipped to site and connected, instead of constructing everything on site.

Factory assembly shortens site work and gives more predictable quality. Common examples are skid-mounted power rooms, packaged CDUs and containerized electrical houses.

PUE

Also: power usage effectivenessData Center

Power usage effectiveness: total facility energy divided by the energy used by IT equipment. 1.0 would mean zero overhead; lower is better.

A PUE of 1.5 means every 1 kWh to servers needs another 0.5 kWh for cooling and losses. Uptime Institute's surveys have shown the industry average at roughly 1.5 to 1.6 with little change for six consecutive years; it is standardized as ISO/IEC 30134-2.

Related: WUE, MW of critical IT, Chiller, TCO

Quick disconnect

Also: QD, UQDRack

A liquid-cooling connector that can be plugged and unplugged without draining the system or spilling coolant.

Quick disconnects let technicians remove a server from a liquid-cooled rack in the same way they would an air-cooled one. OCP lists them as part of the technology cooling system with cold plates, tubing and manifolds.

Rack-scale system

General

A complete AI computer that is designed, sold and installed as a whole rack rather than as individual servers.

The rack's accelerators, switches, power and liquid cooling are engineered together, and the rack is often shipped pre-integrated and tested. NVIDIA's GB200 NVL72 and AMD's Helios, built on Meta's Open Rack Wide design, are examples.

RDMA

Also: remote direct memory accessCluster

Remote direct memory access: letting one server read or write another server's memory directly over the network without involving its CPU.

RDMA cuts latency and CPU overhead, which AI and storage traffic depend on. InfiniBand has it built in; RoCE and Ultra Ethernet bring it to Ethernet.

Related: RoCE, InfiniBand, Ultra Ethernet

Rear-door heat exchanger

Also: RDHx, rear doorData Center

A liquid-cooled radiator mounted as the back door of a rack, which cools hot server exhaust air before it reaches the room.

RDHx units let air-cooled equipment run at higher density without changing the servers. OCP's cooling project has a door heat exchanger sub-project standardizing them for ORv3 racks.

Reciprocating engine

Also: recip engine, gas engineEnergy

A large piston engine, like a scaled-up car engine, that burns gas or diesel to drive a generator.

Engines start in minutes and can be added in blocks, so they suit bridge power and on-site generation as well as backup. Diesel units are standard for backup; gas-fired units are used for longer-running on-site supply.

Redundancy

Also: N+1, 2NData Center

Installing spare power and cooling equipment so the facility keeps running when one part fails or is under maintenance.

N is the amount needed to carry the load. N+1 adds one spare unit; 2N duplicates the whole system. AI training halls sometimes accept less redundancy than traditional enterprise sites to save cost.

Ride-through

Also: fault ride-through, voltage ride-throughEnergy

A facility's ability to stay connected and keep drawing power through brief grid disturbances, instead of disconnecting.

On July 10, 2024, a transmission fault in Virginia caused about 1,500 MW of data center load to switch to backup power at once, according to NERC, creating a sudden surplus on the grid. Grid operators now want large loads to meet ride-through requirements like those for generators.

RoCE

Also: RoCEv2Cluster

RDMA over Converged Ethernet: a way to run RDMA on standard Ethernet so AI clusters can get InfiniBand-like performance on Ethernet gear.

RoCE v2 runs over UDP/IP so it can be routed across a data center. It needs careful tuning of flow and congestion control to avoid packet loss, one of the problems Ultra Ethernet aims to address.

Scale-across

Also: multi-site trainingCluster

Linking AI clusters in separate data centers, sometimes many miles apart, so they can work on one job.

Sites are joined when a single building cannot get enough power for the cluster. It needs long-distance, high-capacity optical links and networking that tolerates higher latency than inside one hall.

Scale-out

Cluster

Connecting many racks or servers through a network so thousands of accelerators can work on one job.

Scale-out networks use InfiniBand or Ethernet in a Clos design. They are slower than scale-up links inside a rack but reach far more accelerators.

Scale-up

Also: scale-up domainRack

Linking a group of accelerators so tightly they can share memory and act like one big chip, usually within a rack.

Scale-up links are much faster and lower latency than the scale-out network between racks. NVLink is the established option; UALink and Ethernet via OCP's ESUN workstream are open alternatives.

Related: NVLink, UALink, ESUN, Scale-out

SerDes

Cluster

Serializer/deserializer: the circuit that turns data into a high-speed electrical signal on a chip's pins and back again. Its speed sets port speeds.

Speeds are quoted per lane: 100G and 200G per lane are current. Broadcom's Tomahawk 6 offers 200G or 100G PAM4 SerDes options.

Sidecar

Rack

A separate cabinet next to an AI rack that holds its power conversion or cooling equipment, freeing the main rack for compute.

Power sidecars appear in Mt Diablo and in NVIDIA's 800 V designs; NVIDIA showed an 800 V sidecar powering 576 Rubin Ultra GPUs in one Kyber rack. Cooling sidecars are liquid-to-air units that let a liquid-cooled rack run in a hall without facility water.

SMR

Also: small modular reactor, advanced nuclearEnergy

Small modular reactor: a nuclear reactor of up to about 300 MW of electric output per unit, designed for factory-built parts and faster construction.

The IAEA uses the 300 MWe ceiling. In May 2025 the NRC approved NuScale's uprated design of 77 MWe per module, up to six modules for 462 MWe. Data center operators have signed agreements with several SMR developers, with first commercial units still years away.

Solid-state transformer

Also: SSTData Center

A transformer built from power electronics instead of iron and copper windings, which can convert medium-voltage AC straight to DC in a smaller unit.

SSTs are being studied as a way to feed 800 VDC AI racks from the grid with fewer conversion steps. NVIDIA says it and its partners are evaluating both traditional transformers and SSTs; large-scale data center deployments are still early.

Switchgear

Data Center

The switches, breakers and protection equipment that route power around a data center and isolate faults.

Medium- and low-voltage switchgear are among the longest-lead items in a data center build. Switchgear and transformer supply have been a frequent schedule constraint in the AI build-out.

Tail latency

Also: p99 latencyCluster

The delay experienced by the slowest small fraction of messages, rather than the average one. In AI clusters the slowest message often sets overall speed.

Engineers quote it as a high percentile, such as p99. Because collectives wait for every participant, a few delayed packets can idle thousands of GPUs.

Related: Collective, Congestion control

TCO

Also: total cost of ownershipGeneral

Total cost of ownership: the full lifetime cost of an asset, including purchase, installation, power, cooling, maintenance and eventual replacement.

For AI infrastructure, energy and cooling costs over a multi-year life can rival the hardware price, so efficiency choices change the answer. TCO is the basis for comparing options such as air versus liquid cooling or AC versus DC power.

Related: Cost per token, PUE

TDP

Also: thermal design powerRack

Thermal design power: the amount of heat a chip is designed to produce under load, which sets how much cooling it needs.

TDP is quoted in watts and is close to the chip's power draw at sustained load. Rising accelerator TDPs, now around a kilowatt or more per chip, are the main reason AI racks have moved to liquid cooling.

Technology cooling system

Also: TCS, secondary loopData Center

The coolant loop that runs from the CDU to the cold plates on the chips, kept separate from the building's water.

OCP defines the TCS as everything from the cold plate to the CDU, including tubing, manifolds and quick disconnects. It uses treated coolant, often water with additives, and is held to strict cleanliness limits.

Tier classification

Also: Uptime TierData Center

Uptime Institute's four-level rating of a data center's resilience, from Tier I (basic) to Tier IV (fault tolerant).

Tier III means any component can be taken out for maintenance without shutting down IT. Tier IV means the site can survive any single unplanned failure. Certification is issued by Uptime Institute.

Related: Redundancy, Commissioning, UPS

Time to first token

Also: TTFTCluster

How long a user waits after sending a request before the AI model starts replying.

TTFT is the main responsiveness measure for chat and agent products. It depends on prompt length, model size and how busy the serving hardware is, and is traded off against total tokens per second.

Token

General

The unit of text an AI model reads and writes, typically a word or part of a word. AI usage and pricing are counted in tokens.

OpenAI's guidance puts one token at roughly three quarters of an English word, so 1,000 tokens is about 750 words. Tokens processed per second, and the cost of each one, are now common ways to compare AI infrastructure.

Tokens per second

Also: TPS, throughputCluster

How many tokens an AI system produces each second, either for one user or across a whole server or cluster.

Per-user tokens per second determines how fast a reply appears; system-wide tokens per second determines how many users the hardware can serve. Operators often quote it per GPU or per megawatt to compare efficiency.

Training

General

Building an AI model by running huge amounts of data through it until it learns patterns. Large training runs use thousands of accelerators for weeks or months.

Training jobs need every accelerator to stay in step, so they depend on fast networks and are hurt by any single failure. That is why training clusters invest heavily in scale-out networking and checkpointing.

Related: Inference, Checkpointing, Scale-out, MFU

Two-phase cooling

Also: 2-phaseRack

Liquid cooling where the coolant boils at the chip and turns to vapor, absorbing much more heat than liquid that only warms up.

Two-phase systems can be cold plates or immersion tanks. They use engineered fluids rather than water and are being evaluated for the hottest future chips; single-phase water-based cold plates remain far more common today.

UALink

Also: Ultra Accelerator LinkRack

Ultra Accelerator Link: an open standard for scale-up connections between accelerators from different vendors, an alternative to proprietary links.

The UALink Consortium ratified its 200G 1.0 specification in April 2025, supporting up to 1,024 accelerators in one pod at 200G per lane. Board members include AMD, AWS, Google, Intel, Meta and Microsoft.

Related: Scale-up, NVLink, ESUN

Ultra Ethernet

Also: UEC, UETCluster

An open set of Ethernet standards from the Ultra Ethernet Consortium, designed for AI and high-performance computing traffic.

The consortium, part of the Linux Foundation, published Specification 1.0 on June 11, 2025. It defines modern RDMA transport over Ethernet and IP and is designed to scale to millions of endpoints.

Related: RoCE, RDMA, InfiniBand, Packet spraying

UPS

Also: uninterruptible power supplyData Center

Uninterruptible power supply: batteries and power electronics that keep IT equipment running instantly when utility power fails, until generators start.

A UPS also cleans up voltage dips and spikes. AI racks with in-rack battery backup units reduce the role of large central UPS systems, and fast swings in AI load are a new design issue for both.

White space

Also: data hallData Center

The floor area in a data center where IT racks go, as opposed to the gray space holding power and cooling plant.

White space was once priced per square foot; for AI it is mostly priced by power. Liquid-cooled racks need far less floor area per MW, so AI halls can be smaller for the same load.

WUE

Also: water usage effectivenessData Center

Water usage effectiveness: liters of water a data center uses on site per kilowatt-hour of IT energy. Lower is better.

The metric was defined by The Green Grid. Sites using evaporative cooling towers have higher WUE; closed-loop designs with dry coolers can approach zero on-site water use but may use more energy.

Related: PUE, Chiller