AI-Ready Infrastructure: Preparing Your Data Centre for the Intelligence Era

Traditional data centre design optimises for CPU-bound workloads: web servers, databases, transactional applications. The fundamental constraint is compute per rack unit, with networking sized for east-west traffic patterns familiar from three-tier application architectures. AI workloads break every one of these assumptions.

Training a large language model or running inference at scale demands GPU density, ultra-low-latency interconnects between accelerators, and network fabric speeds that would have seemed excessive for any enterprise workload five years ago. An NVIDIA H100 GPU draws up to 700 watts — nearly ten times a server CPU. A rack of GPU servers may require 40–80 kW of power, compared to the 5–10 kW typical for compute-dense traditional racks. Most data centres, including many co-location facilities, are simply not designed for this.

AI infrastructure is fundamentally a networking problem as much as a compute problem. GPU-to-GPU communication during distributed training requires extremely high-bandwidth, low-latency interconnects. NVIDIA's NVLink handles intra-node GPU communication; InfiniBand (HDR or NDR, supporting 200Gb/s or 400Gb/s) or RoCE (RDMA over Converged Ethernet) handles inter-node communication at scale.

For AI inference serving — where models respond to user requests in real time — the bottleneck often shifts to the network path between the GPU cluster and the storage layer. NVMe-over-Fabrics (NVMe-oF) with 100GbE or higher Ethernet allows flash storage to be disaggregated from compute while maintaining near-NVMe latency. This is essential for large model deployments where the model weights themselves (sometimes hundreds of gigabytes) must be loaded into GPU memory quickly.

If your current data centre fabric is 10GbE or even 25GbE at the server level, a targeted investment in 100GbE or 400GbE spine infrastructure is likely a prerequisite before any serious AI capability can be deployed on-premises.

Power density is the most immediate physical limitation in most data centres. The shift to GPU-heavy AI workloads has caught many facilities operators off guard: racks that were safely within thermal and power limits yesterday may require complete row redesigns to support AI servers today.

Traditional air cooling — even high-efficiency hot aisle/cold aisle containment — struggles above approximately 25–30 kW per rack. AI GPU racks regularly require 50–80 kW. Direct liquid cooling (DLC), where coolant is piped directly to heat exchangers on CPUs and GPUs, is increasingly the only viable thermal solution. Immersion cooling — where servers are fully submerged in dielectric fluid — offers even greater efficiency but requires more significant facility modifications.

Before committing to AI infrastructure, a power and cooling audit of your existing facility is essential. Understanding your available power capacity (including UPS and generator sizing), available cooling capacity, and structural limits (floor loading for dense racks) will determine whether an on-premises build is feasible or whether a cloud or AI-optimised co-location provider is the right path.

For most enterprises, the right answer is a hybrid model. Cloud platforms — AWS, Microsoft Azure, and Google Cloud — all offer GPU-backed instances that can be provisioned within minutes, making them ideal for experimentation, burst inference workloads, and initial model development. The pay-as-you-go model avoids large upfront capital commitments while the organisation learns what its AI workloads actually require.

As AI workloads mature and usage patterns become predictable, the economics of on-premises or co-location GPU infrastructure improve significantly. A dedicated H100 cluster running at high utilisation will typically outperform the cost of equivalent cloud GPU instances within 18–24 months. The tipping point depends on your utilisation rate and the specific cloud pricing for your region.

The critical infrastructure investment either way is the data platform. AI models are only as good as the data they can access. Building a robust, well-governed data platform — with clear data ingestion pipelines, quality controls, and access management — is the foundational work that unlocks every AI capability above it.