By the end of 2026, discussion in the industry had moved on from a focus on increasing the number of GPUs. Instead, the emphasis turned to the practical problems involved in integrating, cooling, networking and financing these chips. It is expected that 2027 will see the convergence of next-generation AI platforms with physical, power, and supply chain limitations. Far from causing worry, these developments highlight the need for more strategic procurement. The new platforms involve more than just incremental improvements in speed over the previous ones. The NVIDIA Rubin Ultra generation, together with Vera CPUs and the higher-density Kyber rack, includes four GPU chiplets, a terabyte-class HBM4E memory stack, and a much broader scale-up domain than Blackwell. NVIDIA has already started shipping Vera Rubin systems to AWS, and both companies have agreed to deploy an additional two million NVIDIA GPUs in 2027 and 2028. This commitment shows a major investment in agentic workloads which demand a large amount of memory bandwidth as well as high computational throughput; workloads that consume memory bandwidth to the same extent as raw FLOPS.

AMD is taking an active role in this competitive environment, with its Instinct MI450-series components and Helios racks being presented as alternative options for use in both training and inference, thanks to its agreements involving multi-gigawatt capacity. Google is, for the first time, splitting its eighth-generation TPU into separate training and inference chips. AWS is incorporating NVLink Fusion and custom high-bandwidth memory into the next generation of Trainium, so that its own silicon can be used alongside NVIDIA GPUs in the same rack. Meta is aiming for a six-month release cycle when it comes to new MTIA generations. The market has now moved from having a single dominant architecture to consisting of a complex ecosystem which includes merchant GPUs, hyperscaler application-specific integrated circuits (ASICs), and specialised inference engines.

Custom silicon is no longer a side project

Counterpoint Research believes that the number of AI server compute ASICs shipped by the major companies will triple from 2024 to 2027. Google’s TPU fleet is still the main factor behind the high volume of shipments. AWS, Meta, and Microsoft have now grown to such a scale that their proprietary chips affect production decisions, not just contribute to industry debates. Broadcom is still the main design partner for a large number of these projects, and 3D stacking technology is moving from the experimental stage to being included in forecasts covering millions of units.

For buyers, the impact of these developments is greater than that of concerns regarding brand loyalty. The demands for training clusters and for production inference farms have become different, calling for separate hardware solutions. Companies which have the means to use both NVIDIA hardware for advanced training and custom silicon for large-scale inference will attain a higher level of computational efficiency for each unit of power. All the other organisations will obtain this combination of hardware via cloud providers who have already made the necessary investments in it.

2027 is the peak year of constraint, not the peak year of demand

The main point of uncertainty is concerning supply and not demand. Ben Bajarin, an analyst, has stressed that 2027 will be the year when the capacity investments carried out in 2026 start to become reality, even if this does not happen all at once. Having an accelerator does not ensure immediate deployment, since delays can take place because of dependencies on substrates, capacitors, power stages, or site energization. The development of high-bandwidth memory (HBM), advanced packaging, transformers, and interconnection processes proceeds at different speeds, which leads to a continuing discrepancy between the hardware that is ordered and the hardware that is actually in operation.

The capacity of the foundry is just as limited. It is expected that AI applications will account for most of TSMC’s N3 family wafer output by 2027. Once a fabrication node has reached such a high level of usage, it becomes impossible to shift capacity from other areas, for example from smartphones. Packaging and high-bandwidth memory still act as bottlenecks. As a result, companies preparing clusters for 2027 should treat delivery dates as approximations rather than as certain assurances.

Power is the governor on the whole machine

Although a great deal of attention is given to chip development roadmaps, the supporting infrastructure such as substations is discussed much less often. GPUs of the Rubin class are now reaching thermal design power levels of several kilowatts. Rack densities, which had earlier been about 10 kilowatts, are now going beyond 100 kilowatts, and by 2027 the systems will exceed the capacity of conventional air cooling. Liquid cooling has moved from being a luxury feature to becoming a basic requirement for system operation.

Even though there have been improvements in cooling, the basic energy requirements still represent a major limitation since the electrical grid cannot keep up with the growing level of computational demand. In large U.S. markets, the time one has to wait to be connected can reach three or four years. Measures like generating power behind the meter, using on-site gas, and employing long-lead transformers have now become an essential part of infrastructure planning. As ARK’s infrastructure analysis shows, spending on AI infrastructure is expected to rise from about half a trillion dollars in the mid-2020s to well over a trillion dollars by the end of the decade, with custom silicon making up an ever-increasing share of this investment.

Instead, investors should concentrate on metrics like energized megawatts and time-to-power, not just on GPU backlogs; for buyers the cost-effectiveness of a chip is reduced if it stays in storage while the supporting infrastructure is being completed.

The software layer will decide who actually uses the new iron

Hardware that doesn’t have a corresponding software story is just a thing to be displayed in a museum. CUDA continues to be the central force in the industry, which is the reason why NVIDIA can introduce an LPU path derived from Groq for low-latency inference without having to ask customers to set up a second operating model. Although AMD’s ROCm stack has improved, it still has to contend with the difficult middle ground consisting of compilers, kernels, and the engineers who are unwilling to rewrite their training pipelines. Hyperscalers’ ASICs succeed when they have full ownership of the workload, but they fail when a customer requires portability across the different clouds.

The development of agentic systems brings with it both new complexities and new possibilities. Agents that carry out multiple steps keep more information in their memory, interact with external tools, and have varying computational demands. Because of this, architectures featuring high-bandwidth memory, efficient interconnects when scaling up, and CPUs able to maintain accelerator throughput are preferred. NVIDIA’s Vera and AWS’s Trainium together with NVLink are examples of responses to these changing requirements. While neither of these solutions is final, they do show a move away from the conventional approach of allocating a single large GPU to each task.

What buyers should actually do in 2027

Procurement strategies ought to focus on utilization planning rather than just obtaining the most up-to-date generation of hardware. Buyers should ask for figures such as tokens per second per provisioned megawatt, not the peak floating-point operations per second (FLOPS). It is important to assess the cooling system, the power supply, the availability of spare parts, and the software’s ability to adapt if there are changes to the architecture. It is recommended to combine merchant GPUs for flexibility with reserved ASIC capacity for existing workloads. When power availability does not become available until 2028, a reservation made in 2027 should be treated as a financial commitment rather than as an immediate purchase of the product.

The engineering work should be directed towards rack-scale designs since the main area of focus has moved from single cards to full domains that include 72 GPUs, 144 sockets, integrated liquid cooling, and strong networking capable of handling multiple high-intensity workloads. Investors should see the increase in custom silicon as an indication of an expansion and diversification of the total addressable market rather than as a sign that NVIDIA is declining. Although NVIDIA could see a drop in its market share it could still achieve revenue growth if rack densities rise and the margins related to software stay high. This situation is more likely in 2027 than a full market displacement.

The uncomfortable forecast

Next-generation platforms are anticipated to provide much improved capabilities, but they will also involve higher siting costs, greater cooling difficulties, and longer energization times than the promotional materials indicate. Success will not lie in having the best technical specifications, but rather in being able to get the most computational output from limited power resources before the following generations of hardware change the competitive situation.

In 2027 the future of AI computation will not be determined by a single chip but will instead be the result of an integrated ecosystem consisting of silicon, memory, optical components, cooling systems, power transformers, and software, all working together as part of a single infrastructure.

References

1. AWS and NVIDIA plan to deploy an additional 2 million GPUs for use in AI during 2027 and 2028 — https://www.aboutamazon.com/news/aws/aws-nvidia-2-million-gpus-ai
2. 2027: Peak Year of Constraint, Ben Bajarin — https://www.thediligencestack.com/p/2027-peak-year-of-constraint
3. AI Server Compute ASIC Shipments to Triple by 2027, Counterpoint Research — https://counterpointresearch.com/en/insights/AI-Server-Compute-ASIC-Shipments-to-Triple-by-2027
4. The State Of AI Infrastructure: Demand, Costs, And Custom Silicon, ARK Invest — https://www.ark-invest.com/articles/analyst-research/the-state-of-ai-infrastructure-demand-costs-custom-silicon
5. Driving Down The AI System Roadmap With Nvidia, The Next Platform — https://www.nextplatform.com/compute/2026/03/19/driving-down-the-ai-system-roadmap-with-nvidia/5210195

Written and researched by Peter Jonathan Wilcheck