AI Compute Cost Outlook: Quarterly Briefing

AI compute costs are entering a high-price, bifurcated regime: frontier hardware remains supply-constrained, while older GPUs and specialized cloud capacity are getting cheaper. Over the next two quarters, expect modest rental-price inflation for leading accelerators—not a cyclical collapse—despite improving utilization and competitive pressure.

Compute economics: lower unit cost, higher absolute spend

The headline H100 on-demand rate is approximately $4.29 per GPU-hour as of October 25, with hyperscaler pricing materially above specialist providers.^[10] Across the broader market, typical 2026 rental rates span $2–$10 per GPU-hour, reflecting substantial differences in chip generation, networking, reservation terms, and utilization.^[2]

The key distinction is between cost per unit of compute and total AI infrastructure spend. Better software utilization, quantization, sparsity, and mixture-of-experts architectures are reducing compute per token. However, demand for inference, agentic workloads, and larger context windows is expanding faster than efficiency gains. Hyperscaler AI capex is therefore likely to remain elevated into 2027; industry forecasts place combined spending near $1 trillion in 2027.^[9]

GPU pricing: new silicon firm, secondary market softer

New H100, H200, and B200 accelerators generally trade in a $25,000–$45,000 range, while complete eight-GPU servers cost roughly $250,000–$500,000. GB200/GB300 NVL72 racks command approximately $2 million–$4 million.^[1]

This pricing structure is unlikely to normalize quickly. CoWoS lead times remain 52–78 weeks, and HBM capacity is allocated through 2026, limiting the ability of supply additions to produce near-term price deflation.^[1] HBM3E input prices have also risen, with H200 pricing reportedly supported by roughly 20% memory-cost inflation.^[4]

The pressure is less uniform in the installed base. Used H100s can trade from approximately $8,200–$25,000, versus $25,000–$40,000 for new units, creating a growing low-cost alternative for inference and non-frontier training.^[8]

Trading implications

The market favors suppliers with control over packaging, HBM allocation, networking, and rack integration—not merely accelerator designers. GPU-cloud margins will remain polarized: hyperscalers retain pricing power but face heavy capex depreciation, while neoclouds offer lower rates but carry utilization and financing risk. H100 median rental prices were up 15% year over year in late September, reinforcing the absence of broad-based price relief.^[3]

What to watch

• HBM and CoWoS lead times: the earliest evidence of capacity normalization would be the clearest catalyst for GPU price compression.^[1]
• H100 versus B200 utilization: falling H100 rates alongside stable next-generation pricing would signal migration, not weakening AI demand.
• Hyperscaler capex discipline: Microsoft’s roughly $190 billion 2026 capex and Meta’s $125–145 billion range show the scale of exposure to component inflation and execution risk.^[3]

Members-only briefing synthesized by the AISA LLM layer (AISA Perplexity API). 2026-10-26.