Inside a data center, rows of servers generate the computational capabilities necessary to train and operate artificial intelligence models. AI computing is poised to become one of the priciest assets in business history, with significant financing targets hinging on its potential.
When you inquire with business leaders about AI computing, they often gesture towards a warehouse filled with chips, offering a vague description tied to a $500 billion financing initiative.
To clarify, AI compute encompasses the entire stack of technology that is essential for training and utilizing AI models. This includes the chips along with memory, networking, power, cooling, and the software that makes everything functional. It's important to highlight the "AI" in AI compute, as "compute" generally refers to a broader category of technological resources. For instance, Amazon markets compute as a general service that includes servers and serverless code. The distinction is crucial: it illustrates how individuals may sound knowledgeable while still misunderstanding what they are actually purchasing.
Traditionally, "compute" served as a verb related to calculations—a process performed by machines rather than a line item in financial management. When you purchased a computer, the notion of computing was intrinsic to its operation.
However, with the rise of cloud services, computing has evolved into a commodity that can be bought hourly, similar to electricity. This expanded definition now includes a variety of devices—from smartphones to weather prediction systems—all contributing to an everyday understanding of what compute entails.
The introduction of artificial intelligence has further refined this concept. Training AI models demands substantial resources, leading to a situation where one company dominates the supply of data center GPUs globally. Consequently, “compute” now frequently refers specifically to the hardware associated with AI unless clarified otherwise. While traditional computing remains relevant—billed based on usage, usually metered hourly—the conversation has shifted primarily towards this newer interpretation.
Moreover, compute has transformed into a significant asset that companies buy, finance, depreciate, and account for on their balance sheets. Recently, Nvidia and six leading financial institutions signed agreements to mobilize over $500 billion in capital for AI compute, with the completion of these agreements pending execution.
This important change highlights a key factor influencing financial viability, which is often overlooked.
To understand what companies are actually purchasing, one should consider the essential components involved—chips, power, and CUDA software. A GPU, for instance, is not particularly effective by itself; it requires speedy memory, robust networking, processors to supply data, and storage solutions. Collectively, these create an "AI factory"—a specialized data center centered around high-value hardware. Additionally, software, particularly CUDA, plays a critical role, as engineers must interact with it to utilize the chips effectively. This dependency reflects a switching cost: firms are not just acquiring silicon but also committing to a specific set of tools their engineers are trained to use. Elements of a data center do not all depreciate at the same rate; buildings may last decades, while chips are a variable investment. This phenomenon exemplifies how compute functions as an asset, with many discussions concerning AI development branching from this fundamental understanding.
Importantly, compute serves two distinct purposes—training and inference—with each stage operating on different timelines. Training involves building and refining the model, characterized by intensive and finite resource usage. Conversely, inference refers to the ongoing process where a trained model generates outputs continuously. Each individual output may be minimal, but when multiplied across vast user bases, the demand for processing power can become significant. Consequently, older chips that no longer excel at training tasks often find a second life in inference applications, where performance requirements may be less stringent.
The pricing and sales of AI compute typically follow three models, each addressing the question: who assumes the risk if the asset’s value declines? Companies can either purchase the hardware, becoming responsible for its depreciation, or they can rent by the hour, paying a premium while shifting that risk to the provider. Alternatively, firms can bypass hardware acquisition altogether by accessing AI capabilities through APIs, avoiding ownership and depreciation concerns. While costs fluctuate based on usage patterns, organizations often mistakenly invest heavily in hardware, only to utilize a fraction of its potential, allowing their assets to diminish in value. Industry averages suggest that substantial ownership benefits are realized only at around 70% consistent usage, while renting tends to be more economical below 30%. Evaluating how well companies utilize their systems becomes crucial, as discrepancies often exist between perceived and actual capacity usage.
This financial landscape has piqued Wall Street's interest in financing a staggering $500 billion for AI compute. The logic is simple: data centers equipped with GPUs generate reliable cash flow, which can be leveraged for loans. However, challenges arise concerning collateral. Unlike established assets, such as leased planes or toll roads, the market for compute has only recently emerged, lacking robust benchmarks for secondary sales. The absence of clear valuation makes lenders hesitant, as the lifecycle of computing hardware can be unpredictable. As one expert noted, “Technology obsolescence is sudden, not gradual,” highlighting the risk involved. Despite this, older chips can remain functional and profitable, as evidenced by companies like CoreWeave signing contracts for several years of older architecture.
Ultimately, Nvidia's involvement adds another layer of complexity. While the company announced the financing initiative, Jensen Huang pointed out that Nvidia might provide a residual-value support mechanism for a maximum of 25% of an opportunity, determined on a case-by-case basis. This dilutes any assumption about the level of risk the company is willing to absorb, revealing a cautious approach towards potential shortfalls in asset valuation.
