Choosing a model for a generative AI product is less about picking the top name on a leaderboard and more about working through a series of decisions: what the tool must and must not do, where the data is allowed to go, how answers stay grounded, how candidate models are measured, what can go wrong, and what a pilot has to prove. This case study, written as a training exercise, walks through those steps one at a time.

To make the concepts easier to understand, let's consider a fictional example: an AI assistant that helps grant reviewers find evidence in grant applications. Each section below is one step of the discussion, applied to that example.

Step 1: Define the problem and the workflow

Reviewing a grant proposal means reading many documents, in different formats and styles, that belong to one application. A generative AI tool can speed up that review while keeping its quality. The tool helps reviewers find and summarize the information in a proposal that addresses particular review criteria. It is designed only to help find information; it cannot be used to make funding decisions or rank applicants.

The system takes the full application file and a reviewer's question, such as "What relevant experience with multisite clinical trials is described in the application?" It returns a factual answer with references to the relevant parts of the application. The reviewer checks those references and the rest of the material, and makes the evaluation.

Step 2: Let the constraints shape the architecture

Privacy, ownership, and the need to document where data comes from shape the architecture. Proposals contain sensitive information about applicants and other researchers, as well as budget details. Public commercial APIs raise privacy and compliance issues, while self-hosting open-weight models takes significant IT resources and ongoing security work.

We plan to use a hosted enterprise foundation model through a managed cloud service such as AWS Bedrock. That provides enterprise-level security and privacy: private access to the model, encryption in transit and at rest, and a commitment that our data is not used to train the model.

To keep answers grounded, we plan to use retrieval-augmented generation (RAG) rather than fine-tuning. Vector search retrieves the relevant passages from the application and passes them to the model as references during generation. RAG keeps the model from relying on what it memorized during pretraining and limits answers to what the application actually says. We may explore fine-tuning later if we find specific formatting needs, for example for citations or other structured output.

Step 3: Decide how to compare models

Candidate models will be evaluated on an internal test set of historical grant cases, using both automated metrics and expert review.

• Retrieval accuracy: Hit Rate@K and Mean Reciprocal Rank (MRR), to check that the retrieved passages are relevant.
• Generation accuracy and factuality: RAG-specific tools such as Ragas, using Faithfulness (whether each statement is supported by the provided documents) and Answer Relevance. Lexical measures such as ROUGE-L, along with semantic similarity measures, give a baseline comparison against reference answers.
• Operational viability: response time (under 3 seconds per request), tokens processed per minute, and cost per review.

Step 4: Plan for governance, risk, and fairness

The main operational risk is hallucination: the model stating facts or claiming qualifications that the evidence does not support. We plan to manage it with:

• Source-based answers: prompts that require citations to specific parts of the sources, and a default response ("Evidence not found in provided materials") when the application has no relevant information.
• Review interface: a tool that shows the source excerpts the system retrieved next to the answer they support, so reviewers can check them.
• Bias auditing: tests with varied prompts, and with applications from applicants of different backgrounds and institution sizes written in different styles, to catch unequal treatment.
• Access management: role-based access controls consistent with the existing grants management system, no retention of proposal text for training foundation models, and automated removal of sensitive data.

Step 5: Set the pilot success criteria

The tool will roll out on an opt-in basis to a pilot group of 50 reviewers. Moving to general availability depends on meeting these criteria:

• At least a 30% reduction in the time reviewers spend on documents.
• A hallucination rate under 1% in randomly sampled summaries checked by auditors.
• Reviewer ratings of at least 80% for both usefulness and clarity of citations.

The Clerk–Colleague–Coach framework, from Professor Andrew McAfee, describes three roles generative AI can play in almost any organization. The Clerk handles the busy work, like summarizing a report or filling in a template. The Colleague looks at your work and helps make it better. The Coach has seen similar cases before and suggests the next step.

Let's try an example of applying this framework in a fictional situation: a veterinary hospital. New patients need to fill long forms, and some of the questions are open text. Then the admin needs to process these forms and request the pet's previous medical report from the previous veterinarian, which takes a long time.

Clerk: The AI will receive the forms and make them digital by inserting them into the database. It will prepare a summary of the patient notes and, based on the answers from the pet owner, establish if the previous vet record is necessary, for example if important vaccine data is missing.

Colleague: The AI reviews both the admin's records request to the previous vet and the vet's notes, and suggests how to make them better. The admin or vet still makes the final call.

Coach: The AI can guide pet owners, vets and admin staff. For the admin staff, it can show them the information that is missing, the list of previous veterinarians, and how to request the records from them.

I think the Colleague role would have the biggest impact, because it makes the work much more efficient. The admin and vet can move directly to what matters.

The risk is that AI can make mistakes and overlook something important, like a history of allergy to a specific vaccine. If the staff rely only on the AI review, nobody may notice it.

A reflection on the MIT Sloan article How generative AI can boost highly skilled workers’ productivity.

The organization needs to involve all employees and make sure they are aware of the upper limit of AI, where AI excels and where a human can do better. The article calls this the "jagged technological frontier." When consultants used AI on tasks inside that boundary, their performance improved by nearly 40%, but on tasks outside it their performance dropped by 19 percentage points. So training should start there, because AI has its own limitations and not all tasks can be done efficiently with AI.

It is also important to work on a constant culture of employee feedback, to understand where AI works more efficiently than in other places. Along with that, the organization needs a culture of accountability, should reward peer training, and should encourage role reconfiguration.

From my own work, AI is great at developing test cases for an application, however it needs to have the complete context of the code. When it didn't have that context, it provided wrong answers, and I caught them by reviewing the test cases. So training should cover how to give AI the right context in the prompt, and reviewing what AI produces should be part of the training too, not something left to each employee.

Yes, I agree with Sutton's overall argument that advances in computing ability, including GPUs, have contributed to major AI advances by enabling the scaling of learning algorithms. Rather than programming in everything we know about human behavior, we can design AI systems to learn from data and other inputs. Still, I don't think design stops mattering. The design of specific algorithms is important to making the best use of computing resources.

Where I differ from Sutton is on human knowledge. I believe it is valuable to incorporate it, particularly when information that could help with a task is not easily contained in training data. A teacher, for instance, might recognize that a student's frustration with schoolwork is the result of issues at home. An AI trained only on grades and test scores would miss this, because the cause isn't in the data. In a case like that, adding computational tools without addressing the other factors may not be useful.

This means the role of human experts may change from rule writing to helping and reviewing what the AI has learned. For people building AI systems, the lesson is to use approaches that have proven effective and get better with additional computing resources, and to test them on practical tasks. For people training others to work with AI, the lessons include helping trainees identify and contextualize problems, review AI outputs, and decide when human involvement is appropriate.

Supervised learning

The primary purpose of supervised learning is to train a model using input/output pairs (e.g. documents labeled with their respective categories) as opposed to an unsupervised model which is trained on unlabeled data. The quality of output produced by supervised models can often be improved through data preprocessing (cleaning), feature engineering/selection and normalization; each of these steps will typically increase the amount of time spent training the model.

Self-attention

Self-attention mechanisms are used to enable a model (such as BERT or GPT) to learn about relationships between words. Models such as BERT and GPT utilize both supervised learning (when labels are provided), and self-attention mechanisms to enable them to identify how relationships exist between words in a given sentence.

BERT and GPT were initially trained via self-supervised learning. This means that instead of using human labeling to create prediction targets, prediction targets are created based upon the text itself. Both models do however rely on well-prepared training data. For example, BERT was trained by having it predict missing words. The ability to find relationships between words that are far away from one another, and the potential for the simultaneous processing of all tokens in a sequence during training, allow models utilizing self-attention to leverage large scale pre-training into multiple other areas of NLP.

Limitations

Each method has its own limitations. While a great deal of cost savings can be achieved by training a supervised model, costly labeling must occur prior to training. Additionally, there is no guarantee that the real world data you wish to process will resemble your training set. There are numerous drawbacks associated with large transformer-based models. They can take extensive amounts of computational power to run, and while they are capable of producing excellent results, it can be challenging to determine why certain predictions were made. When focusing on smaller, simpler, tasks, less computationally intensive, and easier to interpret supervised models are available.

In practice

Document classification in my organization requires reliable category labels and high-quality samples of representative examples. We reduce our burden of training our models by leveraging pre-trained models, however, we must still clean/preprocess our input data, assess the accuracy of our outputs, safeguard confidential/sensitive information and consider costs.

Short answers

Since large language models’ accuracy is mainly a result of high computation, I found it interesting to understand the infrastructure behind big AI labs like OpenAI and Anthropic. I found out that the big AI labs mostly rent their computers. A small group of cloud companies owns the chips, and the labs build and run their models on top of that hardware. Every number below links back to where I got it.

• Where does the hardware for a massive training run come from? Mostly from cloud providers. OpenAI says GPT-5 was trained on Microsoft Azure AI supercomputers (OpenAI), and Anthropic says it trains and runs Claude on a mix of AWS Trainium, Google TPUs, and Nvidia GPUs (Anthropic).
• Who owns most AI chips? At the end of 2025, Amazon, Google, Meta, Microsoft, and Oracle held an estimated 71% of the world's AI compute (Epoch AI). Google by itself has about 5 million H100 equivalents, which is around a quarter of the world total (Epoch AI).
• Do OpenAI and Anthropic own any? Very little, according to Epoch AI, which says neither one owns most of the compute it uses (Epoch AI).
• If Google has the most chips, why isn't Gemini on top? A lot of Google's compute never goes to Google DeepMind, since it also runs Search, ads, and outside cloud customers (Epoch AI). In fact DeepMind's estimated compute use at the end of 2025 was a bit lower than OpenAI's (AlphaSignal).

One note on units. Most figures here are in "H100 equivalents," which is Epoch AI's way of converting every chip, whether it's a Google TPU or an Amazon Trainium, into the number of Nvidia H100 GPUs with the same computing power (Visual Capitalist).

Why compute is the bottleneck

In practice, more compute tends to get you a better model, and labs can estimate how much better before they spend anything, so they're willing to commit billions before a training run even starts.

In 2020, an OpenAI team led by Jared Kaplan found that a language model's error drops as a power law as you add model size, data, and compute, and some of those trends held across more than seven orders of magnitude (Kaplan et al., 2020). Their fit gave an exponent of about 0.076 for model size and 0.050 for compute (TheoremPath, arXiv 2602.05298). If you run the math, 10x more parameters cuts error by about 16%. 10x more compute cuts it by about 11%.

The best public test of this came with GPT-4. OpenAI fit a curve to small models trained with up to 10,000x less compute and used it to predict GPT-4's final loss, shortly after the big run started and without looking at any partial results. The prediction turned out to be accurate (GPT-4 Technical Report).

But the 2020 paper got the budget split wrong. It said most of a 10x compute increase should go into model size, about 5.5x, with only about 1.8x more data (Lilian Weng). In 2022, DeepMind trained over 400 models for its Chinchilla work and found that size and data should actually grow at the same rate (Epoch AI). Its 70B Chinchilla model, trained on 1.4 trillion tokens, beat the much bigger 280B Gopher using the same compute (EfficientLLM survey).

For investors, the takeaway I see is that each step up this curve needs roughly 10x more compute, and somebody has to own that hardware.

Who owns the chips

Google owns more AI compute than anyone, and together with four other US cloud companies it holds most of the world's total. Their combined share went from 63% in early 2024 to 71% at the end of 2025 (Epoch AI).

These figures run through the end of 2025. For scale, world AI compute passed the equivalent of 16.5 million H100s by early 2026 (Epoch AI), and Chinese companies together own just over 5% of it, which is less than any single top US cloud company (Epoch AI on X).

Very few companies actually make the chips. In Q4 2025, Nvidia shipped about 2.96 million H100 equivalents, more than every other chip seller combined, with AMD (226,000), Amazon (221,000), and Huawei (132,000) far behind (Visual Capitalist).

The chips: GPUs and custom ASICs

AI training runs on two broad kinds of chips. GPUs are general-purpose parallel processors, while ASICs (application-specific integrated circuits) are custom built for one kind of work, in this case AI math, and they give up some flexibility to get more efficiency (How AI Works). Google's TPU is one type of ASIC (NextPCB).

GPUs still dominate, and Nvidia chips make up over 60% of all AI compute, with Google and Amazon chips making up most of the rest (Epoch AI). The pull of ASICs is mostly cost. Google claims its Ironwood TPU costs about 44% less to own per chip than a GB200 server, though that's by its own math (Tom's Hardware). Microsoft (Maia) and Meta (MTIA) build their own chips too (Hashrate Index).

The labs want in as well. OpenAI signed with Broadcom in October 2025 for 10 gigawatts of custom accelerators (Tom's Hardware), and in June 2026 it showed its first chip, Jalapeño, which is built for running models rather than training them and is still being tested (TechCrunch).

Most of these custom chips go through two design partners. Broadcom and Marvell together control roughly 95% of the custom AI chip co-design market (Tom's Hardware), and Broadcom's deal with Google runs through 2031 (Tech Wire Asia).

How much the labs use

Among the labs, OpenAI was probably the biggest user of AI compute at the end of 2025, at about 1.74 million H100 equivalents. These are Epoch AI estimates of what each lab used, whether it rented the chips or owned them (AlphaSignal, Epoch AI).

The ranges overlap quite a bit, so OpenAI's lead over DeepMind is narrow (AlphaSignal).

What stood out to me is OpenAI's growth. It roughly quadrupled its compute in both 2024 and 2025, which works out to about 17x over two years, and its fleet went from about 0.2 GW of power at the end of 2023 to 1.9 GW at the end of 2025. It plans to spend around $50 billion on compute in 2026, about triple its 2025 budget (AlphaSignal).

Still, the labs use a small slice of the world's chips. OpenAI, Anthropic, and xAI together used only 20% to 30% of global AI compute in 2025 (Epoch AI).

Do OpenAI and Anthropic own data centers?

Not in any big way. Both have huge sites built just for them, but someone else owns the chips, and Epoch AI counts rented compute toward the cloud company rather than the lab (Epoch AI).

OpenAI gets its compute mainly from Microsoft, Oracle, and CoreWeave (Epoch AI). Its flagship site is Stargate in Abilene, Texas, where Epoch lists Oracle as the owner of the chips and OpenAI as the user. The site has about 509,000 H100 equivalents today and a projected 1,019,000 by Q4 2026 (Epoch AI). Crusoe developed the campus (iTechGuides). Stargate started out as a joint venture in which OpenAI and SoftBank would each hold 40% (Wikipedia), but in April 2026 OpenAI was reported to have mostly backed away from owning Stargate sites, and it now prefers to lease (Tom's Hardware).

Anthropic gets its compute mainly from Amazon and Google (Epoch AI). Amazon's Project Rainier came online in late 2025 with nearly 500,000 Trainium2 chips, and Anthropic uses it to train and run Claude (About Amazon). The Indiana campus cost $11 billion and covers 1,200 acres (CNBC). On the Google side, Anthropic got access to up to one million TPUs in October 2025 (Wikipedia) and has since signed for multiple gigawatts of next-generation TPU capacity starting in 2027 (Anthropic).

The closest Anthropic has come to its own sites is a $50 billion plan with Fluidstack, announced in November 2025, for data centers in Texas and New York that are custom built for Anthropic and due online through 2026 (Anthropic). Fluidstack is the partner building them, and I couldn't confirm how many are running yet.

xAI is the exception here. Before SpaceX acquired it, xAI was the only major AI lab that owned most of the compute it used (Epoch AI), and its Colossus 2 in Memphis is the largest AI data center Epoch tracks, at about 1.1 million H100 equivalents (Epoch AI).

Where GPT-5 and Claude Opus 5.5 were trained

OpenAI has said where GPT-5 was trained. I couldn't find anything from Anthropic on where Opus 5.5 was trained.

GPT-5 launched on August 7, 2025, and OpenAI says it was trained on Microsoft Azure AI supercomputers (OpenAI). Microsoft said the same thing when it added GPT-5 to its own products (Microsoft). OpenAI didn't share GPT-5's size, its training compute, or which chips it used (AI Wiki).

Claude Opus 5.5 came out on September 22, 2026 (Anthropic). None of the announcements I found name the cluster or chips used to train it. What Anthropic does say is that it trains and runs Claude on AWS Trainium, Google TPUs, and Nvidia GPUs, and picks the chips that suit each job (Anthropic). Amazon has also said Anthropic uses Project Rainier to build future versions of Claude (About Amazon), but I didn't find a source that confirms which cluster trained Opus 5.5.

Who does what

In short, the cloud company owns the hardware and the lab owns the model. Here's how that splits out in the sources:

The relationship goes both ways, though, since the big cloud deals are built around what the lab needs. Amazon developed Rainier for Anthropic's Claude (DCD), and Anthropic's Fluidstack sites are custom built for its workloads (Anthropic).

Google has the most chips. Why isn't Gemini on top?

Having the most chips doesn't mean they all go to frontier models. I could only confirm part of this question from the sources.

Here's where Gemini stands right now. On the Artificial Analysis Intelligence Index on September 29, 2026, Claude Opus 5.5 scored 57.6, GPT-6 Astra 52.7, and Gemini 3.8 Flash 40.9. Google hasn't released a Gemini 3.8 Pro, though, so this is Google's fast model up against the others' flagships (Second Talent). On the Arena text leaderboard from September 13, Anthropic models held 7 of the top 12 spots (Remynd).

But Gemini still wins some tests. Gemini 3.1 Pro scored 94.1% on GPQA Diamond, level with GPT-5.5 and Claude Opus 4.8 (Tech Insider), and it led LiveCodeBench (Git AutoReview). So I'd say Gemini trails right now on the top independent index while still leading a couple of specific tests.

Part of the reason the chip count doesn't carry over is that Google's compute gets split across Google DeepMind, other internal uses, and cloud customers (Epoch AI). Epoch estimates DeepMind used about 1.58 million H100 equivalents at the end of 2025, which is below OpenAI's 1.74 million and only about a third of Google's roughly 5 million (AlphaSignal). And since outsiders can't see how many TPUs go to Gemini versus ads or Search, DeepMind's estimate has a wide range (AlphaSignal).

Google also rents its TPUs to a direct rival. Anthropic has access to up to one million of them (Wikipedia) and has signed for multiple gigawatts more from 2027 (Anthropic).

I couldn't find a source that explains the whole gap. They track compute, and they don't say much about research choices, data, or how each lab trains.

Why this matters for the future of AI

From what I found, control over AI chips keeps getting more concentrated.

The top five cloud companies went from 63% of world AI compute in early 2024 to 71% at the end of 2025 (Epoch AI). Nvidia chips alone make up over 60% of all AI compute, and Google and Amazon chips make up most of the rest (Epoch AI).

Demand is also growing fast. Global AI compute has grown about 3.3x a year since 2022, which means it doubles roughly every 7 months (Epoch AI), and the hardware already draws over 10 gigawatts, about twice what New York City uses (The Decoder). OpenAI alone plans to spend about $50 billion on compute in 2026 (AlphaSignal).

The labs are making different bets on this. OpenAI has moved toward leasing (Tom's Hardware), while Anthropic committed $50 billion to custom-built sites (Anthropic) and spreads its work across three chip families (Anthropic). xAI owns its own hardware (Epoch AI).

There's tenant risk too. A few Stargate partners reportedly felt misled by OpenAI, and one source said they'd rather have Microsoft as a tenant because it's more creditworthy (Tom's Hardware).

Running models eats a lot of compute now, too. Epoch estimates OpenAI's compute is split about 50/50 between research and serving users (AlphaSignal). Labs also train smaller models on much more data than Chinchilla suggests, because smaller models are cheaper to serve, and Llama 3 70B, for example, used about 214 tokens per parameter compared with Chinchilla's 20 (TheoremPath).

For someone following the money, I think the cloud owners and chip makers sit underneath every lab's progress, since each new 10x of compute the scaling curve rewards has to be bought or rented from them.

Sources

Research papers and data

• Kaplan et al., Scaling Laws for Neural Language Models (2020)
• OpenAI, GPT-4 Technical Report (2023)
• Lilian Weng, Scaling Laws, Carefully
• Epoch AI, Chinchilla scaling: a replication attempt
• EfficientLLM survey (arXiv 2505.13840)
• Logarithmic-time Schedules for Scaling Language Models (arXiv 2602.05298)
• TheoremPath, Scaling Laws: Kaplan, Chinchilla

Chip ownership and use

• Epoch AI, Introducing the AI Chip Owners Explorer
• Epoch AI, Five hyperscalers now own over two-thirds of global AI compute
• Epoch AI, AI chip owners data
• Epoch AI, Introducing the AI Chip Users explorer
• Epoch AI, Frontier labs don't use most AI compute (yet)
• Epoch AI, January 2026 Brief
• Epoch AI, home page data snapshots
• Epoch AI on X, hyperscaler ownership thread
• AlphaSignal, OpenAI's computing power grew 17x in two years
• Network World, Google owns the most AI compute
• Visual Capitalist, The companies that sell the most AI chips
• The Decoder, Global AI compute hits 15 million H100 equivalents

Data centers

• Epoch AI, AI data centers
• Epoch AI, OpenAI Stargate Abilene
• iTechGuides, Stargate Abilene explained
• Wikipedia, Stargate LLC
• Tom's Hardware, OpenAI has effectively abandoned first-party Stargate data centers
• About Amazon, AWS activates Project Rainier
• DCD, AWS activates Project Rainier cluster
• Anthropic, $50 billion in American AI infrastructure
• CNBC, Anthropic to spend $50 billion on U.S. AI infrastructure
• Anthropic, Google and Broadcom compute partnership
• Wikipedia, Anthropic

Models

• OpenAI, Introducing GPT-5
• Microsoft, GPT-5 across Microsoft products
• Microsoft Azure, GPT-5 in Azure AI Foundry
• AI Wiki, GPT-5
• Anthropic, Claude Opus 5.5
• Releasebot, Claude Developer Platform updates
• Second Talent, GPT-6 vs Claude Opus 5.5 vs Gemini 3.8
• Remynd, Claude vs ChatGPT vs Gemini, September 2026
• Tech Insider, Claude vs ChatGPT vs Gemini 2026
• Git AutoReview, Claude vs Gemini vs ChatGPT for code review

Chip types

• How AI Works, TPUs vs GPUs vs ASICs
• NextPCB, GPU vs TPU vs ASIC vs FPGA
• Epoch AI, AI chip owners methodology
• Hashrate Index, Hyperscaler AI ASIC market report
• Tom's Hardware, The custom AI ASIC state of play (May 2026)
• TechCrunch, OpenAI unveils its first custom chip
• Tech Wire Asia, Broadcom's deal with Google and Anthropic

In 2005, I began working in Information Retrieval (IR) and Data Mining, which is the foundational technology behind many AI applications, especially search, recommendation, and retrieval-augmented generation (RAG). As IR rapidly grew and became more widely applied, I began offering an elective course for undergraduate computer science students called Introduction to Information Retrieval. The goal of the course wasn’t just to teach IR techniques, but to help students think critically and learn how to design complex, real-world systems. I’ve always seen IR as a powerful intersection of applied computer science and software engineering.

Now, with the rapid growth of AI and the rise of RAG, it is worth revisiting the fundamentals of IR and highlighting why they matter more than ever. It is important for computer science and engineering programs to keep their curricula aligned with the skills future graduates need, so software engineers remain relevant and effective in a rapidly evolving field.

Over the past two months, I worked extensively with generative AI (Claude Code) to build a complex enterprise-grade system, from simple data retrieval and UI interactions to an integrated data mart and backend infrastructure. This was not a static or well-defined task. In fact, the project was intentionally designed to be Agile: the requirements changed rapidly, the priorities shifted frequently, and iterative feedback was an important part of the process. AI was embedded in the development workflow at every step — not just as a code generator, but as a partner.