Choosing a model for a generative AI product is less about picking the top name on a leaderboard and more about working through a series of decisions: what the tool must and must not do, where the data is allowed to go, how answers stay grounded, how candidate models are measured, what can go wrong, and what a pilot has to prove. This case study, written as a training exercise, walks through those steps one at a time.
To make the concepts easier to understand, let's consider a fictional example: an AI assistant that helps grant reviewers find evidence in grant applications. Each section below is one step of the discussion, applied to that example.
Step 1: Define the problem and the workflow
Reviewing a grant proposal means reading many documents, in different formats and styles, that belong to one application. A generative AI tool can speed up that review while keeping its quality. The tool helps reviewers find and summarize the information in a proposal that addresses particular review criteria. It is designed only to help find information; it cannot be used to make funding decisions or rank applicants.
The system takes the full application file and a reviewer's question, such as "What relevant experience with multisite clinical trials is described in the application?" It returns a factual answer with references to the relevant parts of the application. The reviewer checks those references and the rest of the material, and makes the evaluation.
Step 2: Let the constraints shape the architecture
Privacy, ownership, and the need to document where data comes from shape the architecture. Proposals contain sensitive information about applicants and other researchers, as well as budget details. Public commercial APIs raise privacy and compliance issues, while self-hosting open-weight models takes significant IT resources and ongoing security work.
We plan to use a hosted enterprise foundation model through a managed cloud service such as AWS Bedrock. That provides enterprise-level security and privacy: private access to the model, encryption in transit and at rest, and a commitment that our data is not used to train the model.
To keep answers grounded, we plan to use retrieval-augmented generation (RAG) rather than fine-tuning. Vector search retrieves the relevant passages from the application and passes them to the model as references during generation. RAG keeps the model from relying on what it memorized during pretraining and limits answers to what the application actually says. We may explore fine-tuning later if we find specific formatting needs, for example for citations or other structured output.
Step 3: Decide how to compare models
Candidate models will be evaluated on an internal test set of historical grant cases, using both automated metrics and expert review.
- Retrieval accuracy: Hit Rate@K and Mean Reciprocal Rank (MRR), to check that the retrieved passages are relevant.
- Generation accuracy and factuality: RAG-specific tools such as Ragas, using Faithfulness (whether each statement is supported by the provided documents) and Answer Relevance. Lexical measures such as ROUGE-L, along with semantic similarity measures, give a baseline comparison against reference answers.
- Operational viability: response time (under 3 seconds per request), tokens processed per minute, and cost per review.
Step 4: Plan for governance, risk, and fairness
The main operational risk is hallucination: the model stating facts or claiming qualifications that the evidence does not support. We plan to manage it with:
- Source-based answers: prompts that require citations to specific parts of the sources, and a default response ("Evidence not found in provided materials") when the application has no relevant information.
- Review interface: a tool that shows the source excerpts the system retrieved next to the answer they support, so reviewers can check them.
- Bias auditing: tests with varied prompts, and with applications from applicants of different backgrounds and institution sizes written in different styles, to catch unequal treatment.
- Access management: role-based access controls consistent with the existing grants management system, no retention of proposal text for training foundation models, and automated removal of sensitive data.
Step 5: Set the pilot success criteria
The tool will roll out on an opt-in basis to a pilot group of 50 reviewers. Moving to general availability depends on meeting these criteria:
- At least a 30% reduction in the time reviewers spend on documents.
- A hallucination rate under 1% in randomly sampled summaries checked by auditors.
- Reviewer ratings of at least 80% for both usefulness and clarity of citations.