Every sprint, we are required to run Fortify security scans on our applications. Fortify gives us two scans. The static scan reads the source code without running it, and the dynamic scan tests the running application from the outside, the way an attacker would. Each finding comes with a severity level, and the higher the severity, the tighter the deadline. Some findings are real issues. Others turn out to be false positives.

Fixing them has always been slow. Developers need to invest many hours to research each finding and understand the issue. Usually the developers fixing the issues are not the ones who wrote the code, so the team needs time to understand the code before applying the fix. Depending on the complexity, the cycle can take weeks, and the deadlines don’t wait.

We are building an agent team with the Claude Agent SDK to do this work with us. While designing it, I kept coming back to one question: what kind of teammate should the AI actually be?

Research on human-AI teams separates two results. Augmentation is when AI makes the person better than they are alone. Synergy is when the person and AI together beat both the person alone and the AI alone. A large meta-analysis by Vaccaro, Almaatouq and Malone (2024) looked at 106 experiments. Augmentation was common, but synergy was rare, especially in decision tasks. In many studies the AI alone would have done better than the team.

An X-ray example helped me understand why. Say an experienced doctor is right 90% of the time and the AI is right 75%. Together they can beat both, because the doctor knows when the AI is wrong. Now take a new doctor who is right 70% of the time, with an AI at 90%. The team is better than the doctor but worse than the AI alone, because the new doctor overrides the AI at the wrong moments. A team only beats both when the person is good enough to judge the AI.

The same paper recommends dividing the work up front, based on who is better at each part. That is how we designed our agent team. The agents review all static and dynamic findings and categorize them into true or false positives. A senior developer agent fixes the true positives, and another agent reviews the code. A senior QA engineer agent runs unit tests and dynamic tests, and a coordinator agent tracks the work and reports back to me. If an agent thinks an issue is a false positive, it tags it and I make the decision. After the fix we run both scans again.

I keep the false-positive decision for a reason. It’s the most costly mistake. AI can hallucinate, and its explanation can look very real and seem accurate. A real issue that gets wrongly dismissed stays in production as a security hole nobody is looking for.

Personality turned out to matter too. In the MindMeld study, Ju and Aral gave AI agents different Big Five personalities and found that the pairing of human and AI personality changes the quality of the work. I am open and conscientious. For security fixes, I want agents that are highly conscientious and lower on openness than me. There is a set of rules in place, and if a request goes beyond these rules, the agent should push back and let me know the rule forbids it. An agent with high openness like me can go out of context, and a small security fix turns into a bigger change nobody planned. For designing a new feature, a more open agent can work.

Another paper from the same group surprised me. DosSantos DiSorbo, Ju and Aral found that AI agents follow rules strictly even when a person would make an exception. If the limit for flour is $10, the agent won’t buy it at $10.01, while most people would. In a grocery run that is a weakness. In security work it’s exactly what we want.

For communication, we use AI to increase productivity, so I want agents that only focus on business. Ju and Aral found that human-AI teams focused on process and content, and more feedback messages were linked to higher quality. Business focus doesn’t mean the agents stay quiet, though. They need to ask questions and never assume. I also found that an interview-style review works much better for me, like “give me the first 10 and let’s go through them one by one.” After approving 50 good recommendations in a row, it’s easy to stop reading closely.

Claiming AI alone will be 100% correct is unrealistic. But these are security risks, so we need to aim for 100%, and we get there with a human in the loop. Augmentation is the part I’m sure of. Synergy is the goal, and splitting the work by who does each part better is how we’re trying to reach it.

Sources

1. Vaccaro, Almaatouq & Malone (2024), “When combinations of humans and AI are useful,” Nature Human Behaviour
2. Ju & Aral, “Collaborating with AI Agents”
3. DosSantos DiSorbo, Ju & Aral, “Teaching AI to Handle Exceptions”

Research on AI at work keeps pointing to two things. First, AI helps most on tasks within its capability frontier and can hurt performance on tasks outside it (1). Second, people who trust AI use it more, and people who use it more get the gains (2). The trust gap is widest among people who rarely use AI, and a simple explanation of how it works raises their trust a lot (3).

In software engineering, writing unit tests and running dynamic tests on developed code is within the frontier. I can develop unit tests and run dynamic tests in just a few prompts, at a speed and quality that was never possible in the past. The engineering part of development is a different story. With code, you need to gather requirements, design models at least at a high level, and then make architecture decisions. The risk is that the code works, but AI did not take the best approach to solve the engineering problem. A human in the loop needs to know these engineering concepts to catch it.

When we started using Claude Code, we started with small, low-risk applications. We reviewed the code extensively and created several rules and procedures to make sure it follows our standards. That is what built our trust to keep using it.

For employees with less technical background, I would build trust this way:

1. Develop detailed case studies showing how AI improved productivity in their world. The case studies should be real and 100% related to their job description.
2. Offer AI training. People need to feel comfortable with a tool to trust it.
3. Provide a sandbox, so employees can test the tool without worrying about its impact.
4. Don’t force it. Forcing people to use technology when they are not comfortable with it can create a negative impact.

Consistent exposure and training are very important. As we use the tool more, we figure out how it can best help our productivity. More exposure also means we understand the tool better.

Sources

1. Dell’Acqua et al., “Navigating the Jagged Technological Frontier”
2. Brynjolfsson, Li & Raymond, “Generative AI at Work”
3. Li & Aral, “Human Trust in AI Search: A Large-Scale Experiment”

I think how AI can contribute to a problem should follow a standard framework in any company. Too often we start with the AI tool and only later ask what problem it solves. A short questionnaire about the problem, translated into measurable productivity, is a better start. Sinan Aral's AI productivity framework covers the general idea, and people should be able to map the problem they are trying to solve onto it.

The three parts of the framework

The framework has three parts.

• Improvements are about what gets better: efficiency (cost and time saved) and quality (fewer errors and more creativity).
• Complementarities are about what decides how much you gain from working with AI, which depends on the task, the person's skill, and how much the person follows and trusts the AI.
• Synergy vs. augmentation asks whether AI should replace people or support them. Aral splits that into agent effects, meaning who performs better alone, and task effects, meaning whether the task is to decide something or to create something.

Why use a framework at all?

First, it makes productivity measurable. Second, it standardizes the process, so new people follow the same procedure when they onboard. It also lets different teams compare AI projects on the same terms, it shows where a person has to stay in the loop, and it gives a clear reason to say no when AI doesn't fit the problem.

The questions

The questions should cover all three parts of the framework:

1. Improvements: what will improve (time, cost, errors, quality), and how will we measure it?
2. Complementarities: what kind of task is this, what skills does the team bring, and how much will people trust the output?
3. Synergy vs. augmentation: can AI do this better alone, or should it support a person? Is the task deciding or creating?

How it maps to my own work

AI is very dependent on context. For code testing and QA, as long as the skills are defined and AI is instructed what to do, I think AI will do a better job than me. QA is also both kinds of task. Writing the tests is creating. When AI runs the tests and builds the list of tasks that need to be done from the results, it is deciding what needs work. AI is also very good at research and collecting the necessary information before a human makes the final decision. But I am not comfortable letting AI make health-related decisions for me. There, AI supports me and I decide.

Security scan findings (Fortify, in my case) are somewhere in between. I ask AI to recommend with an explanation of why. After reviewing the recommendation, I ask how it plans to fix each issue. After reviewing the plan, I approve or disapprove. AI does the analysis and I keep both decisions.

I also found that an interview-style conversation with AI is much more effective than one big report. I say something like "give me the first 10 and let's go through them one by one." Small batches keep my attention fresh, and it's hard to rubber-stamp something you are actually discussing.

Is this the only AI productivity framework?

It's not the only way to look at it, but it is a good starting point. Companies need to implement it and adjust as they go based on feedback, the same way agile teams work.

Slides

‹ ›

Swipe or use the arrows

Many companies I see treat implementing AI as a large initiative. They plan a multi-year program and buy tools and platforms, but in many cases there is no measurable performance. Did AI work? How do we measure the result? What problem were we trying to solve? I think the real problem is that there is more emphasis on using AI than on how AI can actually help. The goal should be to provide staff with an armada of tools, start with a minimum viable plan, deploy and monitor, assess and adjust, then repeat as we collect more feedback. This doesn't mean giving everyone ChatGPT. It means matching a specific tool to a specific problem.

Treat AI like any other system

I think we should look at implementing AI in an organization the same way we look at any other system. Stop the waterfall approach and use agile. Start with the problem, and how AI can help people do their job better. Each company is different, so there is no single answer, but the question is the same everywhere.

I call the big "transform everything" program "Big T." These programs stall because people resist when whole departments change at once, and because AI models change every few months, so a 3-year plan is out of date before it ships. The alternative is "small t": targeted fixes, small pilots, and value in weeks instead of years. In the Brynjolfsson study, customer support agents with an AI assistant were 14% more productive on average.

Here is what that armada of AI tools can look like in a real company. AI can work as a clerk that does admin work like summarizing tickets, as a colleague that writes first drafts a person reviews, or as a coach that looks up a policy while the customer is still on the call. Each one is a specific tool for a specific job, and every team should have access to the ones that fit their work.

Start simple

On the technical side, start with the simplest option that works. Better prompts may be enough. If the model needs your internal documents, use RAG (feeding your own documents to the model when someone asks a question). Fine-tuning makes sense for some problems, but it shouldn't be the default. Embedding a million pages for RAG costs about $16 for the embeddings alone. Storage, integration, and maintenance add more, but cost is rarely the main problem. Old systems and messy data usually are.

A plan for one quarter can be simple. Pick a few workflows that are repetitive and costly, prepare the documents behind them in a test environment, then ship in short sprints and measure accuracy and how many people actually use it.

Overall, I think we should treat AI like any other system we build, and stop waiting for the big plan.

The slides are below.

‹ ›

Swipe or use the arrows

Note: Much of the content in this article is taken directly from or inspired by the work of Andrew McAfee of the MIT Sloan School of Management. Related reading: Kevin J. Delaney, "The low-hanging gains from AI for businesses and how managers can use AI as clerk, coach, and colleague," Charter, April 21, 2024 (an interview with Andrew McAfee); and Kristin Burnham, "How to kick-start your generative AI strategy," CIO, January 25, 2024.

Choosing a model for a generative AI product is less about picking the top name on a leaderboard and more about working through a series of decisions: what the tool must and must not do, where the data is allowed to go, how answers stay grounded, how candidate models are measured, what can go wrong, and what a pilot has to prove. This case study, written as a training exercise, walks through those steps one at a time.

To make the concepts easier to understand, let's consider a fictional example: an AI assistant that helps grant reviewers find evidence in grant applications. Each section below is one step of the discussion, applied to that example.

Step 1: Define the problem and the workflow

Reviewing a grant proposal means reading many documents, in different formats and styles, that belong to one application. A generative AI tool can speed up that review while keeping its quality. The tool helps reviewers find and summarize the information in a proposal that addresses particular review criteria. It is designed only to help find information; it cannot be used to make funding decisions or rank applicants.

The system takes the full application file and a reviewer's question, such as "What relevant experience with multisite clinical trials is described in the application?" It returns a factual answer with references to the relevant parts of the application. The reviewer checks those references and the rest of the material, and makes the evaluation.

Step 2: Let the constraints shape the architecture

Privacy, ownership, and the need to document where data comes from shape the architecture. Proposals contain sensitive information about applicants and other researchers, as well as budget details. Public commercial APIs raise privacy and compliance issues, while self-hosting open-weight models takes significant IT resources and ongoing security work.

We plan to use a hosted enterprise foundation model through a managed cloud service such as AWS Bedrock. That provides enterprise-level security and privacy: private access to the model, encryption in transit and at rest, and a commitment that our data is not used to train the model.

To keep answers grounded, we plan to use retrieval-augmented generation (RAG) rather than fine-tuning. Vector search retrieves the relevant passages from the application and passes them to the model as references during generation. RAG keeps the model from relying on what it memorized during pretraining and limits answers to what the application actually says. We may explore fine-tuning later if we find specific formatting needs, for example for citations or other structured output.

Step 3: Decide how to compare models

Candidate models will be evaluated on an internal test set of historical grant cases, using both automated metrics and expert review.

• Retrieval accuracy: Hit Rate@K and Mean Reciprocal Rank (MRR), to check that the retrieved passages are relevant.
• Generation accuracy and factuality: RAG-specific tools such as Ragas, using Faithfulness (whether each statement is supported by the provided documents) and Answer Relevance. Lexical measures such as ROUGE-L, along with semantic similarity measures, give a baseline comparison against reference answers.
• Operational viability: response time (under 3 seconds per request), tokens processed per minute, and cost per review.

Step 4: Plan for governance, risk, and fairness

The main operational risk is hallucination: the model stating facts or claiming qualifications that the evidence does not support. We plan to manage it with:

• Source-based answers: prompts that require citations to specific parts of the sources, and a default response ("Evidence not found in provided materials") when the application has no relevant information.
• Review interface: a tool that shows the source excerpts the system retrieved next to the answer they support, so reviewers can check them.
• Bias auditing: tests with varied prompts, and with applications from applicants of different backgrounds and institution sizes written in different styles, to catch unequal treatment.
• Access management: role-based access controls consistent with the existing grants management system, no retention of proposal text for training foundation models, and automated removal of sensitive data.

Step 5: Set the pilot success criteria

The tool will roll out on an opt-in basis to a pilot group of 50 reviewers. Moving to general availability depends on meeting these criteria:

• At least a 30% reduction in the time reviewers spend on documents.
• A hallucination rate under 1% in randomly sampled summaries checked by auditors.
• Reviewer ratings of at least 80% for both usefulness and clarity of citations.

The Clerk–Colleague–Coach framework, from Professor Andrew McAfee, describes three roles generative AI can play in almost any organization. The Clerk handles the busy work, like summarizing a report or filling in a template. The Colleague looks at your work and helps make it better. The Coach has seen similar cases before and suggests the next step.

Let's try an example of applying this framework in a fictional situation: a veterinary hospital. New patients need to fill long forms, and some of the questions are open text. Then the admin needs to process these forms and request the pet's previous medical report from the previous veterinarian, which takes a long time.

Clerk: The AI will receive the forms and make them digital by inserting them into the database. It will prepare a summary of the patient notes and, based on the answers from the pet owner, establish if the previous vet record is necessary, for example if important vaccine data is missing.

Colleague: The AI reviews both the admin's records request to the previous vet and the vet's notes, and suggests how to make them better. The admin or vet still makes the final call.

Coach: The AI can guide pet owners, vets and admin staff. For the admin staff, it can show them the information that is missing, the list of previous veterinarians, and how to request the records from them.

I think the Colleague role would have the biggest impact, because it makes the work much more efficient. The admin and vet can move directly to what matters.

The risk is that AI can make mistakes and overlook something important, like a history of allergy to a specific vaccine. If the staff rely only on the AI review, nobody may notice it.

A reflection on the MIT Sloan article How generative AI can boost highly skilled workers’ productivity.

The organization needs to involve all employees and make sure they are aware of the upper limit of AI, where AI excels and where a human can do better. The article calls this the "jagged technological frontier." When consultants used AI on tasks inside that boundary, their performance improved by nearly 40%, but on tasks outside it their performance dropped by 19 percentage points. So training should start there, because AI has its own limitations and not all tasks can be done efficiently with AI.

It is also important to work on a constant culture of employee feedback, to understand where AI works more efficiently than in other places. Along with that, the organization needs a culture of accountability, should reward peer training, and should encourage role reconfiguration.

From my own work, AI is great at developing test cases for an application, however it needs to have the complete context of the code. When it didn't have that context, it provided wrong answers, and I caught them by reviewing the test cases. So training should cover how to give AI the right context in the prompt, and reviewing what AI produces should be part of the training too, not something left to each employee.