I think how AI can contribute to a problem should follow a standard framework in any company. Too often we start with the AI tool and only later ask what problem it solves. A short questionnaire about the problem, translated into measurable productivity, is a better start. Sinan Aral's AI productivity framework covers the general idea, and people should be able to map the problem they are trying to solve onto it.
The three parts of the framework
The framework has three parts.
- Improvements are about what gets better: efficiency (cost and time saved) and quality (fewer errors and more creativity).
- Complementarities are about what decides how much you gain from working with AI, which depends on the task, the person's skill, and how much the person follows and trusts the AI.
- Synergy vs. augmentation asks whether AI should replace people or support them. Aral splits that into agent effects, meaning who performs better alone, and task effects, meaning whether the task is to decide something or to create something.
Why use a framework at all?
First, it makes productivity measurable. Second, it standardizes the process, so new people follow the same procedure when they onboard. It also lets different teams compare AI projects on the same terms, it shows where a person has to stay in the loop, and it gives a clear reason to say no when AI doesn't fit the problem.
The questions
The questions should cover all three parts of the framework:
- Improvements: what will improve (time, cost, errors, quality), and how will we measure it?
- Complementarities: what kind of task is this, what skills does the team bring, and how much will people trust the output?
- Synergy vs. augmentation: can AI do this better alone, or should it support a person? Is the task deciding or creating?
How it maps to my own work
AI is very dependent on context. For code testing and QA, as long as the skills are defined and AI is instructed what to do, I think AI will do a better job than me. QA is also both kinds of task. Writing the tests is creating. When AI runs the tests and builds the list of tasks that need to be done from the results, it is deciding what needs work. AI is also very good at research and collecting the necessary information before a human makes the final decision. But I am not comfortable letting AI make health-related decisions for me. There, AI supports me and I decide.
Security scan findings (Fortify, in my case) are somewhere in between. I ask AI to recommend with an explanation of why. After reviewing the recommendation, I ask how it plans to fix each issue. After reviewing the plan, I approve or disapprove. AI does the analysis and I keep both decisions.
I also found that an interview-style conversation with AI is much more effective than one big report. I say something like "give me the first 10 and let's go through them one by one." Small batches keep my attention fresh, and it's hard to rubber-stamp something you are actually discussing.
Is this the only AI productivity framework?
It's not the only way to look at it, but it is a good starting point. Companies need to implement it and adjust as they go based on feedback, the same way agile teams work.
Slides
