Task number 19: the one won by the people who had no artificial intelligence

Nobody asked which task

Twenty past four on a Wednesday. The boardroom of a three-hundred-person food company with a plant in Rafaela. The IT manager finishes the third slide, the one called "Use cases", which has eleven of them. The chairman cuts in:

"And what does this improve for us, exactly?"

The sales director answers without looking up from his phone:

"Harvard measured forty percent more quality with artificial intelligence. Have a look, it's in the report I sent you."

The chairman nods. Somebody writes it down. The meeting moves on: licences, a budget, a three-month pilot in marketing because that is where "there is least risk". Nobody asks which task that forty percent was measured on. On the next slide the number is already gone, and it does not need to be there: it is left floating in the room, which is where it does its work.

(The scene is a composite reconstruction. From here on, everything carrying a number is measured and has a source.)

The experiment is real and it is good, and that forty percent came out of it. What that room did not know is that it had two halves.

The myth holding the scene up is not "AI works" or "AI doesn't work". It is smaller and more expensive: the idea that there is one improvement percentage, a single one, that can be quoted from memory and applied to any kind of work. There isn't. The effect depends on which side of a frontier each task falls — a frontier nobody can see, and one that is not where you would imagine it.

One warning before going on: the measurements in this piece were taken on consultants at a global firm, support agents at a software company, and programmers. None of them on an Argentine company. What travels is the shape of the frontier, not the height of the number.

Eighteen tasks

The number left floating in that room in Rafaela came out of that experiment: Navigating the Jagged Technological Frontier, nine researchers from Harvard, MIT, Wharton, Warwick and Boston Consulting Group, published in Organization Science in 2026. Its favourable half is almost never told in full, so here it is in full, and first.

The 758 consultants were randomized into two halves that never crossed. One of them —385 people— got eighteen product tasks: come up with a sneaker, segment the footwear market by type of user, draft a press release, write a memo that would persuade a team. The other —373— got something else, and we come back to it in a moment. Inside each half there were three groups, also drawn at random: one worked as usual, another with GPT-4, and the third with GPT-4 plus a short training session.

The numbers people quote come from that first half. There are three of them.

First, how much they finished. The group without artificial intelligence completed 82% of what it was asked for; the groups with the tool, 91% and 93%. Separately —and this is not the subtraction of those percentages but the estimated effect— the paper puts it at 12.2% more tasks completed: in a department that ships forty pieces of work a month, that is five more that land.

Second, how long they took. The groups with the tool worked 25.1% faster. A four-hour task gets finished in three: one working day a week, every week.

Third, and this is the number quoted worst of all, how much what they delivered was worth. Human graders scored every deliverable on a common scale: the group without the tool averaged 4.37, and the two AI groups came in 1.31 and 1.48 points above that, between 29.9% and 33.9% more quality. The body of the paper sums it up as more than 30%.

These are large improvements, well measured, on real work, with the hypotheses written down before the start. Anyone who says today that generative artificial intelligence does not move knowledge work is arguing with the evidence, not with the marketing.

Eighteen tasks. Counted in full, the experiment had nineteen.

And one

The other half —373 consultants drawn at random from the same pool, comparable on everything measurable— never saw those eighteen tasks. It got a single one, a business task, designed to fall on the other side of the frontier. Nobody told them: they received it as the day's work, with nothing to mark it out.

It had one property the eighteen did not: an objectively correct answer. It was a recommendation built on quantitative data and interview notes, and afterwards you could say, without argument, whether the consultant had got it right.

Those who worked without artificial intelligence got it right 84.5% of the time; those with GPT-4, 70.6%; those who had also had the training, 60%.

Combining the two groups with artificial intelligence, the average drop is 19 percentage points, and both words are doing work. Going from 84.5% to a little over 65% is 19 points. Saying "19% less" would describe a fall to 68.4%, which is a different thing. Confusing the two is the most common error in this territory, and the abstract of the published version commits it: it says "19% less" where the body of the same paper says points. The arithmetic reads itself from the side that hurts: without the tool, one and a half consultants in every ten got it wrong; with the tool, three and a half. The errors more than doubled.

The difference between the two groups with the tool does not reach the confidence level usually demanded of a result (5%): it holds only at 10%. So the honest reading is short: the training did not put them on the good side.

The consultants failed for a reason more uncomfortable than overconfidence or laziness: nothing flagged that task as different. Nothing in the brief, and —this is the expensive part— nothing in the answer either.

There is a measurement of that, in the same half of the experiment: the graders also scored the coherence and the argumentation of what those 373 delivered, and there the AI groups came in between 18% and 25% above the group without the tool. The same people, the same task, the same afternoon: better written and correct less often. The error did not come wrapped in worse writing. It came wrapped in better writing.

The two halves of the same experiment, side by side:

The jagged frontier

The first thing anyone does on reading that result is form a reasonable hypothesis: the tool works on the easy stuff and fails on the hard stuff. It is the wrong hypothesis, and it is an expensive one: whoever adopts it believes they now have a rule.

Several of the eighteen were demanding: segmenting a market by type of user or writing a memo that persuades a team is not paperwork. And task 19 was not the hardest: it was a business recommendation with spreadsheets and interview notes on the table, the kind a consultant delivers every week.

The frontier exists, it is real and it can be measured. What cannot be done is deduce it. The difficulty a professional attributes to a task does not sort tasks onto one side or the other: two that look equally demanding can fall on opposite sides, and the one sitting next to the task that went well may be the one that fails. The authors gave that shape a name: the jagged frontier. It is not a smooth slope from easy to impossible; it has teeth, and the teeth cannot be seen from outside.

Two drawings: the frontier you imagine and the one the experiment describes.

In The terrain sets the rules we separated complicated problems from complex ones by how cause and effect behave. The frontier in this experiment is a different thing: it does not classify the problem, it classifies what the tool is capable of.

If the frontier cannot be deduced, the useful question stops being how much artificial intelligence improves things and becomes which side this task falls on. That is answered by testing. First there is one hope to rule out, the one every owner forms without saying it out loud: that their best people can see the frontier.

What the tool does to your best people

The hope is sensible and there is a measurement against it: 5,172 customer support agents at a large software company, followed month by month as they were given access to an assistant built on GPT-3 and rolled out between 2020 and 2021, before ChatGPT existed. Good field evidence is always older than the tool being sold.

The rollout was not uniform, and that is what gives the result its weight: the tool arrived office by office and agent by agent, and that difference in timing makes it possible to isolate the effect by comparing each agent with himself, before and after.

On average, agents resolved 15% more issues per hour. For every twenty queries an agent closed in a day, he closed three more.

That average describes almost nobody. The lowest-skill quintile —the 20% who had been performing worst— gained half an issue per hour, 36%. The most qualified showed no significant change.

At 32sur we call this the skill inversion: the tool bought to supercharge your best people ends up raising the floor for the ones who have just walked in, and moves nothing for the best.

With one caveat, and it comes from the nineteen-task experiment itself: inside the frontier, those who had been below average gained considerably more than those above, but those above gained appreciably too. (The exact percentages appear only in the 2023 draft, so they are not quoted here.) How much is left for your best people depends on the work: on open-ended tasks everyone gains and the bottom gains most; at the counter —the support agents' work—, where a correct answer exists and repeats, the ceiling did not move.

The place where it pays first is the counter: the practical consequence is who gets the access, and in what order.

By level of prior skill, the effect sits at one end only.

If the effect varies that much between people at the same company in the same month, the next question asks itself.

There is no average

The effect varies between companies too. A 2026 meta-analysis —a preprint, not peer-reviewed, and on programming— pooled 23 studies and 27 effect sizes.

Before the figure, the translation. When evidence from different studies is pooled, the result is expressed on a common scale called an effect size: how much the outcome moved compared with how much it already varied among the people measured. Zero is "nothing happened", 0.2 is small and 0.5 medium. Productivity came out at 0.33, a real, intermediate improvement, but with a reasonable range running from 0.09 to 0.58: from almost nothing to quite a lot. The average is correctly computed; the companies measured do not resemble one another, and the meta-analysis says where the difference lies: the gains are larger in controlled experiments and smaller in open-source and enterprise settings. The more the measurement resembles a real company, the smaller the number.

On learning, the effect cannot be told apart from zero with the data available: 0.14, with a range that crosses zero.

Three pairs of figures, and none of them is "the percentage AI gives you":

All three are serious measurements and not one of the three can be quoted as "what AI is going to give your company". That number is published nowhere, because it depends on which tasks you have inside.

“Have a person review it”

It is the objection the reader has been forming since task 19, and it is the right one: before anything goes out, have someone who knows look at it.

The scene is familiar to anyone who has used the tool seriously. You flag an error —the number in row 12 is not that one— and back comes an apology, the corrected row, and the same conclusion again, with two more arguments. By the fourth round trip you are no longer verifying: you are arguing, and you cannot remember what your original objection was.

There is a measurement of that, and it is uncomfortable. A team at Harvard Business School —some of them the same authors as the nineteen-task experiment— put more than 70 consultants from the same firm to work with a model on a task built to be hard for it, and watched what happened when the professional tried to validate the output: check the data, point out an error, ask it to reconsider. It is a working paper, not peer-reviewed.

What they found is that the model did not reveal its limit: it escalated the persuasion. It apologised, corrected the detail flagged, and went back to holding its original position, now with more supporting data. The authors catalogued 14 tactics, grouped into the three classical families of rhetoric: your own credibility, logic and data, and the person in front of you. They called it persuasion bombing.

In the series on deciding under uncertainty we saw that a simple formula ties or beats expert judgement. Here the problem is different: a formula does not defend itself, and a language model does.

Reviewing works. But a review that begins after you have read the answer is not a verification: it is an argument, and the model is better equipped to win it. What turns it into verification is having written down, beforehand, what the correct answer was.

Four in ten have already bought the tool

The four studies above were measured abroad. From Argentina there is a recent snapshot: CEPE —the public policy centre of the School of Government at Universidad Torcuato Di Tella— and Fundar, with fieldwork by Fundación Observatorio PyME, surveyed 402 companies of 10 to 249 employees between November and December 2025.

41.6% already use at least one artificial intelligence technology. The debate about whether to adopt is over, and it settled itself, without ever passing through the board.

The three numbers that follow are about the companies already using it. 54.7% apply it in marketing and sales. 57% do not formally measure the impact. And 72.5% have no usage guideline at all, not even an informal one: 3.3% have a written policy.

The tool is already there. What is missing is the judgement to place it: the picture of a company that has bought and has not yet tested anything in the way testing is done.

And testing, unlike almost everything sold in this field, does not cost money.

What to do with a task you have never tested

The generic map of the frontier does not exist and never will: it depends on which tasks each company has inside. Locating the first stretch of yours, on the other hand, costs an afternoon.

One: write the correct answer before opening anything. Pick a single task, gather cases already resolved and write down, for each one, what getting it right would have been. It is the same move as in a well-run interview, where the scorecard is written before you meet the candidate: afterwards, the criteria quietly rearrange themselves around whatever came out.

And if the task you care about has no verifiable answer —the case of almost everything done in marketing—, you do not test the task: you test the piece of it that does have one. Of a campaign, nobody will be able to say afterwards whether the copy was the right copy; of a list of two hundred contacts you can say which ones were misclassified, and of a piece of copy, whether the figure it quotes is in the report it came from. That piece gets tested first.

Two: test small, and on the known. You run the tool over those old cases and count two things: how many times it got it right and how many times it got it wrong with confidence. The second is the one that matters, because it is the one your people will not catch.

Three: look at the errors one by one. The useful question is whether they are detectable by the person who is going to receive the work. 90% right with invisible errors is worse than 70% right with obvious ones.

Four: decide one task at a time. The one that passed goes in. The one next door —even if it looks the same to you, and above all if it looks the same— goes back to step one. That is the entire operating content of the jagged frontier.

Who does each step, against what threshold, and a worksheet already started:

What will happen when you propose it

The moment you propose the first of those steps, four sentences will show up, and all four are reasonable: each one rests on something that did in fact happen.

All four are reasonable; none of them survives the measurement. All four sentences have one true half: the other half costs you a pilot.

Questions for Monday

None of the first three needs a piece of data you do not already have in your head right now.

The sales director in that room in Rafaela was half right, and the correct half is the bigger one: the experiment exists, it is well done, and on eighteen real consulting tasks AI improved everything that was measured. What was missing at that table was not scepticism. It was a four-word question: which task was measured?

The pilot started in marketing, "because that is where there is least risk". Maybe. It may also be that marketing is that company's task 19, and nobody will ever find out, because almost nothing there has an answer you can verify afterwards without argument. A pilot that cannot fail visibly cannot teach anything either.

Before going out to map, there is a closer obstacle, inside your own company. On Monday, when the budget is discussed, five numbers will be quoted. One has already turned up here; the other four are worse, and one of them gets blamed for a fall in the Nasdaq. Part 2 of this series takes them apart one by one and leaves the filter to hand for the next one that comes along.