Skip to main content
Fleur Lamont Logo Fleur Lamont

The demo moved. The decision did not.

AI
AI Small Business Agentic Vendor Risk Procurement

An open laptop showing an out-of-focus chat interface on a wooden meeting-room table, with a printed sheet of notes, a mug of tea and a stack of supplier papers beside it

A supplier sits down with you, opens a laptop, and walks your week through a demo. The set piece runs well. It reads an email, drafts a reply, and books the follow-up before you have finished your coffee.

Then the meeting turns to what you would actually sign.

Something moved in the last few months, and it moved in the demo. The tools sitting behind that laptop got quicker, cheaper and more willing to act on their own. That makes the demo better, and it makes the decision harder, because the thing you are being asked to buy now does more than the thing on offer this time last year.

The five questions worth asking a supplier have not changed. They matter more, not less. That is the argument of this piece, and the rest is the evidence.

What actually changed

Three releases in September are worth knowing, because each one moved the demo without touching the decision.

OpenAI, GPT-6 Sol and GPT-6 Luna, September. OpenAI carried the abilities of its largest model, in professional work, factuality, coding and computer use, down into two faster, cheaper models, and cut the API price of both by half against the versions they replace. The intelligence you see in the flagship now sits at a price a firm like yours can run every day.

Anthropic, Claude Sonnet 5.5, 28 September. A mid-tier model that runs more than 30% faster and costs up to 30% less per task than the one before it. On an agentic coding test, its score rose from 10.3 to 70.6. A model from the middle of the range now finishes long jobs on its own that, a year ago, needed the biggest model on the market.

Google, Gemini 4 Argon, 30 September. Google raised the model's single-run output limit from 64,000 tokens to one million, so it can produce a long job in one pass, and gave it the ability to find, validate and patch software flaws unattended. It is going out first to a set of trusted cyber defenders, and Google is releasing it to them without the cyber guardrails it applies elsewhere.

The pattern across the three is the same. The mid-priced tools caught up to last year's top of the range. The price of a token fell. The willingness to act unwatched went up. None of that answers a single question on your list.

Read the three together and the shape is clear. Last year the good enough tool was expensive and waited to be asked. This year it is cheaper, and it is built to run a job from start to finish without a person watching the middle of it. The labs call that agentic. For a supplier it means a better demo. For you it means the thing you are buying does more on its own, which is exactly the property that decides how much the questions matter.

The five questions, run against the new tools

The questions were written for a market where the tool sat in a chat box and waited to be asked. The tools have moved. Run the questions again.

What data of mine does this touch, and where does it sit? A straight answer now names the model, the data it reads, and the region it is held in. That answer is harder to give than it was, because a model that reads a whole document in one run, or drives a browser on your behalf, can touch a folder, a screen or a mailbox in a single pass. The surface is wider than last year's chat box. Ask what it reads, where that sits, and which country holds it.

What happens to what I put in, and who at that company can read it, under what rule? A straight answer now is a policy you can point at, by name, rather than a reassurance on a call. The bar is higher than it was, because an agent that runs on its own sends more of your working life through the tool than a chat box ever did, and because the models now sit inside larger systems that a salesperson may not be able to describe from memory.

If it gets something wrong, how would I find out? A straight answer now is your own detection, not the supplier's. Google bundles misalignment monitors and prompt-injection defences into Argon, and both of those protect the model on Google's side. They do not tell you when your agent has filed the wrong receipt. A tool that admits a mistake only when it already knows it made one is not a control on your business. Ask how each run is logged, and who reads the log.

If I want to turn part of it off, how long does that take? A straight answer now is a number of hours and a named person. The exit route matters more than it did, because the new agents run in the background. Turning off a chat box takes a click. Turning off an assistant that has been handed a browser, a mailbox and a standing task is a small project, and you should know its length before you sign, not when you want out.

What does it cost when I actually use it, not when it was priced? A straight answer now is the cost of a real task, not the price of a token. OpenAI halved the per-token price of its two new models. Google raised the tokens one model can spend in a single run from 64,000 to a million. The price of a token went down and the number of tokens in a job went up, and on your invoice those two moves pull against each other. Ask for the cost of one job you can name.

One live product makes the point better than any of the three releases. OpenAI sells a feature called company knowledge to its business, enterprise and education customers. It reads across a company's connected apps, Slack, SharePoint, Google Drive, GitHub and the rest, and answers with a citation on every claim. It is a genuinely useful thing. Read its own description closely, though, and it answers most of the list above by itself. It reaches only what each person can already see, so the access question is answered in a sentence. It reads, and it does not send, file or schedule anything under your name, so there is no approval to give and no exit route to price. And the data stays in OpenAI's cloud, on their terms. None of that is a criticism. It is the shape of the answer, and you can read it off the page before the meeting.

Where the gap still sits for a firm like yours

The controls got better and the gap is still open. For a firm like yours, whether that is twenty people in an office or two hundred on a site, it sits in five places.

Where the data sits, and who can read it, is a sentence in a policy you have to ask for by name, in a meeting, from a salesperson who may not hold it.

When an agent gets a step wrong, nothing rings a bell. The error is found late, by a customer or by an accountant, and by then it has been copied into a system you cannot easily unwind.

The off-switch is a project rather than a switch, and its length is the exit route you are buying into.

Nobody inside the firm owns the tool yet, in a way the accounts, the insurance or the health and safety file is owned. When the agent acts on your behalf and something goes wrong, the question of who answers for it has no name attached.

The bill arrives per task, after the work, and it is set by how many tokens the tool chose to spend, not by the seat you bought. The per-token price fell; the per-task bill is a different number, and it is the one that lands in your accounts.

None of those five is a feature question. Each one is a question about how the business runs after the demo ends.

What to do before you sign anything

Take the five questions into the meeting as a list, and write the answers down. Ask each one twice if the first answer is a sentence you could have written before the meeting.

  1. Which of my data does this touch, and where does it sit? You want the model, the data it reads, and the country it is held in.
  2. What happens to what I put in, and who can read it, under what rule? You want a named policy, not a promise.
  3. If it gets something wrong, how would I find out? You want your own check, not the supplier's monitor.
  4. If I want to turn part of it off, how long does that take, and who owns it? You want a number in hours.
  5. What does one real job cost when I run it? You want the per-task figure, not the per-seat price.

The five come from a shorter piece on LinkedIn, which asks the same questions in the same order. This piece is the longer half behind it: what the tools shipped this autumn, and why the questions outlast the tools.

The tools will keep moving. The questions will keep separating the suppliers who built something from the suppliers who bought something. Ask the five, and watch which one is hardest to answer.

This piece offers a way to read the market, not advice on any one supplier.

Sources