When to let AI do a job on its own
Key takeaways
- Every job starts in draft. Move it to act only on evidence from a few weeks of drafts you approved unchanged.
- Move one narrow job at a time, note the date, and keep checking a sample of what it sends.
- Moving a job back to draft is the system working. Anything that moves money stays with a person.
Short version: let an AI do a job on its own only after you’ve watched it draft that same job for a few weeks and approved nearly every draft unchanged. Move one narrow job at a time, on that evidence, keep checking a sample afterwards, and be ready to move it back the moment it slips.
What are AI autonomy levels?
They describe how much an AI is allowed to do without a person. Technical frameworks list lots of levels. A small business needs three, and they’re the ones in act, draft or never:
- Never. The AI has no way to do the job at all. Moving money sits here permanently.
- Draft. The AI prepares the work and a person approves it before it goes anywhere.
- Act. The AI does the job alone, within limits, and a person checks a sample afterwards.
Every new job starts in draft. The real question is when a job has earned the move to act.
Human in the loop vs human on the loop
Those two phrases are a tidy way to see the difference. In the loop means a person approves each piece of work before it happens. On the loop means the AI acts, and a person watches what it did and steps in when something looks wrong.
Moving a job from draft to act moves a person from in the loop to on the loop. It doesn’t remove them. Somebody still reads a sample of what went out, still owns the result, and can still switch it off.
The evidence to look for
Gut feel isn’t enough, because the AI will feel reliable long before it is. Look for this in a few weeks of drafts for one specific job:
- You approve it unchanged, again and again. If you keep tweaking a word here and there, it isn’t ready. If you keep fixing facts, it’s nowhere near.
- The cases were varied. A run of easy cases proves little. You want to have seen the awkward ones too: the cross customer, the odd order, the question that doesn’t quite fit.
- It handed over when it should. When a message didn’t fit, did the AI say so and pass it to a person, or did it guess?
- A wrong answer is cheap and easy to put right. If one mistake costs an apology, it’s a candidate. If it costs money or a customer, it stays in draft.
This is why keeping a record of what you approved, changed and rejected matters. Without it, you’re deciding from memory.
Move one job at a time
Pick the narrowest job you can describe. Not “customer emails”, but “replies to where is my order? when the tracking shows it’s on its way”. Move that, note the date, and leave everything else in draft.
Say you sell garden planters online. After a few weeks you might find the tracking replies are always right, the product questions are mostly right but sometimes guess at sizes, and the damage claims vary too much to judge. Only the first one moves. The second stays in draft until the product facts are fixed, and the third may never move at all.
Autonomous AI customer service: what never moves
Some jobs stay out of the act pile however good the drafts look:
- Refunds and anything else that moves money. The reasons are in Why your AI should never issue a refund.
- Admitting fault, or promising something you haven’t offered before.
- Complaints, disputes and anything a customer has escalated.
- Anything new. If the AI hasn’t drafted it before, it hasn’t earned the right to do it alone.
After the move: keep watching
- Check a sample. Read some of what it sent each week. Make it a fixed habit, not something you do when you remember.
- Move it back on the first real slip. Moving a job back to draft isn’t a failure. It’s the system working.
- Look again after changes. A new AI model, a change to your returns policy or a new product range can all change how well it does. Treat each one as a reason to read more of its work for a while.
- Keep the off switch handy. If something goes badly wrong, you should be able to stop it in minutes. There’s more in Every AI needs an off switch.
Not sure which jobs in your business are worth handing over? Grafto’s AI Tools Report works it out from a short questionnaire.
Frequently asked questions
What is an AI autonomy ladder?
A way of describing how much an AI may do without a person: not at all, drafting work for a person to approve, or acting alone within limits. A small business can manage with those three rungs.
What is the difference between human in the loop and human on the loop?
In the loop means a person approves each piece of work before it happens. On the loop means the AI acts and a person watches what it did, checks a sample and steps in when something looks wrong.
Is autonomous AI customer service a good idea for a small business?
For a few narrow, low-risk jobs, such as tracking replies, once a few weeks of drafts show it gets them right. Not for refunds, complaints or anything new. Keep a person on the loop and an off switch close by.
Questions, or want a tool we don't have yet? Email hello@grafto.co.uk — a real person replies.