An AI Support Agent on Your Store: What It Can Answer and What It Must Never Touch

Support automation has changed shape. The old chatbot matched keywords and offered you a menu; the current generation reads your order data, understands the question, and can take actions on the customer's behalf. That is genuinely more useful and genuinely more dangerous, and the dangerous part is not hallucination — it is scope.

An agent that can read an order is helpful. An agent that can cancel one is a liability the moment its confidence exceeds its evidence. This is the framework we use to decide where that line goes.

Three Tiers, Not Two

Most teams think in terms of the bot handles it or a human handles it. Three tiers works far better, because the middle one is where most of the value is and where all of the risk lives.

Tier 1: Read and Answer — Deploy Confidently

Questions answered from data the agent can look up, where being wrong is embarrassing rather than costly.

• Order status and tracking. The single largest ticket category in most stores, and almost entirely deflectable.
• Delivery estimates for a given product and destination
• Stock availability and expected restock dates
• Policy questions — returns windows, warranty terms, shipping thresholds — answered from your actual published policy, not from the model's general knowledge
• Product specification questions answered strictly from your catalogue attributes
• Account questions — where to find an invoice, how to change an address

Note the qualifier that appears twice: from your data. An agent answering a returns question from its training data rather than your returns policy will be plausible, confident and occasionally wrong in a way that is legally binding on you. Ground every policy answer in retrieved text from your own content.

Tier 2: Act, With Guardrails

Actions that change something, permitted only inside explicit, mechanical limits — enforced in your code, not in the prompt.

• Change a delivery address — only before dispatch, only for the authenticated account holder
• Start a return — only inside the returns window, only for eligible products, only generating a label your existing process already permits
• Cancel an order — only before fulfilment begins, only below a value threshold you set
• Apply a goodwill credit — only within a hard per-interaction cap and a rolling per-customer cap
• Update marketing preferences — genuinely low-risk, and a common irritation to customers

Tier 3: Never — Hand Over Immediately

Categories where the agent's only correct action is to collect context and escalate.

• Anything involving payment details or card data. No exceptions, no clever exceptions.
• Fraud, chargebacks and disputes — an adversarial counterparty is not a support conversation
• Safety, injury, allergy, adverse-reaction or product-recall reports. These are regulated, time-sensitive and often legally reportable.
• Medical, dosage, legal or financial advice, however conversationally the question arrives
• Complaints about the company, especially anything mentioning legal action, regulators or the press
• Distress of any kind. A customer who is upset needs a person, and every minute an automated system spends on them makes it worse.
• Anything about a minor's account or data
• Bulk, wholesale or B2B pricing negotiations — a commercial conversation, not a support one

Write this list before you write the prompt. It is the artefact your legal, compliance and support leads should all sign, and it is the thing that will still be right in a year when the model underneath has been replaced twice.

Escalation Is the Feature

Most deployments fail at the handover rather than at the answer. Get these right:

1. Escalate on repetition. If the customer asks the same thing twice, the agent has already failed. Hand over on the second attempt, not the fifth.
2. Escalate on sentiment, not just on keywords. Frustration is detectable and it is the earliest reliable signal.
3. Always offer a visible way out. A customer who cannot find the human is a customer writing a review about it.
4. Carry the context across. The agent must hand the human a summary, the order in question and what has already been tried. Making the customer repeat themselves is worse than not automating at all.
5. Escalate on low confidence. An agent that says I am not sure, let me get someone is more trusted than one that guesses well most of the time.

The Metrics That Get Gamed

Vendors report containment rate — the share of conversations that never reached a human. It is the easiest metric in support to make look good, because a conversation the customer abandoned in frustration counts as contained.

Measure these instead:

• Resolution rate — did the customer's problem actually get solved, verified by no follow-up contact within 72 hours?
• Escalation quality — of conversations handed over, how many did the human resolve in one reply? Poor context transfer shows up here first.
• Cost per resolved contact, compared against your human baseline — the number that justifies the project
• Post-interaction conversion and repeat purchase rate for customers who used the agent versus those who did not. If this is negative, containment is meaningless.
• Complaint rate mentioning the agent. Track it explicitly; it will not surface on its own.

Before You Switch It On

A short pre-launch list that catches most of what goes wrong:

1. Build an eval set of 100 real historical tickets with the correct outcome written down, and run every prompt or model change against it. Without this you are changing things by feel.
2. Red-team it. Try to get it to refund more than the cap, reveal another customer's order, or give medical advice. Someone will; better it is you.
3. Check what it says about competitors, prices and promises. A support agent promising next-day delivery you do not offer has created an obligation.
4. Confirm the data-protection position. What customer data leaves your systems, to which vendor, under what retention terms, and is that documented?
5. Label it clearly as automated. Beyond any regulatory obligation, undisclosed automation is a trust cost you do not need to pay.
6. Give it a kill switch that a support lead can operate at 2am without an engineer.

What This Is Worth

Handled well, Tier 1 alone typically removes a substantial share of ticket volume, because where is my order dominates most stores' inbox and is entirely answerable from data you already hold. That is a real saving, and it arrives without touching anything risky.

Tier 2 adds more, and adds the possibility of a bad day. Tier 3 is where the reputational damage lives, and it is free to avoid — you simply do not build it.

The stores that regret this project are the ones that started at Tier 2 because it demoed better. The ones that are pleased with it shipped Tier 1, measured it honestly for a quarter, and expanded from evidence.

We build these against live stores through our AI and machine learning practice, and the wider question of where automation belongs in a business is what our AI integration work is for. In regulated settings — our healthcare clients among them — Tier 3 is longer and the discipline matters more, not less.

Start With the Scope Document

Before any tooling decision, write the three tiers for your business and get support, legal and commercial to sign them. We will draft it with you and build against it. Talk to our team; we reply within a business day.