Artificial Intelligence
Using AI Assistants Well: Prompts, Limits and Verification
AI assistants have moved from novelty to daily tool for a very large number of people, largely without anyone explaining how they work well enough to use them safely. The result is two failure modes: treating the output as an oracle, or dismissing the tool entirely after one confident falsehood. Neither is necessary. What follows is a practical account of where these systems are genuinely strong, where they are structurally weak, and how to work with both.
What the model is doing
A language model predicts likely continuations of text. It has no database lookup step and no internal sense of certainty about facts. Fluency and accuracy are separate properties, and the model optimises the first.
This explains the single most important behaviour to anticipate: a plausible, well-structured, entirely incorrect answer, delivered with the same tone as a correct one. There is no stylistic tell.
It also explains what these systems are excellent at. Transformation of text you provide, summarising, restructuring, drafting, translating, explaining a concept in different words, and generating options are all tasks where fluency is the point and where you supply the ground truth.
Prompts that reliably work better
Specificity, context, format
Give the role and the audience. Asking for an explanation of a tax rule for a first-time filer produces a different and more useful answer than asking for an explanation of a tax rule.
Supply the source material rather than relying on recall. Pasting the document, the code, the policy or the data and asking for work on it moves the task from recall, where the model is weak, to transformation, where it is strong.
State the output format and the constraints up front: length, structure, tone, what to exclude. Then iterate by criticising the draft rather than restarting. Feedback on a concrete draft steers far better than a longer initial prompt.
- Name the audience and the purpose
- Paste the source instead of relying on memory
- Specify format, length and exclusions
- Iterate on the draft rather than re-prompting from scratch
Where it will fail, predictably
Precise figures, dates, citations, legal clauses, prices and statistics are the highest-risk outputs. These are exactly the details that look most authoritative and are most often fabricated or outdated.
Arithmetic over more than a few steps, counting items in a long list, and reasoning about very recent events are similarly unreliable unless the tool has explicit calculation or search capability and visibly uses it.
Anything where being wrong has consequences — medical, legal, financial, or safety — needs a human expert in the loop. Use the assistant to prepare questions and understand vocabulary, not to make the decision.
A verification habit that takes seconds
Ask for sources and then open them. A real citation resolves to a real page containing the claim. A fabricated one either does not resolve or resolves to something that does not say what was claimed.
Cross-check any number that will be reused. If a figure is going into a document, a spreadsheet or a decision, it needs a primary source regardless of how confident the answer sounded.
Re-ask the same question in a fresh session with different phrasing. Consistent answers across independent attempts are weak evidence of correctness; inconsistent ones are strong evidence of a problem.
Privacy and what you paste
Assume the text leaves your device
Unless you are running a local model, your prompt is transmitted and may be retained. Treat the input box like a message to a third party, because that is what it is.
Do not paste identity documents, full account numbers, private keys, client contracts under confidentiality, or medical records. Redact identifiers and paste the structure you need help with instead; the model does not need real names to fix your paragraph.
Check the retention and training settings of the tool you use. Most offer a way to exclude your conversations from training, and business tiers usually make it the default.
The right mental model
Treat it as a fast, tireless, widely-read assistant who is occasionally and confidently wrong, and who cannot tell you which times those are. You would happily hand that person a draft to improve. You would not hand them a decision.
Used that way it is a genuine productivity gain. Used as an oracle it is a liability, and the difference is entirely in the verification habit, not in the model.
