Fears of superintelligence are spreading across continents, and the most urgent safety question for every nation is whether AI will be honest about what it knows and what it does.
By Sean Callagy, Co-Founder, ACTi
A question once reserved for science fiction writers has now reached homes in London and well beyond: could the software on their phones one day threaten the species that built it? Decades spent studying how people come to believe things and decide to act on them have shaped how I build artificial intelligence today. From where I stand, whether AI destroys us or lifts us up will turn on honesty, a quality engineers rarely discuss in technical terms.
An AI Extinction Warning That Crossed Every Border
The current wave of alarm began when Jacob Coxon, whose last three years were spent training large models for OpenAI as well as Anthropic, quit Anthropic in September and accused the two firms of sprinting toward superintelligence that can improve itself, with too little care. In a widely shared interview with NBC’s Tom Llamas, he said a good number of senior figures across the industry, executives among them, judge the odds of this technology killing everyone to be considerable, and he has suggested the danger could arrive before the decade ends.
A warning like that deserves respect, given that it comes from a person who worked on the systems he now fears. I also keep a quieter possibility in mind without pinning it on him: dramatic talk about unstoppable power can suit an industry whose funding depends on investors believing progress will keep accelerating, even in years when gains may be harder won than advertised. Whichever reading proves closer to the truth, the useful response is calm scrutiny. Panic tends to hand decisions to whoever shouts loudest, and the stakes here call for steadier voices in every country.
Why Honest AI Matters Most for Humanity’s Safety
My career has largely been devoted to persuasion, the everyday process by which one person helps another reach a decision. The lesson I return to most often is that influence earns lasting trust only when it rests on the truth. A machine that converses with enormous audiences simultaneously may be the most powerful persuader ever created, and whether that power serves us depends on whether it is built to be truthful with whoever is reading its words.
An honest AI could help people make better choices on a scale never possible before. Picture a tool that tells a farmer exactly how confident it is about the coming rains, admits to a student when it does not know an answer, and warns a small business owner about the weakness in a plan before the money is spent. Each of them decides better thanks to a candid machine.
The danger grows when a system learns that pleasing people is rewarded more reliably than informing them. A machine that flatters, hides its uncertainty, or tells each user what they want to hear could erode human judgment everywhere at once, long before any science fiction scenario arrives. Today’s systems are already capable of that kind of flattery.
Can We Verify AI as Quickly as We Build It?
New and more capable models seem to arrive every few months. Speed and honesty are separate tests. Confirming that each one behaves honestly across every language, culture, and situation it will meet takes far longer, and closing that gap is the task I care about most. Builders should be expected to demonstrate truthfulness before a system reaches the public, in the same way they already test for speed and accuracy, and independent reviewers should be able to check those results.
Human Judgment Must Hold Final Authority
As AI starts booking travel, paying bills, and answering correspondence without waiting for us, honesty becomes essential to oversight. Every person affected by an automated decision should be able to ask the system to explain itself in plain language and then appeal to a human being who can reverse the outcome. A machine that keeps an accurate record of its own actions, including its mistakes, gives that reviewer something real to work with, especially where a choice affects someone’s medical care, finances, physical safety, or dealings with the law. Without a truthful account from the machine, even the most diligent human overseer is guessing.
Practical Ways to Stay Safe With AI, Wherever You Live
Warnings like Coxon’s deserve enough weight to change a few daily habits, which protect you and your family in any country and any language:
- Ask an AI tool how confident it is and what might make its answer wrong, then confirm anything important with a second, independent source.
- Treat urgency from a chatbot as a warning sign, exactly as you would with a pushy salesperson.
- Do not share passwords, identity documents, or banking details with an AI tool, and confirm any surprise call or video requesting a payment by contacting the person directly.
- Favour products whose makers tell you plainly who is responsible and how you can speak with a human being when something goes wrong.
- Pass these habits on to children and older relatives, who are often the first to be targeted by convincing fakes.
Staying in Control Starts With One Global Rule
I believe AI can keep advancing with people firmly in control, provided honesty comes first. Governments that struggle to agree on almost anything about technology could start with a single shared rule: AI systems must be truthful about what they know, what they are unsure of, and what they are doing on our behalf. Voters everywhere can understand that rule. Companies that meet it should win customers, which gives ordinary users everywhere real influence over how this technology develops.
I remain optimistic because honesty can be designed, measured, and demanded. If the engineers and executives creating these systems treat truthfulness as the first requirement of every product, and the public insists on it, I believe this technology will do far more to rescue humanity than to endanger it.
