“Man’s reach exceeds his imagination!”
The magician shouts it from the stage in Christopher Nolan’s The Prestige, at the peak of his greatest illusion. The crowd sees only the wonder. What they never see is the cost, paid quietly, just out of view. The whole film turns on that gap between the marvel the audience applauds and the thing it actually took to produce it.
I think about that gap every time someone asks whether we can trust AI.
The reflex is to answer yes or no (maybe). But the word is carrying too much. “Trust” bundles several different things together, and most arguments about AI are really arguments about which of them we mean without saying so. Pull them apart and the question gets a lot more useful.
Start with the sense that holds up fine. It’s the plain confidence that a tool will do its job. A bow doesn’t put every arrow through the same hole, but if the spread stays inside a group you can work with, you trust it. Every real instrument comes with a tolerance. Trusting one has never meant it behaves identically every time. It means the variation is bounded and you know the bounds. For the predictive models I’ve worked into real systems, that was the whole exercise: characterize the error, find the range where the thing is reliable, and use it there. That kind of trust is calibration by another name, and it survives perfectly well.
Large language models and agents complicate this, though not in the way one might first think of. The trouble isn’t that they vary. Plenty of trustworthy tools vary. The trouble is that their output is open-ended, so the tolerance is much harder to pin down. What counts as acceptable spread on an analysis, an answer, a piece of code. With a bow you can see the group on the target. Here you often can’t, and deciding what “within tolerance” even means is the hard step almost everyone skips because of the “wonder” it can do. These tools are genuinely powerful. They do things I understand but couldn’t do, or at least not as fast, and they fail at the edges in ways I find quickly. The functional trust is still on offer. You just have to work much harder to earn it.
Then there’s a second sense of trust that doesn’t belong here at all. It’s the trust you give a person’s judgment, the kind that assumes something on the other side is aware of what it is doing and answerable for it. A model has none of that. When people say they trust the AI and mean this, they are handing it a responsibility it cannot hold. This is the trust that gets people hurt, and it is the one many have been warning about for decades.
Human factors has studied human trust in automation since long before this wave, and the central finding was counterintuitive: the goal was never to maximize trust. Too much is a documented failure mode. People defer to a confident system, stop checking, and rubberstamp an answer that happened to be wrong. Automation bias and complacency are the names for it (Skitka, Mosier & Burdick (1999); Parasuraman & Manzey (2010)). Too little is the opposite failure, ignoring a system that was actually right (Parasuraman & Riley (1997)). What you want sits between them, and it is proportion. Reliance matched to real reliability, no more and no less (Lee & See (2004)). You trust the tool exactly as far as it has earned, and not one step into judgment it doesn’t have.
So if part of trust survives and part of it has to go, the question is what frame keeps them straight. I keep coming back to an idea from 1945. In an essay called “As We May Think,” Vannevar Bush imagined a device he named the memex: a machine to extend human memory and the way we connect ideas, an augmentation of the mind rather than a substitute for it. Not a prophet to consult and obey. An extension of reach.
That frame earns its keep twice. First, an extension leaves you holding the responsibility in a way a prophet never does. A telescope extends your sight, but you still choose where to point it and what the faint light means. The tool widens what you can do; the judgment about what to do, and the accountability for the result, stay with you.
Second, it rescues the last and strangest sense of trust, the one where people confide in these tools as if asking for counsel. Taken as outside judgment, that is just the second sense wearing a friendlier face, and the same objection applies. But read through reach, it becomes something else. An extension of your own thinking is not a counselor across the desk. It is closer to a mirror. Used that way, the value is not the wisdom it hands back; it is that it can help you see your own thinking more clearly than you manage alone. The reflection is still yours to read, and the moment you take it as a verdict instead of a reflection, you have slid right back into trusting a judgment that was never there.
The reach is extending faster and further than most people have noticed, and the real temptation is not to use these tools. It is to let the reach carry the judgment along with it, to applaud the wonder and leave the cost out of view. That was the magician’s mistake. The danger was never the trick. It was reaching past what he could account for and refusing to look at the difference.
The narrative for intelligence on its own, something that can reflect, judge or even empathize has been marketed so that we see this tool as something it isn’t. Maybe we will get there someday, but the tool we have cannot hold responsibility, it is a reach extender. If we change the paradigm on how we view the tool, we might find a better understanding for its role in society.
So trust the tool to work, within bounds you have actually checked. Trust it to extend you. Do not trust it to judge, and do not mistake its reflection for a verdict. The question was never whether to trust AI. It is whether you can see the whole of what it does, keep the reach proportionate, and own the consequences. Which is not really a question about the machine. It is a question about us.