When the Chef Gets It Wrong

The lesson

The branch that never existed

It is 10 at night. The queue is gone, and the wok is cooling. A man leans over my counter, and he asks the new apprentice a simple question: does the stall have a branch in Toa Payoh? The apprentice does not miss a beat. Yes, he says. Lorong Four, opened in 2019, same prawn stock, same sambal. It sounds as smooth as the gravy on the noodles. And my spatula stops in mid-air, because there is no branch. There never was one. The apprentice simply had an answer for everything, and no way to know when he should say “I do not know.”

3 ways the chef gets it wrong

Here is the first way the chef gets it wrong. Researchers call it hallucination. A language model is built to predict plausible words, 1 after another, and it does that job beautifully. But nothing inside it stops to ask whether the words are true. A wrong answer and a right answer leave the kitchen in exactly the same confident voice.

Here is the second way: bias. Picture a stall that learnt to cook from 1 neighbourhood only. It serves the taste of that neighbourhood, extra chilli and extra meat, and it calls that everyone’s taste. A model trained on 1 slice of the world serves that slice back, just as smoothly. Bias in, bias out.

And here is the third way, the deepest one. The model has never tasted the noodles. It can describe the first fry perfectly, eggs and garlic in hot oil, and it has no idea what those words smell like. It matches patterns without carrying meaning. Fluency is 1 dish. Truth is another.

Confident mistakes in the wild

This has already happened. In America, a lawyer once asked a chatbot for past court rulings to support his case. The chatbot invented them, complete with case names. The lawyer filed them. The judge checked. The rulings did not exist. Bias travels the same road. A famous company trained a hiring tool on 10 years of its own past hires. The tool quietly learnt to keep hiring the same kind of person. The data decided that long before anyone approved it.

Which claim is the lie?

Now I will give you 3 claims, all delivered by a chatbot, all in the same confident voice.

One: some MRT trains in Singapore drive themselves. Two: a phone can recognise its owner’s face and unlock. Three: I check every answer before I give it.

Which one is the confident lie? I give you three seconds.

Pause here for three seconds.

Driverless trains are real. Face unlock is real. The third claim is the lie, and notice that it sounded exactly as sure as the true ones. That is the sound of hallucination: perfect confidence, from a machine that cannot check itself. And the fix is not more confidence. The fix is a second source, a human glance, or the facts you pasted in yourself.