Learning by Reward

The lesson

Nobody tips at a hawker stall

It is 10 p.m., and I am counting the takings. My mind goes back 20 years, to my first weeks at this wok. The old boss coached me for 1 month. Then he stepped back, and the customers took over. I burned the garlic. I over-salted the stock. Slowly, I got good. Nobody retrained me. Nobody tipped me either, because nobody tips at a hawker stall. So what taught me? The customers did, plate by plate.

The plates do the teaching

Everything so far had a teacher. The prawn sorter learnt from thousands of pre-labelled examples. My first months had none. I learnt from what happened after I cooked. I made the sambal sharper, and watched. Empty plate back: sharpen it again. Half-eaten plate back: something I changed did not suit my customers. After 1,000 plates: whatever brings empty plates back happens more. Whatever leaves plates half-eaten happens less. That is behaviour shaped by reward. Engineers call it reinforcement learning. Classification had its labels first. Here, the feedback arrives after the act, 1 plate at a time, and never tells you why. Half a plate could mean too salty, or a bad day at the office. The plate never explains itself.

The machine that taught itself Go

You have met this machine in the news. A famous AI taught itself the board game Go. It played millions of matches against itself, with winning as its only reward, until it beat the world champion. No engineer can write every step for a walking robot leg. So they let the machine fall, thousands of times, and reward whatever stays upright a moment longer.

You set the rewards

Now rewind to my first week at this wok. I may not ask a customer for a review. No stars. The plates are my only language. I change 1 thing each day. So design the reward rules in your head. What tells you it worked? What tells you to change back? And what if you count only empty plates, and serve less? I give you three seconds.

Pause here for three seconds.