The lesson
The stock pot remembers school
The food centre is still dark when I taste the stock that simmered through the night, and my mind drifts back. Three years of culinary school gave me my hands, from stock to knife work. One month at this stall, 20 years ago, made me its chef. The old boss stood beside me and coached the style in. Darker here. More stock! Our customers like it that way. The school was the long and expensive part, and the coaching took 1 month.
School first, coaching after
Big models take the same 2 roads. Pre-training is the culinary school. The model spends months reading a slice of everything ever written. It costs many millions, and it nearly always belongs to a big lab with a giant kitchen. Fine-tuning is the month of coaching. You take the finished chef and teach him your style with a few hundred of your own examples. Your dishes. Your voice. It is cheap and quick, and the food starts tasting like yours.
Then comes 1 question: how do you want your model served? There are 3 doors. Door one is food delivery. The dish arrives in a sealed box, and you just eat. That is a closed model. Door two is the packet of Hokkien Mee stock that comes with its ingredients list. You take it home and cook from the instructions. Those are open weights, a model you can download and run. The packet stock recipe stays with its maker. Door three is the open kitchen. You stand beside the chef and watch him prepare every ingredient, including his “secret” stock, step by step. You could cook it from scratch. That is open source, with the training code and data published for anyone to study.
When the menu changes
My recipe has not changed, but my customers have. The gym crowd downstairs wants less oil. The lunch queue gets younger every year. A deployed model drifts the same way, because the world it learnt from keeps moving. Word by word. Price by price. So the people running it keep tasting. They watch the answers, and they tweak the recipe to suit the times.
3 products, 3 doors
Now you, at the stall. I name 3 products, 1 per door. A chat app you subscribe to by the month. A model file you download and run on your laptop. A model published with its training code and data, free for anyone to study. Which kitchen did each come from? I give you three seconds.
Pause here for three seconds.
Show the answer
… The chat app is delivery, sealed box. … The download is the takeaway dish plus its ingredients list. … The published model is the open kitchen, recipe included.