Hubsessed Office Hours: AI-Native · Every clip

The surgeon riddle: a training-data fallacy

Nico Lafakis · Monday, October 26, 2026

Can your AI catch a trick question, or does it repeat the popular answer? I gave it a classic puzzle: a car accident, the father is killed, the son is taken to the hospital, and the doctor says "I can't operate on this child, it's my son." It's been answered incorrectly so many times. Then I tried to trick it: I made the father the surgeon, and said the mother was killed too. The model still said the surgeon is the boy's mother. "She died. Did you not hear the first part of that puzzle?" That's what I'd call a training-data fallacy. My read: if enough bad answers to a question exist, a model can repeat them. That's my read, not a test result. You get a quick way to check whether a model is reasoning or repeating. What question have you seen your AI answer confidently and get wrong? Replay clip from Hubsessed Office Hours: AI-Native. Trisha and I hosted. Clip page: https://www.hubsessed.io/clips/ain-2026-10-26-the-surgeon-riddle-a-training-data-fallacy