Recently I started to try to wrap my head around this question. As it turns out, my history on face detection and hand gesture detection might have given me a clue on why the current LLMs have a tendency to hallucinate and provide false confidence.

Warning: This might get more mathematical than you’re used to or prepared for. But don’t be afraid. I’ll try my best to make this understandable.