···
2
+
title = "Jeopardy! world"
6
+
tags = ["ai", "culture"]
9
+
Some time ago, there was an anime available on Netflix — *Godzilla Singular
10
+
Point*. It wasn't a spectacular success, but it featured a plot device that I
11
+
think reflects something increasingly common today: you need to know the answer
12
+
to your question before you can ask it.
14
+
This is something I see all the time in the current wave of AI hype. You need to
15
+
know what the answer *should* be before you can write a useful prompt.
19
+
The issue I have with many AI use cases is this: unless you have specialized
20
+
knowledge about the topic you're asking about, you can't reliably tell the
21
+
difference between a solid AI answer and complete nonsense.
23
+
I've had a few discussions about this on various Discord servers. The example I
24
+
often use is this simple question posed to an AI:
26
+
> Does 6 character long identification number, that contains digits and upper
27
+
> case letters (with exception to 0, O, 1, I, and L) is enough to randomly
28
+
> assign unique identification numbers for 10 million records?
30
+
You can see for your self answer from ChatGPT [there][chatgpt].
32
+
At first glance, the answer looks valid and sensible. The math checks out. It
33
+
calculates the number of available combinations correctly. Everything seems
38
+
There is huge issue there, and probably most of the people who have been working
39
+
with basic statistic or cryptography will notice it. ChatGPT (and any other AI
40
+
that I have tested out) fail to notice very important word there
42
+
> \[…] randomly \[…]
44
+
This single word invalidates the entire reasoning, despite the correct
45
+
calculations. Because of the [birthday problem][], the answer isn't feasible.
46
+
While it's technically possible to assign a unique ID to every record, doing so
47
+
randomly introduces a high probability of collisions.
49
+
- At around 30,000 generated IDs, there's already a 50% chance of a collision
50
+
- At around 42,000, the chance of at least one duplicate reaches 99.9%
52
+
So even though the math is correct, the logic fails under the randomness constraint.
54
+
## *Jeopardy!* world
56
+
This is my main issue with AI tools: if you already have knowledge about the
57
+
subject, you don’t really need to ask the AI. But if you don’t have that
58
+
knowledge, you have no reliable way of knowing whether the answer makes sense or
59
+
not. It’s like playing *Jeopardy!* — you need to know the answer before you can
60
+
phrase the right question.
62
+
In my view, AI is most useful in areas where the results can be quickly reviewed
63
+
and discarded if needed. That’s why the whole “vibe coding” (aka slop
64
+
generation) approach falls short. If you don’t have a good sense of what the
65
+
output should look like, you probably don’t have the expertise to verify it.
67
+
[And gods forbid you from allowing AI to do anything on production][replit-fuckup].
69
+
[chatgpt]: https://chatgpt.com/share/68879fe7-d4e0-8007-9a30-3a9e2ace791d
70
+
[birthday problem]: https://en.wikipedia.org/wiki/Birthday_problem
71
+
[replit-fuckup]: https://www.businessinsider.com/replit-ceo-apologizes-ai-coding-tool-delete-company-database-2025-7?op=1
hauleth.dev