GPT Guesses Between 1 and 100

Editor's take

Researchers demonstrated that GPT-4, when tasked with guessing a number between 1 and 100, exhibits a non-uniform distribution in its guesses, favoring numbers that are multiples of 10 and prime numbers. This finding suggests that LLMs, despite their probabilistic nature, are not truly random guessers and may be influenced by internal biases or patterns learned during training, rather than simply sampling uniformly.

This matters because it reveals a subtle but significant limitation in current LLMs' ability to perform truly random tasks, impacting applications requiring unbiased sampling such as simulations or certain security protocols. It highlights that even seemingly simple tasks can expose the underlying architecture and training data's influence on model behavior, a concern for developers aiming for predictable and fair AI outputs.

Future research should investigate whether this bias is present across different LLMs and what specific training data or architectural features contribute to it. Understanding this phenomenon could lead to methods for mitigating such biases, enabling more robust and reliable random number generation from AI models.