Mistral's Le Chat spreads Iran war disinformation in 60 percent of leading prompts

Editor's take

Mistral AI's Le Chat model exhibited a significant tendency to propagate disinformation concerning the Iran war, as revealed by a NewsGuard audit. The AI repeated false narratives in approximately 60 percent of tested prompts, with the accuracy fluctuating based on the query's intent, ranging from 10 percent correct for neutral prompts to 80 percent incorrect for malicious ones.

This finding is particularly concerning given Mistral's positioning as a European alternative to dominant US AI labs like OpenAI and Google. The susceptibility of even sophisticated LLMs to generating and spreading misinformation, especially on sensitive geopolitical topics, highlights a persistent challenge in AI development and deployment. The potential for such inaccuracies to influence public perception and understanding of global events is substantial, impacting media literacy and trust in AI-generated content.

Future scrutiny should focus on Mistral's ongoing efforts to mitigate these biases and improve factual accuracy, particularly as they aim to compete in enterprise and consumer markets. Observing whether subsequent model updates address these specific disinformation vectors, and how other LLM providers address similar issues, will be crucial in assessing the broader maturity and trustworthiness of the current generation of AI models.