Lies™ by AI 🤥

Mistral must have had a bad day. I had it create a table with two columns listing all tracks from the 2000 album "The Eminem Show" and their respective producers.

1. First it claimed, it can't create a table.
2. As compensation, it showed the information in an ordered list instead. The track listing looked correct at first glance, especially since it showed the tracks in the original track order of the album. When I looked more closely though, I noticed that a few tracks were missing (among them, not only bangers "Say What You Say" and "'Till I Collapse" but interestingly also the most successful single of the entire album, "Without Me")
3. While ignoring Mistral's mistake at first, I had it create a similar list–yes, a list, since it momentarily forgot how to create tables–for the Marshall Mathers LP, Eminem's third studio album that was released before The Eminem Show. This time, it very obviously hallucinated, inventing track names such as "Hate Me This Much" and "The Kids", mixed in with track names that actually existed. Moreover, the LLM claimed that the album had a total of 30 tracks (and just kept repeating track names after track number 18).
4. Next, I gave Mistral the chance to correct its mistake from before, when I simply prompted it to "try again with The Eminem Show". This resulted in a similarly faulty list, but at least it remembered its ability to create tables ("You can use this information to create a table in your preferred format". If you need a table, here is an example in markdown").
5. Finally, I corrected its obvious mistake, namely that it forgot about "Without Me". Now, it created a wonderful table from the get-go (without showing a list at first), included "Without Me" in a random spot (as track number 2 rather than track number 10) but again omitted "Say What You Say" and "'Till I Collapse".

This is a harmless example but it illustrates how easily AIs lie to us. Have a guess how likely I am to trust the actual information I enquired Mistral about (the name of the individual producers) after so much hallucination and confusion on its part?

All of this chaos, even though the requested information is so easy to look up–there are Wikipedia articles for both, The Eminem Show and The Marshall Mathers LP–with the individual track listings.

The age of lies

Today, already, you can't escape AI's grip. For many of us, it's replacing a good old web search–it doesn't help truth that search engines themselves advertise AI results above the traditional results, allegedly faulty.

Apart from replacing search, almost no mainstream software product has been spared from introducing AI to its user interfaces, by adding an "ask AI" button, search bar or similar. Today, software products that resisted enshittification are extremely rare. I have developed a new-found respect for software makers who would not let themselves get carried away with the hype but stayed true to their values (and truth)–honorary mention: note-taking app Bear.

I assume that many users are getting mislead by AI without consciously noticing that they are being lied to, since so often the information looks correct at first glance. We adopted AI as a technology, because we consider it a time saver. Since no one seems to have time for anything these days anyway, we don't notice that the time it supposedly saves leads us to believing information that is actually untruthful.

Unintentional versus engineered lies

What we used to know as "software bugs" with "traditional", AI-less software are taken onto a completely new level with AI. Software bugs are old news. When software messes up today, the results are hallucinations and lies. Many times they are unintentional, as a result of the nature of how LLMs work: they are probabilistic and actually don't know anything.

But apart from the hallucinations, there's something even more troubling. The case, where AI companies actually engineer and orchestrate output of LLMs in order to create alternative facts, aligned with political interest: Be it US-American AI companies, who are Donald Trump-friendly or Chinese AI companies, where undoubtedly, criticism of political leaders or geopolitical matters such as the sovereignty of Taiwan would be unthinkable.

At this point, three years into the AI craze, we should ask ourselves this: do we want to live in a world where a delicate matter such as truth is dictated by the likes of AI companies?