Google's AI made up a fake cheese fact that wound up in an ad for Google's AI, perfectly highlighting why relying on AI is a bad idea

First reported by Ars Technica, the "big cheese" of online search appears to have really curdled things, and in the heart of America's dairyland, no less. An ad for Google's Gemini AI that played in Wisconsin during the Super Bowl was first shown online last week with a "hallucination"—that is to say, a bald-faced lie—visible during a demonstration of the "writing aid." The ad was quietly edited to remove the hallucination before it went live during the big game.

Apparently everybody just soft-launches their Super Bowl ads on YouTube a week in advance now, and that goes for Google's gaggle of 50 spots showcasing how Gemini could help small business owners, with a unique 30-second showcase for each state of the Union. In Wisconsin, the ad focused on a cheesemonger writing online store copy for his assorted victuals.

The weirdness came partway through, when the ad actually showed Google Gemini in action. It told the cheese vendor that Gouda accounts for "50 to 60 percent of the world's cheese consumption." Now, Gouda's hardly a hardcore real head pick like Roquefort or BellaVitano, but there's also no way it's pulling in cheddar or mozzarella numbers. Travel blogger Nate Hake and Google-focused Twitter account Goog Enough documented the erroneous initial version of the ad, but Google responded by quietly swapping in a more accurate Gemini-suggested blurb in all live versions of the ad, including the one that aired during the Super Bowl.

Adding another wrinkle to the story, the erroneous statistic can seemingly be sourced to the Gouda page on cheese.com, an SEO-focused website with a love for cheese and a loose cannon, devil may care approach to the facts. Google Gemini will supposedly provide links to the sources it pulls from, much like AI Overview in search, and you could charitably argue that this feature just wasn't on display during the ad.

But that raises other concerns: Is the ad then just pure "bullshot" like all those pre-rendered E3 game trailers of yore or all of Elon Musk's silly robots? No matter how much Gemini cites sources, isn't there something ethically questionable about an automated process presenting itself as a neutral arbiter of the internet, offering potentially rotten information as authoritative statements of fact? Google states Gemini is a "creative writing aid, not intended to be factual," and notes that suggestions from Help Me Write in Chrome "can be inaccurate or offensive since it's still in an experimental status."

But that truly begs the question of who would want writing help from a program that spits out "inaccurate or offensive" information. There is also something darkly humorous to me about Google having to eat some crow due to the output of an SEO slop farm whose existence was incentivized by the company in the first place. Billions of dollars and untold amounts of compute power are being thrown at AI models, but they don't seem to be making much progress on the "hallucination" problem.