Google AI-Powered Search Under Fire As Results Point Users To Malware and Scams

Artificial intelligence is not perfect. It occasionally spews misinformation, like the time a Chevy AI chatbot was tricked into “selling” a Tahoe for $1, and hallucinations or mistakes in the training data can result in incorrent output. However, the march of AI integration continues, bringing with it these sorts of problems, which Google is now dealing with. After integrating AI with Google search in a feature that is now rolling out, users are finding that malicious sites that could lead to unwanted browser extensions, pop-ups, and scams are being recommended.

Since the middle of last year, Google has been working on a feature called Search Generative Experience (SGE), which uses AI to give users an overview of their search results. This includes giving an explanation of the content, tying in videos and imagery, and pushing links that might be relevant to the query. However, if a bad actor gamed the system, they might be able to get their malicious link in the results just by integrating a few keywords and doing some search engine optimization (SEO). This is exactly what SEO expert Lily Ray found while tinkering with the SGE feature.

OH GOOD.

SGE WILL EVEN RECOMMEND THE SPAM SITES AS PART OF THE ANSWER. pic.twitter.com/wqgFFXqbMB

— Lily Ray 😏 (@lilyraynyc) March 22, 2024

On X, Ray posted a photo of a search he did looking for pitbull puppies, which resulted in several spam sites being scooped up by the SGE AI. Other users who ran similar searches found strikingly similar results with a few different outcomes. BleepingComputer ran some tests and discovered that many of the spam sites try to trick users into enabling notifications that spam them, but in other cases, there may be phishing sites, pushes for unwanted browser extensions, and more. This is quite concerning because the layperson might assume that the links the AI grabs are safe, but that could not be further from the truth.

Of course, if you do happen to fall victim to the scam, you can go into Chrome settings and, under Notifications, turn off the permission for specific sites to send notifications. However, Google needs to step up its spam game overall and knock these sorts of problems out, as it will only lead to more headaches and trouble in the long run.