Social media users have transformed into red teams for AI features with the help of memes

Google’s AI search feature has recently come under fire for providing some questionable advice, including the suggestion that “running with scissors is a cardio exercise that can increase your heart rate and require concentration and focus.” This response was pulled from a comedy blog, but it has sparked a wave of criticism on social media.

The incident has highlighted the importance of red teaming in the development of AI products. Red teams, or ethical hackers, are hired by companies to identify vulnerabilities in their products before they are released to the public. Despite extensive testing, Google still managed to ship a flawed AI product, leading to widespread ridicule online.

In addition to the running with scissors blunder, Google’s AI has also been criticized for suggesting adding glue to pizza to make the cheese stick, based on an eleven-year-old Reddit comment. These errors have raised concerns about the reliability of AI content, especially as tech companies continue to invest millions in licensing content for AI model training.

While some of these mistakes may seem harmless, others have the potential to be dangerous. For example, Google’s AI recommended applying a tourniquet and cutting a wound to treat a rattlesnake bite, contrary to medical advice. These incidents serve as a reminder of the risks associated with relying on AI for accurate information.

As these flawed AI responses continue to circulate on social media, there is a growing concern that the AI may become even more confused by the new content surrounding these topics. This highlights the challenges of training large-scale AI models on internet data, where misinformation and errors can easily propagate.

In the end, the saying “garbage in, garbage out” rings true, emphasizing the need for companies to prioritize accuracy and reliability in their AI products.