The U.K. Safety Institute has made a groundbreaking move in the field of AI safety by releasing a new toolset called Inspect. This toolset, available under an open source license, aims to strengthen AI safety by making it easier for industry, research organizations, and academia to develop AI evaluations.
Inspect is designed to assess key capabilities of AI models, such as core knowledge and reasoning abilities, and generate a score based on the results. This marks the first time that an AI safety testing platform spearheaded by a state-backed body has been released for wider use, according to the Safety Institute.
Ian Hogarth, the chair of the Safety Institute, emphasized the importance of collaboration in AI safety testing and expressed hope that Inspect would become a building block for the global AI community. The toolset consists of three main components: data sets, solvers, and scorers, which work together to evaluate AI models.
Deborah Raj, a research fellow at Mozilla and AI ethicist, praised Inspect as a testament to the power of public investment in open source tooling for AI accountability. Clément Delangue, CEO of AI startup Hugging Face, even suggested integrating Inspect with their model library or creating a public leaderboard with the toolset’s evaluation results.
This release comes on the heels of a similar initiative by the National Institute of Standards and Technology (NIST) in the U.S., highlighting the growing importance of AI safety and accountability on a global scale. The U.S. and U.K. have also announced a partnership to jointly develop advanced AI model testing, showing a commitment to addressing risks associated with AI and generative AI technologies.