I am quite inspired by the latest move from the National Institute of Standards and Technology in releasing Dioptra, a tool for testing AI model risk. With AI becoming ever more pervasive in our lives, there is a requirement that these systems be safe and secure.
Dioptra is designed to measure how the performance of an AI system could be degraded by malicious attacks, which I believe is a very important part of the development of AI.
I'm of the view that AI benchmarks are hard to come by because the most sophisticated AI models out there are black boxes, whose inner infrastructure and training data, among other key details, remain closed to the world by the companies creating them. Other than anything else, it is this lack of transparency that considerably raises challenges in determining whether an AI model is safe to be deployed in the real world or not. I think Dioptra is quite useful in shedding light on forms of attack that may make an AI system perform less effectively and quantifying this impact.
I also learned about the limits of dioptra. For example, out-of-the-box, it works with those models that are downloadable for use locally—for instance, Meta's Llama family. Models gated behind an API are incompatible with Dioptra, at least not yet. It's a limitation that does make me feel there should be more collaboration and openness among the AI developer community.
I feel that the development of AI safety tools like Dioptra is a response to the growing concerns about AI safety and security. The U.S. government has taken steps concerning such issues, including the establishment of an Institute on Artificial Intelligence Safety and releasing guidelines for developing AIs. To me, these efforts have been very necessary, yet more should be done to ensure the safety and security of AI systems.
To the best of my judgment, the question of AI safety is really complex and calls for a multifaceted approach. In this context, I personally feel there is a need for developing more sophisticated tools like Dioptra; still, at the same time, there is an underpinning need to fix issues of transparency and accountability in AI development. Unless we make sure that AI system developers are held liable for the safety and security issues of their products and that users are informed about some of the risks and limitations within AI, we will not truly be able to tap into the all-out living potential associated with AI.
