Yes, we can evaluate AI.
Landers and Behrend (2023) demonstrate how to audit AI models for fairness and bias.
The fiction writer Arthur C. Clarke wrote: “Any sufficiently advanced technology is indistinguishable from magic.” And indeed, some technology does seem magical. But, every magic trick can be understood, and so can every technology:
Landers and Behrend (2023) have written a great article on how to audit an AI model for fairness and bias. True, we may not be able to understand exactly how an AI model determines which students are likely to drop out of school, or which job applicants are likely to succeed. But we can understand how the model was developed, how it is used, and whether and how much concerns about fairness and bias guided the model’s development and use. And that means we can decide whether or not to use the AI model for high-stakes decisions.
AI models aren’t magic, they are a technology that we can understand. Work like Landers and Behrend’s shows us how to stop marveling at technology, and start controlling its use.