Actionable Meta-Requirements for Auditing AI Robustness

Summary

AI-enabled systems can become less dependable when inputs, data distributions, or external dependencies change, making robustness auditing important for deciding whether available evidence supports a defined robustness conclusion. This study addresses the lack of a consolidated set of requirements for robustness-auditing frameworks by deriving higher-order meta-requirements from research literature and validating them with practitioners. A systematic review retained 74 of 2,890 records and produced 43 meta-requirements across 16 categories. In an evaluation involving 128 practitioners, the researchers collected 5,504 ratings and 586 substantive comments; 81.20% of ratings expressed agreement or strong agreement, and 41 of the 43 literature-derived requirements were classified as strongly supported. Practitioner feedback refined the requirements and added one concerning external components and supply-chain dependencies. The final set contains 44 meta-requirements across 16 categories, with 43 receiving quantitative validation. The authors say the set can help organizations implement robustness-auditing frameworks, assess existing frameworks for completeness, and refine them.