Session 1A: Operational T&E of AI-supported Data Integration, Fusion, and Analysis Systems
Adam Miller has been a Research Staff Member at IDA since 2022. He is a member of the IDA Test Science team supporting HSI evaluations of Land and Expeditionary Warfare systems. Previously, Adam worked as a behavioral neuroscientist at The Hospital for Sick Children in Toronto, ON, where he studied how memories are encoded in the brain. He has a PhD in Psychology from Cornell University, and a B.A. in Psychology from Providence College.
AI will play an important role in future military systems. However, large questions remain about how to test AI systems, especially in operational settings. Here, we discuss an approach for the operational test and evaluation (OT&E) of AI-supported data integration, fusion, and analysis systems. We highlight new challenges posed by AI-supported systems and we discuss new and existing OT&E methods for overcoming them. We demonstrate how to apply these OT&E methods via a notional test concept that focuses on evaluating an AI-supported data integration system in terms of its technical performance (how accurate is the AI output?) and human systems interaction (how does the AI affect users?).
Session Materials: https://dataworks.testscience.org/wp-...