BRIDGING OFFLINE DESIGN AND ONLINE ADAPTATION IN SAFE LEARNING-ENABLED SYSTEMS

Maintaining the safety of a learning-enabled system that navigates in an unknown environment is a major challenge owing to uncertainty in the environment, the system’s goals, and the system’s learning-enabled components. This project proposes a novel approach to mitigating these uncertainties. The project’s novelties are the development of a two-phase design and deployment process integrated into a tight feedback loop: (1) an offline design process aimed at synthesizing systems that are provably robust and resilient to known unknowns, and (2) an automated online safety monitoring phase, during which a deployed learning-enabled system seeks to detect, learn about, and adapt to unknown unknowns. The project’s impacts include: (i) a mathematical guarantee on the end-to-end safety of the design and deployment process described above for learning-enabled systems; (ii) methods that ensure safety with respect to known unknowns during the offline design stage, and safety with respect to unknown unknowns during deployment, when possible; and (iii) techniques that identify and learn about unknown unknowns, that is, novel sources of uncertainty, so that they can be integrated into the design of future systems.
Link