Every AI system is deployed by a human organization. In high risk applications, the combined human plus AI system must function as a high-reliability organization in order to avoid catastrophic errors. This short note reviews the properties of high-reliability organizations and draws implications for the development of AI technology and the safe application of that technology.
Paper
References (11)
05South Wales Police2018 · south-wales
08Reluctance to simplify interpretationsHROs cultivate
09Sensitivity to operations. HROs maintain at all times a small group of people who have deep situational awareness
10Commitment to resilience. Teams practice managing surprise. They practice recombining existing actions and procedures in novel ways in order to attain high skill at improvisation
11cation of structures. HROs empower every team member to make decisions related to his / her expertise. Any person can raise an alarm and halt operationsarise