The Yuan requests your support! Our content will now be available free of charge for all registered subscribers, consistent with our mission to make AI a human commons accessible to all. We are therefore requesting donations from our readers so we may continue bringing you insightful reportage of this awesome technology that is sweeping the world. Donate now
Challenge of Human Oversight
By Ivana Bartoletti  |  Aug 12, 2021
Challenge of Human Oversight
Image courtesy of and under license from Shutterstock.com
In the discourse around the use of AI one often hears that humans must remain in the loop. This human oversight element is at the heart of conformity assessments in the recent European Union AI Act. But what does human oversight really mean, and how does it work in practice?

LONDON - Algorithms increasingly make decisions in the healthcare space, playing a big role in determining health outcomes. They can determine the precise dose of medicine for an individual patient or spot an anomaly on a scan that requires further investigation, sometimes better than an expert human eye.

In the discourse around the use of artificial intelligence (AI), we often hear that humans must remain in the loop. This human oversight element is, e.g., at the heart of the requirements for high-risk AI and conformity assessments in the recent European Union AI Act. The draft legislation would mandate that high-risk AI must allow for its oversight by humans to offset risks. A good example of high-risk AI is when algorithms determine the allocation of public services - who gets what.

But what does human oversight really mean, and how does it work in practice? This is a fundamental question, not only because it cuts to the heart of the human-machine relationship. Most importantly, it serves as a prime example of how a concept that looks wholesome could really be the cheese in a giant mousetrap to legitimize the lack of accountability around AI.


The Context

Algorithmic systems contribute ever more to decisions on health, public administration, allocation of benefits, or even whether an immigrant gains refugee status, citizenship, or neither. Concern for human rights, privacy and non-discrimination has led to a global movement toward ethics. This has spurred the production of numerous ethical guidelines and policy recommendations which are supposed to analyze, prevent, and mitigate the risks or collateral damage of an algorithmic society.

Insisting on the appropriate degree of human oversight is widely seen as a key policy toward this end and lies at the heart of society’s calls for procedural safeguar

The content herein is subject to copyright by The Yuan. All rights reserved. The content of the services is owned or licensed to The Yuan. Such content from The Yuan may be shared and reprinted but must clearly identify The Yuan as its original source. Content from a third-party copyright holder identified in the copyright notice contained in such third party’s content appearing in The Yuan must likewise be clearly labeled as such.
Continue reading
Sign up now to read this story for free.
- or -
Continue with Linkedin Continue with Google
Comments
Share your thoughts.
The Yuan wants to hear your voice. We welcome your on-topic commentary, critique, and expertise. All comments are moderated for civility.