Abbink: "At its core, 'meaningful human control' revolves around the view that responsibility should never be attributed to an 'autonomous system' but rather to people. There are actually no autonomous systems, only systems with autonomous properties that are developed, implemented, and used and abused by people. It’s important that human users feel the right level of responsibility, and have a realistic chance to intervene. This is when we speak of meaningful human control – a concept that ethicists, business experts, designers and engineers have to consider together. It is both scientifically and socially complex."

Abbink: "Within AiTech, five of our postdocs from different backgrounds have been researching 'meaningful human control' over the past few years. Precisely because they came from different disciplines, they first had to work hard to integrate each other's methodologies. Every week they have met in what we call ‘agora’ meetings: open forums where anyone can come and contribute to the discussion. They also organised symposiums to bring together even more people with the same interests. From the very beginning, we wanted to write a kind of position paper setting out how to build systems that are under meaningful human control. A key question was how to operationalise a philosophical concept so that engineers can work with it. After three years, we have a joint paper that I am very proud of – it’s more than just a paper because it represents the concrete setting up of interdisciplinary research."

Abbink: "In our position paper, we defined four features to help with this. The first is the moral operational design domain needed to operationalise meaningful human control. To illustrate, we chose two practical applications: automated persuasion and AI-based employee recruitment. Consider a semi-autonomous car, where you can define a technical domain within which the car drives autonomously, for example on the motorway. If an unexpected situation arises, such as a motorcyclist suddenly crossing in front, the human has to take over. We can therefore delineate a moral operational design domain based on the situations where this is needed. Equipped with this, we can delineate what humans are responsible for and what the machine is responsible for. We can also do the same with AI-based recruitment or other human-AI systems.

For a human-AI system to function well, both the people involved and the AI system must also possess a certain imagination relating to tasks, role divisions and desired outcomes, as well as to the environment and mutual opportunities and constraints. This is the second important feature. Such imaginations or representations are often called ‘mental models’, and are based on shared mental models of all people involved in the human-AI system. We recommend explicitly including conversations about this feature during the human-AI system development process – something that does not happen automatically."

Abbink: "The third feature is making sure that the responsibility assigned to a human being must be proportional to that human being's ability and authority to actually control the system. We believe that you can only be held responsible as a human being if you also have a practical and realistic ability to intervene. With this in mind, you cannot hold a driver of a semi-autonomous car responsible for an accident when he does not have enough time to regain control of the car from the autopilot.

Fourth and finally, the actions of the AI system should always be linkable to at least one person who is also aware of moral responsibility."

Abbink: "Absolutely. We also specify 'at least one', and in practice this often means organisations and groups. Things can become diffuse when it comes to software that passes through many hands, and which spreads quickly and is modified quickly. Our point is that all organisations and people involved should also be aware of their responsibility during design or implementation. If not, you have to recognise that the system is no longer under meaningful control. You cannot pretend that it is, or pass it off as such to the end user."

Abbink: "In the fields of both mobility and recruitment algorithms, we have published many papers. For instance, we used simulation studies to show how people assign responsibility to automatic systems when driving. This shows that even when people know that you cannot reasonably hold the driver responsible, they still say the driver is responsible. It seems that people are unaware of the impact of automated driving on anyone’s ability to intervene when necessary. There is a gap between how ordinary people see responsibility and what is realistic in practice."

Abbink: "For cars, you can inform people better about the challenges that automation poses for drivers, and you can adapt driver training. With recruitment algorithms, we have explored how you can design systems in such a way that the people who use them, for example within human resources, better understand the basis used for decisions. They are then equipped to reject the advice."

Abbink: "The four features we have described are a first important step towards fully operationalising the concept of 'meaningful human control'. We put these alongside research by colleague Simeon Calvert from Civil Engineering and Geosciences, who has been working on the structured conditions needed to operationalise the concept for automation in transport systems – not only on the road but also on water and in the air.

The big challenge we have not yet got around to as a community is really translating all of these features and conditions into practical system design. Ideally, you want to experience the impact of more or less meaningful control – and we have not yet got around to that either in Delft or worldwide. In the coming years, we want to use concrete experiments to show empirically that if you change the process parameters across manual work, technology, algorithms and even within an organisation, the experience of human responsibility changes in this way or that way. I call this 'where the rubber hits the road'. It’s what we as researchers are going to work with now."

Abbink: "I think our contribution is in trying to stimulate international cooperation to achieve sharper debate, better research and more collaboration. We have achieved this concretely with international colleagues through our initiative writing the first handbook on meaningful human control. It brings together perspectives from ethics and philosophy, law and governance, and design and engineering, among others. The handbook will be published in December this year. In spring 2024, we are organising a symposium to bring together even more international researchers around the theme. Through all our activities, we are collectively thinking more holistically about meaningful control. People from different backgrounds and application areas have discussed and reviewed each other's papers to arrive at this book.

I feel there is a movement to achieve more debate and consensus outside our network as well, but we are not there yet. Some people still find the concept too little operationalised, while others think you should just legally cover responsibility. But I am confident that our typically Delft approach – integrating different perspectives and translating concepts into practical applications – will gain more ground."