“No other technology has changed the world as quickly and intensely as web- and data-driven intelligent systems (AI)”

“Our mission is to develop technology that is used by people and integrated into everyday life, making the world a better place. We want to educate responsible engineers. This can be achieved only by entering into actual conversations with all the different stakeholders who will be using that technology – precisely the area where methodologies from the design world can help.” A conversation with Geert-Jan Houben, pro-vice rector AI, Data and Digitalisation and leader of the AI Initiative, and Alessandro Bozzon, professor of Human-Centered Artificial Intelligence at the Industrial Design Engineering (IDE) faculty of TU Delft – committed to improving well-being and promoting inclusion through personalized social computing systems. They were invited to discuss the role of design in AI.

Alessandro, what is your personal motivation for doing this job?

Alessandro Bozzon: “Geert-Jan and I both grew up in a different time. We have experienced the transition from the analog world to the digital world, and we were there at the emergence of the digital society. I've been playing with web technology from the very beginning, out of curiosity, and have also witnessed the emergence of intelligent systems in society. That has allowed me to see how these systems have changed people, and how they have changed society as a whole. Digital technologies possess enormous power. We cannot deny how they can also be disruptive; how they can even cause harm. It is very important that we learn to control them.”

Can you explain what your research is about?

Alessandro Bozzon: “My personal interest has always been the human side of this movement. I wanted to do research into that, but also to be well informed by technological insight. So, I'm interested in the technical elements of AI, along with how they manifest themselves in relation to people who use AI. You have to understand – especially in the case of AI – who those people are, what they do, what they want, how they want it, and the context in which they will use the technology. Only then can you create useful AI technology that really helps us, without all kinds of unintended side effects.”

You say you have to talk to people understand them. That sounds almost like psychological or sociological research.

Alessandro Bozzon: “It is an interdisciplinary way of doing research. We do indeed tap into knowledge created by other disciplines, and also knowledge created by colleagues who work on the applied side of AI, such as within the themes of health and mobility.”

How should we understand ‘design’ when applied to AI systems?

Alessandro Bozzon: “When you hear the word ‘design’, it doesn’t mean aesthetics in this case, not as we use it for products such as chairs and tables. AI systems do not operate in isolation. They are part of socio-technical systems, organizations, and society. In this case, design is about the interplay between people, algorithms, and data from people used to train those algorithms. Our job is to shape all of that.
When we think of engineering, we think of a straight line, from data to model to user, or from requirements to software to user. In practice, this is not a straight line but a circle. The design evolves over time, as we get a better understanding of people and a better understanding of technology; it keeps changing.”

Geert-Jan Houben: “Exactly. If all goes well, there will be an interplay between software, data, technology, and people. And that interplay must be carefully designed. I think that’s the crux. Our goal is to make sure that the interplay between the automated part and the human part works well, managed effectively and with care.”

Alessandro Bozzon: “When it comes to the relationship between people and technology, we often talk about trust, or about adaptation – about how we can successfully integrate technology into everyday life. What we need to find out is how people can use and want to use and interpret the technology. This is where design can play a role. We need to understand first what the problem is that the technology is trying to solve, then look at how to make that technology successful. For me, success means not only working to a fixed plan, but also integrating real-world findings. Sometimes you see AI technology having an effect that was not foreseen. Instances of prejudice, discrimination, or harm all arise from a misalignment between the desired effect and the actual manifestation of the technology. That’s exactly where design comes into play, helping us to envision the future. We want to involve the people who will use the technology in the design and engineering process.”

Can you give an example?

Alessandro Bozzon: “The virtual assistant ‘Alexa’ is a classic example of AI technology that has become normal in our daily lives. Ethical principles come into play here: we want Alexa to do no harm, and to be helpful, fair, useful. There is a technical aspect, developing the right algorithms. But there are also design questions: what data and from which people are we going to use? What will the interaction between the algorithm and the person look like? What kind of conversations will be held and which not? Those are all things we ‘shape’.”

Geert-Jan Houben: “On the one hand, it is important that people understand this form of technology properly: it is about correct interpretation. On the other hand, the form the technology takes is not always the same, because it depends on the context. So I think meaning and context are two very important elements.”

Alessandro Bozzon: “Many AI projects and innovations are being developed that never go on to find a place in society. There are plenty of inventions that no one uses or wants, often because no effort was ever made to understand what was really needed. If you're developing an AI innovation for a factory, you want the people on the shop floor to be a part of the design process from the start. I'm working on a Horizon project called COALA where we are creating technology through the eyes of the people who will use it. We understand their level of education, their fears of technology, and so on. This is the basic principle of what we mean by AI at Delft: it is not a technology to replace people, but to support people.”

Geert-Jan Houben: “The word ‘adaptation’ is important. Saying ‘use’ gives the impression that once you've determined how you want to use something, all you have to do is create it. In the case of AI, there is an interactive and continuous process. Our understanding of people is evolving, and our relationship with technology is also evolving at lightning speed. It is always evolving.”

Does this also apply when you design an agricultural machine?

Alessandro Bozzon: “I would like to say that AI is a revolutionary technology, of a kind we have never seen before. That’s because AI technology changes in response to how we humans interact with it. There is a deep codependency: the way we use the technology changes it. Think of filter bubbles, or polarization, as you see in the world now. Was it possible to predict in advance that social media algorithms would have this effect? Perhaps, but above all these phenomena demonstrate that it is impossible to consider a particular AI innovation as ‘finished’. In that sense, AI is fundamentally different from designing an agricultural machine.”

Geert-Jan Houben: “You design a solution for a particular problem, then individuals start using it and they are also influenced by their environment and by other users. There are all kinds of processes and effects at play, and we see these reflected in a technology that adapts to that use.”

Alessandro Bozzon: “There has always been a human-technology relationship, but unlike other systems, AI systems do not always display behavior that can be predicted in advance. Effects may occur that were difficult to foresee and which are dynamic. They affect us personally, and they affect the society in which we live. No other technology has changed humanity and the world as quickly and intensely as web- and data-driven intelligent systems (AI).”

Alessandro Bozzon: “The design perspective is about seeing people as they are, not as you want them to be. And yes, people have different technical capabilities, different levels of education, and so on. Design can enable adaptation without people having to think about it. Good technology is technology that disappears, that just ‘merges’.”

Geert-Jan Houben: “With good design you can get the user to concentrate on resulting actions, and on the actions he or she desired – the user is not burdened with extraneous things. The aim is trying to get the user to make good decisions.”

And you can make this here at TU Delft: technology that ‘merges’ and enables people to do good things?

Alessandro Bozzon: “The great thing about TU Delft is that researchers work together, which I believe is necessary to develop and design the desired AI technology. In the Delft AI labs, the technological, ethical, and design perspectives come together, and dialogue takes place. That is actually what society asks of us, and our students learn this too. Not only do they learn to work together, but they come to understand what it takes to create AI and systems that will have a positive impact on the world.”