"Checklists no longer suffice"
An interview with the new co-directors of the TU Delft Safety & Security Institute
Justin and Eleonora, you have just become co-directors of the TU Delft Safety & Security Institute, but surely you were involved before. What is your connection to or experience with the institute?
Justin: As an AI expert I have been working with the local police force for years, both at my previous job in Singapore and now with the Netherlands police since I joined TU Delft in 2021.
Amongst others, I’m now leading the recently established Model-Driven Decision-Making Lab which falls under the framework agreement that TU Delft signed last year with the Netherlands police. Both parties have been collaborating for years on topics like forensic technology and cyber security and with this agreement it has become a more formal partnership.
The TU Delft Safety & Security Institute played a key role in bringing that agreement about and will continue to coordinate the collaboration. So that is how I became involved with the institute.
Eleonora: Transport safety is the central topic of my research. I have been following the institute quite closely since I came to Delft, as its first director, Pieter van Gelder, is part of my section. It was very nice to get some seed funding from the institute to collaborate with people from different fields and really start new research pathways.
And then during the past year I got involved in the transition team, a group of researchers from all the TU Delft faculties brainstorming on plans for the prolongation of the institute. This was something that I liked very much. The discussions we had were very interesting and constructive.
What did the transition team discuss? Did you identify certain challenges for the field that the institute should address?
Eleonora: The meetings with the transition team made me realize that it is becoming more and more important to address safety and security in an integrated way, as both the research agendas and the policy agendas are connected.
We live in a in a time where things are happening very fast in terms of technology developments and geopolitical developments. There are many different societal issues, like climate conditions, an aging population and new digital technologies, that both Justin and I and our TU Delft colleagues are working on. I think the challenge is to keep safety and security a primary focus in all that work.
And we cannot look at transport safety or industry safety or infrastructure safety or digital vulnerability separately, because one risk is associated with many others. For example, physical infrastructures increasingly become part of digital networks.
Justin: I don't like to think in terms of challenges, I prefer opportunities. Yes, both safety and security are big and important themes. But at the same time, they are not yet. Sure, there are many researchers at TU Delft working on it and they have found their way to the institute. But I think there are also still many researchers at TU Delft who don't explicitly address safety and security, even though they could as they work on topics that raise such issues. They are just not sufficiently aware of the safety and security aspects of their work, or don't have the incentives, the means, the avenues, to address them.
The challenges mentioned here by Eleonora were put central in the 10-year anniversary magazine of the TU Delft Safety & Security Institute, published in October 2023 with the title progreSSIon; Safety & Security in a Changing World.
So, there are opportunities to do more. I think one of our goals should be to facilitate these researchers not yet working on safety and security by further expanding the work that the institute has been doing in the past 10 years. This includes organizing events, creating funding opportunities and building a strong ecosystem also involving external partners from domains like the police and defence, and certain companies.
If I may play the role of the devil’s advocate for a moment... It surprises me to hear you say that many researchers who could or should be working on safety and security don’t do that yet, especially when it comes to safety. Is that not a very basic value in engineering?
Justin: No, it's not surprising. Safety is not the most attractive topic. Think for example of autonomous cars, a technology that I work on. Engineers tend to get excited by the amazing opportunities that new technologies such as AI open up, and they don't care about crashing sometimes.
Especially in startup companies, safety may often be an afterthought. OK, something went wrong. Who cares, right? As long as we push the envelope, and the investors are happy and we can raise money, it is OK.
Safety is then seen as something that somebody else will take care of at some point, for example the government must worry about. But of course, to really deploy something in society, in practice to really commercialize something, safety suddenly becomes of primary importance.
To make sure that important values are not ignored or overlooked, TU Delft pays a lot of attention to integrating ethics in the curriculum and to responsible research and innovation. Despite these efforts, could it be the case that exactly because safety has been a basic value in engineering for a long time, that people have the tendency to relegate it to the background without thinking about it too deeply?
Eleonora: Safety is indeed often taken for granted. Even though every engineer or technology researcher is aware that safety is important, there is a risk that it is seen as something that can be addressed quickly. People think that if they follow the rules or tick the boxes of design guidelines, that things will be OK.
But the systems are becoming more and more complex and with more and more interfaces with other systems. As a consequence, just following rules and regulations does not guarantee safety anymore. We need to critically rethink what safety and security mean in this day and age, and in the case at hand. Checklists no longer suffice.
This is even more the case with new technologies like the autonomous cars that Justin just mentioned, areas where the industries are rushing, and authorities are lagging with regulation. We are moving into new areas of technology or applications where guidelines don't exist about what is safe or secure.
We need to make sure that future engineers and planners are aware of these challenges and are trained to address them in their work. Safety and security education has a key role in this.
Would more focus on safety help or hinder innovation?
Justin: The European AI act does limit certain things. It will probably hinder certain technological developments, but would those really have been progress? Perhaps we can make it our business model in Europe that we have high ethical quality standards. That our products are green and safe. But yeah, it could also have drawbacks for business, there are two sides to it.
Eleonora: I fully agree that safety is sometimes seen as a barrier while it could be a strength. When you have it, you take it for granted. But when you don’t have it, for example when you have a major aviation accident, you see again that there are serious consequences in terms of human losses, economic losses, reputation damage.
But since Justin mentioned the ethical aspect, another point that was very much discussed within the transition team was that of inclusiveness. We want safety for all. And this is something that, I think, deserves a lot more attention from researchers. There is also a very important role for national and international governments to play here, to ensure that safe infrastructures, services and so on are accessible by all. We can think of countries that are not technology ready, or vulnerable and disadvantaged populations.
Can you give some examples of how inclusiveness and safety relate?
Eleonora: Sure. One example from my own field, traffic safety, is the automated vehicles which are developed in industrialized countries for industrialized countries to make the roads much safer by eliminating human errors. But the largest share of traffic fatalities takes place in low to middle income countries, which lack the required digital infrastructures. How can we make the roads safer over there?
Another example is that industry has in the past prioritized the safety of the car occupant over the safety of pedestrians and cyclists. These people are unprotected, so this is a baseline vulnerability in our traffic system. Yet as a society we want people to be able to safely use active transportation modes like walking and cycling, these are more sustainable. So here you see three values coming together: safety, inclusiveness, and sustainability.
So in the coming 5 years, equality and inclusion will be an important horizontal aspect cross-cutting our main four research themes [see text box and image], along with finding sustainable solutions, working transdisciplinary and integrating safety and security [see image].
Key research themes for the coming 5 years
- Safe by design
- Digital vulnerabilities
- Forensics & failure analysis
- Security & geopolitics
To round off this interview, this is the first time that the institute has two directors instead of one. Will you have a certain division of work?
Justin: Well, in a way it is straightforward, I will be more focussed on security, while Eleonora has more of a background in safety.
Eleonora: But that does not mean that we see them as separate things.
Justin: No, of course not, integrating safety and security concerns is key to the institute.
Eleonora: Exactly. I think that one of our first steps will be to put together a new management team with people from different faculties. We look very much forward to talking to the faculties and to people who are interested in contributing.
Our general approach will be to first build on what is already there, because in the previous periods a lot has been done and there is a very strong basis of activities and things that are already going on. And then we will also start thinking about new initiatives and opportunities.
Safety versus security
- Safety relates to unintentional threats caused by natural disasters and unintentional human behaviour, such as technical failures and human error.
- Security relates to intentional threats caused by intentional human behaviour, such as sabotage, crime, fraud and terrorism.