Finding Common Ground on Facial Emotions

AI systems need input from facial emotion databases to be able to recognize emotions. But in order to know what a certain emotion looks like, humans have to rate images of facial emotions. But they don’t always agree: One person might rate is an emotion as ‘surprise’, when in reality that person is scared. That’s why Mana’s bachelor research project focused on the different methods used to determine interrater agreement when it comes to recognition of facial emotions. 

Which bachelor's programme are you doing, and why did you choose this one?

I am doing the Computer Science & Engineering bachelor's. And I chose this bachelor's degree because of the future prospects I have once I finish my studies at TU Delft. I will be able to go into the field of artificial intelligence, and help improve the world through technology. Right now I’m doing a research project that will last for around 10 weeks. It gives us a glimpse into the future of what it’s like to work on a master or PhD thesis. So that when we have write a big thesis, we have some hands-on experience of how to manage it.

What is your research project focused on?

My research focuses on facial emotion recognition. I analyse databases to see how they account for subjective assessments of emotion recognition and raters' biases. Because it can be difficult for different raters to recognize a facial emotion. Contempt can be confused for anger, and surprise can be interpreted as fear. Those databases will be used for machine learning algorithms and AI and systems that will be used in the  real world. That’s why it’s important that the data is correct. My research question is: What are the differences in interrater agreement methodologies in the different published data sets about facial emotion recognition? 

And what's the societal impact of the project?

These databases are used to shape the world as we know it. They are really important to be able to move forward in any field. Emotion recognition can be used for self-driving cars, and it’s actually used by Uber. And the example that’s given in the introduction just shows how important good working emotion recognition features can be. And emotion recognition can also be used to check-up on elderly people, to see if they're happy or sad. And by doing research you also contribute to society’s knowledge and technological advancements.

Were there any challenges within this research project?

Yes, I think that every project has its challenges. And we learn the most from challenges. The challenge here was that it was my first “real” research in an academic environment. I had some challenges when it came to the systematic literature review methodology process. I had gathered 12,000 papers at some point, and I was not able to read them all of course. It was important to get the literature review correct, because my research has to be reproducible. The aim for the literature review was to learn to search in the right way and to go in the right direction.

Something that I found challenging in general, when it came to the study, was the application process. The application process is quite strict. There are many applicants, and only around 550 students are accepted for the Computer Science & Engineering bachelor. As part of the process you have to do a Cognitive Skills Test and a team assignment. It was quite stressful to be able to prepare correctly. But I am glad to say that I passed with flying colours! I learned that everything is doable if you have a plan.

And something that was also challenging, was the housing situation. There’s not a lot of student housing available. And Dutch students sometimes post about available spots on Facebook, for example. But because I was an international it was more difficult to get housing, because Dutch people prefer to live with other Dutch people. I really didn’t like that. Fortunately I did find a place to live after a while.

What do you like about studying at TU Delft?

I would like to do a master's and also a PhD if possible. That’s why I really liked to gain experience with research. I think my bachelor's has already given me a good base to be able to build upon. I want to continue within the field of Computer Science & Engineering. I would like to work on artificial intelligence, and specifically health-related AI.

I got some hands-on experience with health-related research during the second year of the bachelor. I worked on a project for a hospital in Belgium. The hospital wanted an application with which they could send medical images from one hospital to another hospital in Belgium. And they wanted an application that could be used on a national level and that everyone could use. Transferring medical images is not as easy as sharing an image on WhatsApp. There’s a lot of data involved with it. Together with four other teammates I developed this application. We did a lot of coding and building the application. A LOT of debugging. This project didn't involve any AI, but I still really liked it. We went to a meeting in Brussels with some big companies, like Phillips, that were actually making a similar system. I thought it was really interesting!

I think it would be amazing to be able to use artificial intelligence to improve health-related issues. I want to use AI to find new treatments and new solutions to treat patients. I think it’s really important and rewarding to make an impact on society and in that way be able to help others.