“Hey Siri, can you hear me?”
Article - June 21 2021
Being able to interact with technology with our voices may have seemed far-fetched decades back, but with the advent of AI-assisted technologies like voice recognition, smart home devices that we control with our voices are increasingly commonplace. “For many, this is a nice-to-have feature,” said Wiebke Toussaint, a PhD candidate at the Faculty of Technology, Policy and Management, “but for example, for the visually impaired, voice activation can be life changing.”
However, as users of voice technologies, we also realise that they do not always perform perfectly: sometimes your phone’s voice assistant reacts to someone else's voice, other times it misinterprets what you are saying. In her current research, Wiebke investigates speaker verification, a form of biometric identification that grants access to voice assistants. Her research suggests that voice assistants’ success at authorising users is strongly influenced by a user’s accent and gender. This impacts not only how smoothly we interact with these technologies, but also has implications for privacy, safety and security. As they become parts of our everyday lives, when should we trust these voice-operated systems?
A double take on trustworthiness
Trustworthiness is a precondition for trust, and an important consideration in engineering design. Traditionally, technologies are built on established engineering principles and knowledge, there are agreed-upon specifications and tests to ensure that they behave as expected, and engineers have a long history of thinking about and making complex trade-offs between design gains and risks. “Take road design as an example: roads are designed with a particular curvature so cars can efficiently go around a corner, while keeping in mind that the sharpness of the curve or where a crossing is placed will impact safety.”
Running machine learning models on smart devices, however, introduces new layers of complexity. As a start, it is important that application requirements are considered when making decisions about trade-offs. “Imagine that you use your voice to stop your smart car: it’s critical that your car will always stop when you instruct it to, but it’s probably OK if the voice system confuses you with your partner and lets them drive your car,” Wiebke illustrated, “but when opening your front door, it is the opposite- it is more important that unauthorised people’s voices are not recognised so to keep intruders out.” Voice assistant failures are not just matters of safety and security, which are considered important aspects of trustworthiness, but can also lead to discrimination. Consumers expect smart devices to work equally for everyone, but that is typically not the case: voice assistants, for example, could be more or less likely to open your front door to intruders depending on your accent, age or gender.
Be transparent to gain trust
Perfect performance for everyone is generally accepted as unattainable, but negative effects can be mitigated by following good design practices. “Think of it like specifying a maximum load for a ladder: if a ladder is rated for a 30kg load, you can use it if you weigh 60kg, but there’s a good chance it will break. Clearly communicating that the ladder has a maximum load of 30kg is the first step. The next step is to create a standard that specifies that ladders for children can carry 30kg and ladders for adults should carry a 110kg load.” By being open about for whom models work, and how well they work, end users can understand better what they can expect from these smart devices. Standards offer a way for system designers and developers to describe the performance and limitations of their systems, for individual applications. Tests and certification then ensure that products and systems meet the required quality and ethical standards.
Incorporating user voices in trustworthy designs
To understand the trustworthiness of smart systems and create specifications, it is crucial to have a sufficiently large dataset from diverse users when building and testing machine learning models, but this data is not easy to obtain. To this end, Wiebke and her colleague Alejandra Gomez Ortega at the Faculty of Industrial Design Engineering are in the process of setting up a data donation project, where they will ask Google Assistant users to opt-in to data collection for 3 months, download the data from Google and share it with them. By creating an open collection of voice assistant data, the research and user communities can work towards building a fuller picture of system performance and creating specifications and tests.
Beyond data collection, how can the research and engineering community engage end-users in designing trustworthy and fair technologies? “Many people are passionate about these issues, but we all have vastly different understanding of, for example, what ethics is or means, and not everyone has the technical expertise or language to talk about it,” Wiebke drew from her experience, “Discussing topics like data ethics is difficult because they’re inherently abstract.” To this end, Wiebke and her colleagues have recently designed a series of workshops, in which participants co-created Data Daemons, fictitious beings that are owned by individuals and have the responsibility of managing and controlling their owner’s personal data in their owners’ best interest. This conceptualisation created a shared language for the participants that allowed them to explore their relationships more concretely with their personal data. “As AI becomes an increasing integral part of our lives, I’d like researchers to become a bit more creative about involving users in participatory research, to build true user-centred and fairness-based designs.”
Wiebke’s research is supported by her supervisor Dr Aaron Yi Ding, at the Cyber Physical Intelligence Lab in the Information and Communications Technologies Section, Department of Engineering Systems and Services, Faculty of Technology, Policy and Management.
Related stories:
This story is part of the Open AI research at TU Delft series - also read the introduction and other stories in this series.