A virtual inner voice can help develop social skills

Social interactions make up a large part of our life, but social skills don’t come naturally to all. You can train them with self-help books or courses, and in recent years also with training systems based on supporting technologies such as virtual reality. These systems are low-cost  and accessible, but have several limitations. TU Delft’s Ding Ding proposes a system that combines virtual reality experiences with virtual cognition: an inner voice that offers reflection, instruction and encouragement. This system has the potential to increase people’s self-efficacy: the belief in their own abilities. Ding will defend his PhD on this subject on 4 June. 

Current technology-supported systems for social skills training offer users the opportunity to observe or practice social skills, but fail to explain to them why they should behave in a particular fashion. “Such learning by doing can help you master a skill, but there is also the risk that you experience failure and lose heart”, explains Ding Ding, researcher in the Interactive Intelligence group. Ding’s research focuses on self-efficacy, the belief people have in their own abilities. Self-efficacy can increase people’s motivation and willingness to engage in social interaction. “Other systems only look at whether you are able do something or not, but you will only undertake things in real life once you are confident you can do them”, says Associate Professor Willem-Paul Brinkman, Ding’s promotor. 

The inner voice
To increase the effectiveness of the VR learning experience, Ding came up with the idea of adding virtual cognition: an inner voice that simulates the thinking process. Your inner voice is the voice you hear in your head when you are reading, for example. Ding first examined sound parameters for inner voices, as people experience their inner voice differently from their speaking voice. This inner voice was then added to a series of VR experiments, a bit like a voice-over in a movie. 

Negotiations
Though many people may equate negotiations with buying a new house or car, negotiating is in fact part of everyday life. We negotiate with spouses or children about chores or where to go on holiday, and in a professional setting about our workload or deadlines. That makes the ability to negotiate an important social skill. Hence, it also provides an interesting setting to research the effectiveness of a social skills training extended with virtual cognition.

Ding set up a series of experiments where participants could observe conversations between an employer and an employee. They could see and hear what was being said, but thoughts were also made explicit. This inner voice, or virtual cognition, was used to introduce knowledge and principles and to reflect on the process, all from the point of view of the employer. In the first scene, the employer would reflect on the upcoming negotiation. The following scenes ranged from an employee being always late to an employee handing in their immediate notice. “That way, participants could experience the theoretical side of negotiations being applied to real-life situations”, Ding explains. 

Self-motivation
Even though participants did not have an active role in the negotiations, they did report increased knowledge of what works and what does not work in such social interactions, compared to a control group. They also felt they had become better negotiators by taking part in the experiment, meaning their self-efficacy had increased. If the virtual cognition included self-motivational statements (e.g. ‘I think I can do this, I am quite a good negotiator already’) this effect would be even greater. The effects were not short-lived either, as participants also reported them in a follow-up two weeks later.

“Taking part in a social skills training is awkward. By giving people a virtual experience, they can build up enough confidence to take part in an actual training. So such a system can be an easily accessible addition to the total training offer out there”, says Brinkman. While Ding’s research has shown that virtual cognition is a promising concept to improve social skills training systems, it may have a wider scope too. “Having difficulty with social interaction can also have a  pathological cause. The inner voice concept may in the future also play a role in therapy sessions.”

More information
Thesis defense Ding Ding: Design and evaluation of simulated reflective thoughts in virtual reality exposure 
Promotor 1: Prof.dr. M.A. Neerincx, Promotor 2: Dr.ir. W.P. Brinkman.

Link to thesis in TU Delft Repository.

Contact information: Ilona van den Brink (press officer TU Delft), i.vandenbrink@tudelft.nl, +31 (0)15 2784259