News – Could an Algorithm Help Tackle Online Gender Abuse?

Despite the epidemic of abuse facing women and non-binary people, identifying and defining it can be difficult.

Researchers at the National Robotarium have received £1 million in funding to help develop advanced ‘machine learning’ algorithms to improve the detection, intervention, and prevention of online gender-based abuse.

The funding came from UK Research and Innovation (UKRI) and will help create state-of-the-art AI tools that use a broad variety of viewpoints, perspectives and experiences to spot online abuse.

The project was created by researchers at the National Robotarium, hosted by Heriot-Watt University and the University of Edinburgh, in response to a recent report published by Glitch, the UK’s leading charity against online abuse.

This highlighted the current “epidemic of online abuse” being experienced particularly by women and non-binary people, with 46% of those reported experiencing attacks online since the beginning of COVID-19. 29% of people who had experienced online abuse prior to the pandemic reported it being worse during lockdown.

Gender was the most frequently cited reason for online abuse, with 48% of respondents reported suffering from gender-based online violence.

Principal investigator and professor of conversational AI at the National Robotarium, Verena Rieser, said: “Figures like those included in the Glitch report continue to highlight the discriminatory effect of gender-based online abuse and the disproportionate impact it has on non-binary people and women.

“One challenge is that the current means of identifying and defining abuse often relies on majority social viewpoints. From our research, we know these viewpoints don’t necessarily correlate with the experiences of victims.”

As such, National Robotarium researchers will engage with experts on gender-based violence and online harassment to build mechanisms that automatically generate counter-narratives aimed at perpetrators and a chatbot for providing proactive support to victims and survivors.

A key ambition of the project is to increase digital literacy concerning the safe use of social media from an early age. This will be achieved through the development of learning materials which focus on helping children identify what online gender-based abuse is and how to deal with it.

Additional outcomes from the project will include data on the type and frequency of online abuse experienced by victims. New and updated datasets will be shared with the wider research community for the purposes of developing further strategies.

Rieser continued: “Our approach is based on the framework laid out by the Scottish Government’s Equally Safe strategy which aims to prevent violence against women, young people and children. In addition to automatically detecting online abuse, it will aim to rethink what it is we need to detect, how best to support victims and the role education can play as a tool for prevention.

“Outcomes from the project will help create online spaces that are equally safe irrespective of someone’s gender, race or background and provide more effective and transparent means of moderation – giving users more control over their online experiences.”

Co-researcher and reader in computer science education at the National Robotarium, Dr Fiona McNeill, added: “Activities to mitigate online gender-based violence tend to focus on adults, but this is a huge problem for children and young people as well.

“As part of the project being driven by the National Robotarium, we will be working with young people to understand their experiences of online abuse, the language they use around this and the way in which young victims need to be supported.

“Through this interactive work, we will create educational materials that help young people understand and recognise online gender-based violence, to gain confidence in responding to it – either as a victim or a bystander – and to recognise if they are perpetrating it.”