As part of its ongoing efforts to create safer and more enjoyable gaming environments, Ubisoft is introducing ToxBuster, an innovative tool designed to help eliminate harmful behaviour and abuse in online chats. This cutting-edge solution is the result of a major collaboration between Ubisoft La Forge, Montreal’s user research lab Pixel, and the Ubisoft Data Office (UDO).
Domenico Tullo, a research and development scientist, and Andrea Feder, a production manager, share more about this groundbreaking technology.
The problem of online toxicity
Online toxicity is a major challenge in video games, especially in chat channels where players talk in real time. Insults and inappropriate comments do more than just ruin the gaming experience; they can affect mental well-being, increase stress and anxiety, make playing less fun, and make the environment less welcoming for new players. Andrea Feder explains:
“We realized that toxic behaviour and offensive language could significantly impact players’ well-being and overall enjoyment. We needed to find a solution.”
Ubisoft has been working for years to fight online toxicity with initiatives like the Fair Play Program, the Good Game Playbook, and the Zero Harm in Comms research project. Now, the experts at Ubisoft La Forge are introducing a new solution. Domenico Tullo shares:
The main motivation for developing ToxBuster was to improve the online gaming experience by making the multiplayer space more enjoyable and reducing harmful texts. And we think we’ve achieved that.”
An innovative solution
ToxBuster is a sophisticated tool designed to spot harmful or offensive language in chats, with the potential to be used in real time. It leverages a type of AI called transformer algorithms. Simply put, “this means it can understand and analyze the context of words and phrases in a conversation, allowing it to accurately detect toxic behaviour and help create a safer and more enjoyable gaming environment,” explains Feder.
So, what makes ToxBuster special?
ToxBuster is unique because of its advanced ability to understand context. It uses sophisticated algorithms to interpret the context of conversations and distinguish between game-specific language and genuinely harmful language. This is how it detects toxic behaviour.
Understanding conversations was a big challenge for the tool. Domenico shares:
Making sure the model could correctly understand and interpret the context of conversations was tough. For example, telling apart a game-specific term like ‘headshot’ in a first-person shooter from genuinely harmful language required complex algorithms and a lot of work.”
The creativity of trolls and their strategies to bypass traditional detection methods also put ToxBuster’s capabilities to the test, as the scientist explains:
“Malicious users use slang, abbreviations, and coded language to bypass preventive measures in online chats. Adapting ToxBuster to these evolving tactics has been and remains a major challenge.”
Lastly, user privacy was a top priority. The La Forge team made sure to protect players’ data and privacy, handling data carefully and ensuring GDPR compliance. Andrea Feder reassures: “We want to help our players, but it was crucial to ensure their privacy. We store the data securely, use it responsibly, and guarantee its confidentiality.”
Towards a healthier future
The team behind ToxBuster is excited to share that this tool is already making a difference in the fight against toxicity by being part of the moderation process. Thanks to collaborations with various Ubisoft services and production teams, moderators can now better manage toxic behaviour. Domenico is confident that the tool will keep getting better:
“While we’ve made progress, this is just the beginning. The team is committed to continuing to refine and expand its capabilities in the future. ToxBuster will greatly contribute to creating a positive and respectful gaming environment for everyone,” he concludes.
Acknowledgements
The creation of ToxBuster wouldn’t have been possible without the dedication of several members from the Ubisoft Data Office (UDO), Montreal’s user research lab Pixel, the user research department, and La Forge teams: Zachary Yang, a research intern, led the research and played a key role in building the model. Amanda Jarrell and Nicolas Grenon-Godbout guided the tool’s direction. Josiane Van Dorpe, Souhail Ben Ali, Andrea Feder provided invaluable help throughout the development process. Domenico Tullo leads the tool’s transition to practical applications.
