Constructing Online Harms: How “Trust and Safety” Translates Safety Into Risk
2026-09-08
How are online harms recognized and prioritized within platform companies? Academics, journalists, and regulators have focused on identifying how digital platforms produce dangerous effects, but how harms are constructed within these organizations remains largely opaque. Based on 42 interviews with Trust and Safety specialists—the workers who define, operationalize, and create mechanisms to prevent or mitigate harmful effects—and ethnographic observation of industry forums, I describe how these workers transform concerns about platform dangers into actionable rules and processes. First, they position themselves as advocates for users, claiming moral authority to represent them within their organizations. Second, they translate abstract concerns into concrete organizational risks. Third, they publicize their companies’ harm-mitigation efforts in industry forums, performing responsiveness to critics and distributing responsibility across the sector. User harms become organizationally visible when they can be translated into corporate risks—legal liability, reputational damage, or revenue threats—and technical problems.