When Government Actors Are the Safety Problem: Why Trust & Compliance Frameworks Need Human Rights Foundations

Recent events involving state-directed human rights abuses have highlighted an uncomfortable truth: Platform trust and safety frameworks were built primarily to address individual bad actors and non-state groups, not governments perpetrating systematic violations. This gap is an operational reality that platforms are navigating constantly, often without adequate preparation.

Online platforms have a clear responsibility under the UN Guiding Principles on Business and Human Rights to respect human rights and avoid complicity. Upholding that responsibility in the face of state-directed abuses is complex and requires capacity built in advance, including foundational governance, risk assessment, and due-diligence capabilities. Embedding those capabilities across internal functions enables platforms to respond to systematic violations in rights-protective and context-sensitive ways.

The responsibility to assess and build those capabilities lies with senior leadership: platform executives, policy leads, and legal, risk, and government affairs teams.

State Actors Pose Unique Challenges

Most platforms effectively give state actors the benefit of the doubt, for sometimes understandable reasons.

Policy Mismatches. State-actor conduct tends to fall outside standard policy prohibitions.

• Policy definitions of violent extremism and terrorism (in line with academic and legal definitions) generally focus on non-state actors who lack the coercive authority of the state.

• Dangerous organizations policies often reflect the foreign policy interests of Global North governments, especially the Five Eyes, and therefore do not include actors from those governments.

• Policies on graphic violence or hateful content often don’t extend to coordinated state campaigns justifying, celebrating, or denying systematic atrocities—either because individual content items don’t cross policy thresholds or because the content comes from verified government accounts granted presumptive legitimacy.

Documentation Dilemmas. Atrocity content can spread harmful propaganda, but it also constitutes evidence for humanitarian fact-finding, criminal prosecution, international accountability, and public awareness. Over-removal actively undermines justice and transparency, but under-removal can amplify state propaganda and normalize violence.

Open Coordination. Frameworks addressing coordinated inauthentic behavior target covert manipulation, not overt state propaganda. When governments use official channels to systematically dehumanize populations or justify abuses, it’s “authentic” in the narrow sense that it really comes from the government, even when the harm patterns are similar.

Geopolitical Minefields. Determining which state actions constitute systematic human rights abuses requires judgments about contested events in complex political contexts. Platforms lack the expertise and mandate to make those judgments fairly and consistently, yet waiting for indictments or investigative findings could mean doing nothing while atrocities unfold.

Failure in this context isn’t the absence of perfect neutrality or globally uniform outcomes. Rather, it’s organizational drift under government pressure, crisis-driven inconsistency, and ad hoc rationalization without responsible governance.

Why Simple Policy Fixes Fail

Quick-fix options include extending existing policy prohibitions to fill these gaps or creating a new policy category like “state-perpetrated systematic abuses.” But these runs into immediate problems.

Definitional Quicksand. Determining whether a government’s “counter-terrorism operation” is actually a “systematic human rights violation” is deeply fraught. Platforms making these assessments essentially become arbiters of international legitimacy—a role they’re ill-equipped to perform.

Selective Application. Will platforms only act against geopolitical adversaries while overlooking abuses by actors/agencies that are commercial partners? The concern isn’t hypothetical—sanctions enforcement and content moderation both show consistent patterns of selective application.

Commercial Retaliation. States sometimes respond to reactive platform actions with regulatory threats, tax investigations, operational restrictions, and market access limitations.

Speed Mismatches. Atrocities unfold faster than platforms can investigate and process them. Emergency responses made without adequate preparation typically involve either over-removing and suppressing documentation or under-responding and thereby amplifying harm.

Employee Impacts. State-directed abuse cases place psychological and moral burden on policy staff, moderators, and operational personnel, forcing them to make decisions beyond their training or authority.

In short, mere policy changes tend to shift the burden downward, when the problem requires upward accountability.

Foundational Capabilities Platforms Need

Far more important than new policy categories are the foundations for responsible decision making that enable context-sensitive responses to major human rights risks. These foundational capabilities (decision architecture, escalation paths, cross-functional collaboration) contrast starkly with surface-level measures (policy tweaks, enforcement labels, reactive takedowns).

When state actors perpetrate systematic abuses:

• Governance structures provide decision frameworks and accountability mechanisms for politically sensitive determinations. These structures include clear escalation protocols, documented decision frameworks, and senior leadership accountability, not just trust and safety teams making best-guess calls.

• Risk assessment capabilities enable detection of systematic patterns, rather than evaluating bits of content and conduct in isolation. State-directed campaigns become visible when coordination, targeting, and narrative progression are analyzed over time and by geography.

• Evidence-aware content handling facilitates accountability while avoiding amplification of harmful propaganda. When platforms deploy content preservation, provenance, and chain-of-custody tools—all of which enable objective fact finding—the tension between visibility and intervention eases. In essence, platform leaders can choose a responsible path of preservation and non-amplification.

• User rights and due process frameworks ensure defensible, documented rationales for high-stakes decisions. Such frameworks must be robust enough to handle politically explosive appeals and transparent enough to demonstrate principled decision making under pressure from affected governments.

• Transparency mechanisms communicate complex choices to users, civil society, and regulators without exposing vulnerable individuals or compromising security. This requires pre-built infrastructure, not crisis-driven scrambling.

• Incident management protocols activate crisis response and cross-functional coordination, connecting policy, legal, security, and business leadership. State-directed abuse cases inevitably involve government relations, regulatory strategy, and commercial risk and can’t be handled by one team in isolation.

• Human rights due diligence frameworks provide context for addressing heightened risks in specific scenarios involving authoritarian regimes, conflict zones, elections, and protests, before crises emerge.

These foundational capabilities are the equivalent of strength training and conditioning: They build the muscle and capacity that enable platforms to navigate the most difficult safety and geopolitical challenges. Platforms without them face impossible choices: over-remove and suppress documentation, under-respond and amplify propaganda, or make inconsistent decisions that please no one and undermine trust.

Building Capacity Before the Crisis

Growing geopolitical instability suggests that systematic human rights abuses by state actors may become more commonplace. Given the unique role that online platforms play in transmitting information and awareness, they have a similarly unique responsibility to respect and uphold the values that undergird human rights.

They can do so by developing the capabilities outlined above before human rights crises occur, not during or after them—including through relationships with trusted partners in civil society, humanitarian organizations, and fact-finding bodies. Platforms that invest in these capabilities ex ante position themselves to navigate state-directed abuse scenarios in principled, operationally sustainable ways.

To be clear: This is not about platforms becoming human rights arbiters. It is about platforms becoming organizationally capable of confronting state power without abandoning their own principles.