Online Hate Ecosystem Primer

logo

Online Hate Ecosystem Primer

In order to combat hate online, it’s important to understand the ecosystem in which it thrives. To that end, experts at ADL have developed diagrams that map out the web of connections between and among bad actors, targets, institutions, behaviors, causes, and outcomes. The connections often overlap and influence one another. The ecosystem is complex and involves interactions among various actors across multiple ideologies, biases and functions, with the ensuing hate expressing itself in ways that range from subtle harassment to explicit calls for violence.

By visualizing the online hate ecosystem, we can create a shared vocabulary around how online hate arises—the reasons why people engage in hate, the targets of that abuse, the forms the abuse takes, the tools and platforms used to commit or mitigate abuse, and the damaging consequences of hate.

As a starting point, we use three diagrams to depict the ecosystem of online hate.

The Perpetrators/Targets/Mitigators/Amplifiers diagram identifies and examines the four primary “players” within the online hate ecosystem and the context in which each acts. The Perpetrators section examines who perpetrates online hate, what types of hate are happening, and what compels people to post hateful content. The Targets section explores why and how people are targeted, as well as the outcomes of being targeted by online hate. These two sections can mirror each other: religious bigotry, for example, can be committed by those of one creed who target those of another, with targeted individuals and communities including Jews, Muslims, Christians, Mormons and other faiths. Outcomes for targets can be subtle (hurt feelings) to devastating (lost jobs, reputations and lives).
 

Ecosystem

The Amplifiers and Mitigators sections explore the people, places and tactics across social spaces through which hate spreads and rhetoric is broadcast and amplified, or, helpfully, can be mitigated and blocked. What are the activities that occur in gaming, on social platforms, and in other media that spread online hate? Who is working to stop online hate? Looking at the Mitigator section, we see system actors—governments, lawmakers, media—as well as individualș—teachers, parents, celebrities and bystanders—all working in various ways to educate, intervene and develop systems for stopping online hate.

The outer ring of this first diagram lists possible reasons why someone might hold a specific ideology or participate in online hate; what might happen as a result of being targeted by online hate; the types of mitigation efforts and policies that can be used to fight online hate; and the areas within social platforms where we most often see online hate amplified.

By then breaking down the higher-level classifications into discrete categories, actions, and outcomes, we can better understand the breadth of each of these classifications and the relevant context, beginning to understand where there are overlaps between areas (e.g., the media can act as a perpetrator, mitigator and amplifier of hate) and strategize solutions based on causalities and other relationships. We also create a shared vocabulary and start to understand the nuances between the contexts. 

Perpetrators/Targets/Mitigators/Amplifiers Diagram
 

Hate Ecosytem

The Relationships Between Actors diagram looks at the problem of online hate from a different lens, examining the deeper relationships between the actors, specific places and opportunities to intervene and how to make changes in the system. Read from left to right, it walks us through who posts online hate (inputs), where they post and what happens when they participate in hate against their targets (outputs). What are the relationships between the parts, what paths connect them? Where are the perpetrators found, and where the mitigators? It also includes an option to follow along the path that those who fight hate can take and note where it intersects with the inputs and outputs. This helps illustrate possible entry points for those countering hate. For example, one path traces how nonprofits train law enforcement to recognize hate speech – speech that, as we’ve seen so often in recent years, can radicalize and incite offline violence and terror. This is a very high-level visualization of the landscape and is intended to be an overview to prompt discussions and deeper dives into more granular details around each area.

Relationships Between Actors Diagram
 

Hate Ecosytem

The Corporate Value Chain diagram depicts companies that provide infrastructure to online businesses. Who makes money from online hate and by what means? How is online hate supported through site visitors and advertisers? The complexity of the number and types of infrastructure companies -- by which we mean that part of the tech world that provides services such as payment processors, application hosts, and security providers -- is often hidden from the general public. When tech companies choose to not address hate on a platform they have built and managed, it becomes important for infrastructure companies to evaluate whether to provide services or products to that company. These companies have tended to stay neutral when considering online hate content beyond what is strictly prohibited by law, but we are now beginning to see some of them act to deny their services to egregious purveyors of hate. Hopefully through awareness and pressure, more of them can be compelled to reject supporting online businesses that profit off hate.

These diagrams are not exhaustive or all-inclusive. They represent one expert-informed view of the ecosystem. We reviewed these across multiple disciplines, both inside and outside of ADL, with subject matter experts across the technology industry and academia as well as user experience design experts to develop these visualizations and refine them through dozens of review sessions.

Corporate Value Chain Diagram