Facebook 202X: How It Becomes The Multitool For The Modern Totalitarian, Part 1


Image Credits:

Dan Kaplan


Dan Kaplan helps startups craft stories and build marketing funnels that attract and close more customers and recruits. He blogs about marketing and growth at Threadling and is preparing to launch Dispatches From The Future, a podcast about the future of humanity.

More posts from Dan Kaplan

Editor’s note: Dan Kaplan helps startups tell their stories. He’s done marketing for Twilio, Asana and Salesforce and blogs about marketing, growth, and storytelling at Threadling.

Maybe it’s naive, but I’d always hoped that Facebook would evolve into the Internet’s humanizing, unifying force — the one that would stretch across geographical, national, religious and racial boundaries to show us that, even when we disagree about the tactics, we all want the same things for ourselves and our children: to be safe, to be at peace, to have food and shelter, respect and opportunity and hope.

Facebook 202x_1

Indeed, with the combined social force of Facebook and the immersive power of Oculus Rift, I still think this vision remains possible.

But let’s face some realities:

  • For all of the promise it may hold, Facebook is a massive data maw that uses variable reward mechanisms to feed our brains dopamine while collecting copious amounts of our personal information.
  • With this all-consuming hunger for data on the things that make us tick, Facebook is creating the biggest online identity database and the most powerful media-targeting engine ever known.

Facebook 202x_2

As a marketer who believes that content marketing will become the dominant form of advertising, this is a boon. When you understand your target market and know how to produce content that engages it, there is simply no better way than Facebook to get this content in front of exactly the right people.

But as someone who cares about the future of democracy and human autonomy and wants these things to persist and not collapse into some form of corporatist totalitarianism, my feelings are a bit more mixed.

In the next couple of posts I write, I’m going to explore some potential trajectories for Facebook’s future. While it’s possible that Facebook’s walls won’t hold up against the multi-front assault from social mobile apps, I think it’s more likely than not that Facebook will be around in some form in 10 years. Some of these forms are beneficent. Others, not so much.

Today, let’s take a look at one of the less ideal paths, where Facebook evolves into the multitool for the modern totalitarian. Like most projections into the future, this is just speculative fiction.

Tool No. 1: Surveillance at Scale

In East Germany under communism, the secret police (the Stasi) gathered data by hand, relying on stakeouts and a massive network of civilian informants to build detailed files on its targets, whose ranks included pretty much everyone.

Under the watchful eyes of the Stasi, whose file cabinets stretched for miles, case agents were responsible for keeping tabs on individuals and groups. Though these agents had large and ever-growing dossiers at their disposal, they were essentially confined to what they could capture on paper and keep in their minds.

The 21st century totalitarian has no patience for this antiquated approach. And nor should he: compared to Internet-scale data collection, the Stasi’s methods are slow, inefficient, and prone to lapses in human judgment.

No, the 21st century totalitarian needs a modern way to capture and catalog the activities, psychologies, and political sentiments of the population in real time. Ideally, this would be a place online where people share freely and regularly about themselves.

Tool No. 2: Data Mining

Of course, collecting massive amounts of data is the (relatively) easy part. Once you have access to a large, rich and ever-growing online identity database, you need algorithms that can mine its activities and uncover threats to the status quo.

Along with potential terrorists, these threats include:

  • subversives
  • whistleblowers
  • radicals

Wouldn’t it be nice if you, the 21st century totalitarian, could build a detailed psychological profile of these types and use your massive identity database and a bit of sophisticated machine learning to find others? Sure, you’d surface a substantial number of false positives, but it’s nothing some light investigation and/or interrogation couldn’t suss out.

With such a tool at your disposal, it’d be like having the pre-cogs from Minority Report, but instead of weirdly-psychic humans drugged up and living in a bath, you’d have an All-Seeing Machine.

Tool No. 3: Mass Manipulation

The 20th century totalitarian relied on absolute control of the flow of information. Writing official textbooks, burning subversive materials, censoring the media, and publishing a non-stop flow of propaganda was simply the way of things. But for the 21st century totalitarian, the fragmentation of TV and the free-form frontiers of the Internet complicates this strategy.

Sure, you could just co-opt cable news, but there are so many channels! Radio? That’s so 1933.

No, while these blunt forms of manipulation work for Fox News and Rush Limbaugh, they are too obvious to work on everyone else.

Today’s totalitarian demands a more subtle way to influence cultural and political sentiment. But if you got your hands on an algorithmically filtered newsfeed? One that could control the stories people see every day and influence their emotions across geographic, political and economic lines? You’d be in business.

Where Does That Leave Us?

I don’t know Mark Zuckerberg personally, but I believe he has a vision that involves eliminating artificial barriers between people and building a better world.

While Facebook has made a few boneheaded privacy blunders in the past, I have been willing to give the company the benefit of the doubt. I have trusted that Zuckerberg and his senior staff have benevolent intentions for the power they are amassing.

But then there was the mood-influence study that scandalized us for a couple of weeks this year. Facebook changed the tone of content showing up in people’s feeds to test the impact it could have on their moods. The results, not too surprisingly, suggested that Facebook has the power to manipulate sentiment at scale.

I’d always sort of worried that Facebook, despite the good intentions of the people building it, could eventually evolve into the Totalitarian Multitool. The disclosure of this study brought the whole thing home.

Even though I’m hopeful that the ideals of democracy and human freedom will persevere against the paranoia of our modern age, there’s a strong case to be made that some of these ideals face a risk of death in the not-too-distant future.

Given how easy it is to scare people about the scary-seeming-but-actually-low-risk Ebola, and how dumb we all get when we are afraid, it is not crazy to think that under the wrong circumstances — like one or two more mass-scale terrorist attacks on major cities — modern democracy gives way to something akin to 1984.

If Big Brother were to seize the reins of power, sure, he’d use the cable news the way it’s being used today. But Facebook’s data maw, targeting power and sentiment-manipulation capabilities would be far more insidious.

Whether this is what we become or not comes down to the future we choose to build.

More TechCrunch

After Apple loosened its App Store guidelines to permit game emulators, the retro game emulator Delta — an app 10 years in the making — hit the top of the…

Adobe comes after indie game emulator Delta for copying its logo

Meta is once again taking on its competitors by developing a feature that borrows concepts from others — in this case, BeReal and Snapchat. The company is developing a feature…

Meta’s latest experiment borrows from BeReal’s and Snapchat’s core ideas

Welcome to Startups Weekly! We’ve been drowning in AI news this week, with Google’s I/O setting the pace. And Elon Musk rages against the machine.

Startups Weekly: It’s the dawning of the age of AI — plus,  Musk is raging against the machine

IndieBio’s Bay Area incubator is about to debut its 15th cohort of biotech startups. We took special note of a few, which were making some major, bordering on ludicrous, claims…

IndieBio’s SF incubator lineup is making some wild biotech promises

YouTube TV has announced that its multiview feature for watching four streams at once is now available on Android phones and tablets. The Android launch comes two months after YouTube…

YouTube TV’s ‘multiview’ feature is now available on Android phones and tablets

Featured Article

Two Santa Cruz students uncover security bug that could let millions do their laundry for free

CSC ServiceWorks provides laundry machines to thousands of residential homes and universities, but the company ignored requests to fix a security bug.

17 hours ago
Two Santa Cruz students uncover security bug that could let millions do their laundry for free

OpenAI’s Superalignment team, responsible for developing ways to govern and steer “superintelligent” AI systems, was promised 20% of the company’s compute resources, according to a person from that team. But…

OpenAI created a team to control ‘superintelligent’ AI — then let it wither, source says

TechCrunch Disrupt 2024 is just around the corner, and the buzz is palpable. But what if we told you there’s a chance for you to not just attend, but also…

Harness the TechCrunch Effect: Host a Side Event at Disrupt 2024

Decks are all about telling a compelling story and Goodcarbon does a good job on that front. But there’s important information missing too.

Pitch Deck Teardown: Goodcarbon’s $5.5M seed deck

Slack is making it difficult for its customers if they want the company to stop using its data for model training.

Slack under attack over sneaky AI training policy

A Texas-based company that provides health insurance and benefit plans disclosed a data breach affecting almost 2.5 million people, some of whom had their Social Security number stolen. WebTPA said…

Healthcare company WebTPA discloses breach affecting 2.5 million people

Featured Article

Microsoft dodges UK antitrust scrutiny over its Mistral AI stake

Microsoft won’t be facing antitrust scrutiny in the U.K. over its recent investment into French AI startup Mistral AI.

18 hours ago
Microsoft dodges UK antitrust scrutiny over its Mistral AI stake

Ember has partnered with HSBC in the U.K. so that the bank’s business customers can access Ember’s services from their online accounts.

Embedded finance is still trendy as accounting automation startup Ember partners with HSBC UK

Kudos uses AI to figure out consumer spending habits so it can then provide more personalized financial advice, like maximizing rewards and utilizing credit effectively.

Kudos lands $10M for an AI smart wallet that picks the best credit card for purchases

The EU’s warning comes after Microsoft failed to respond to a legally binding request for information that focused on its generative AI tools.

EU warns Microsoft it could be fined billions over missing GenAI risk info

The prospects for troubled banking-as-a-service startup Synapse have gone from bad to worse this week after a United States Trustee filed an emergency motion on Wednesday.  The trustee is asking…

A US Trustee wants troubled fintech Synapse to be liquidated via Chapter 7 bankruptcy, cites ‘gross mismanagement’

U.K.-based Seraphim Space is spinning up its 13th accelerator program, with nine participating companies working on a range of tech from propulsion to in-space manufacturing and space situational awareness. The…

Seraphim’s latest space accelerator welcomes nine companies

OpenAI has reached a deal with Reddit to use the social news site’s data for training AI models. In a blog post on OpenAI’s press relations site, the company said…

OpenAI inks deal to train AI on Reddit data

X users will now be able to discover posts from new Communities that are trending directly from an Explore tab within the section.

X pushes more users to Communities

For Mark Zuckerberg’s 40th birthday, his wife got him a photoshoot. Zuckerberg gives the camera a sly smile as he sits amid a carefully crafted re-creation of his childhood bedroom.…

Mark Zuckerberg’s makeover: Midlife crisis or carefully crafted rebrand?

Strava announced a slew of features, including AI to weed out leaderboard cheats, a new ‘family’ subscription plan, dark mode and more.

Strava taps AI to weed out leaderboard cheats, unveils ‘family’ plan, dark mode and more

We all fall down sometimes. Astronauts are no exception. You need to be in peak physical condition for space travel, but bulky space suits and lower gravity levels can be…

Astronauts fall over. Robotic limbs can help them back up.

Microsoft will launch its custom Cobalt 100 chips to customers as a public preview at its Build conference next week, TechCrunch has learned. In an analyst briefing ahead of Build,…

Microsoft’s custom Cobalt chips will come to Azure next week

What a wild week for transportation news! It was a smorgasbord of news that seemed to touch every sector and theme in transportation.

Tesla keeps cutting jobs and the feds probe Waymo

Sony Music Group has sent letters to more than 700 tech companies and music streaming services to warn them not to use its music to train AI without explicit permission.…

Sony Music warns tech companies over ‘unauthorized’ use of its content to train AI

Winston Chi, Butter’s founder and CEO, told TechCrunch that “most parties, including our investors and us, are making money” from the exit.

GrubMarket buys Butter to give its food distribution tech an AI boost

The investor lawsuit is related to Bolt securing a $30 million personal loan to Ryan Breslow, which was later defaulted on.

Bolt founder Ryan Breslow wants to settle an investor lawsuit by returning $37 million worth of shares

Meta, the parent company of Facebook, launched an enterprise version of the prominent social network in 2015. It always seemed like a stretch for a company built on a consumer…

With the end of Workplace, it’s fair to wonder if Meta was ever serious about the enterprise

X, formerly Twitter, turned TweetDeck into X Pro and pushed it behind a paywall. But there is a new column-based social media tool in town, and it’s from Instagram Threads.…

Meta Threads is testing pinned columns on the web, similar to the old TweetDeck

As part of 2024’s Accessibility Awareness Day, Google is showing off some updates to Android that should be useful to folks with mobility or vision impairments. Project Gameface allows gamers…

Google expands hands-free and eyes-free interfaces on Android