AI

How generative AI is accelerating disinformation

Comment

TC Disrupt 2023 disinformation
Image Credits: Mark Reinertson/TPG for TechCrunch

People are more aware of disinformation than they used to be. According to one recent poll, nine out of 10 American adults fact-check their news, and 96% want to limit the spread of false information.

But it’s becoming tougher — not easier — to stem the firehose of disinformation with the advent of generative AI tools.

That was the high-level takeaway from the disinformation and AI panel on the AI Stage at TechCrunch Disrupt 2023, which featured Sarah Brandt, the EVP of partnerships at NewsGuard, and Andy Parsons, the senior director of the Content Authenticity Initiative (CAI) at Adobe. The panelists spoke about the threat of AI-generated disinformation and potential solutions as an election year looms.

Parsons framed the stakes in fairly stark terms:

Without a core foundation and objective truth that we can share, frankly — without exaggeration — democracy is at stake. Being able to have objective conversations with other humans about shared truth is at stake.

Both Brandt and Parsons acknowledged that web-borne disinformation, AI-assisted or no, is hardly a new phenomenon. Parsons referred to the 2019 viral clip of former House Speaker Nancy Pelosi (D-CA), which used crude editing to make it appear as though Pelosi was speaking in a slurred, awkward way.

But Brandt also noted that — thanks to AI, particularly generative AI — it’s becoming a lot cheaper and simpler to generate and distribute disinformation on a massive scale.

She cited statistics from her work at NewsGuard, which develops a rating system for news and information websites and provides services such as misinformation tracking and brand safety for advertisers. In May, NewsGuard identified 49 news and information sites that appeared to be almost entirely written by AI tools. Since then, the company has spotted hundreds of additional unreliable, AI-generated websites.

“It’s really a volume game,” Parsons said. “They’re just pumping out hundreds — in some cases, thousands — or articles a day, and it’s an ad revenue game. In some cases, they’re just trying to get a lot of content — make it on to search engines and make some programmatic ad revenue. And in some cases, we’re seeing them spread misinformation and disinformation.”

And the barrier to entry is lowering.

Another NewsGuard study, published in late March, found that OpenAI’s flagship text-generating model, GPT-4, is more likely to spread misinformation when prompted than its predecessor, GPT-3.5. NewsGuard’s test found that GPT-4 was better at elevating false narratives in more convincing ways across a range of formats, including “news articles, Twitter threads, and TV scripts mimicking Russian and Chinese state-run media outlets, health hoax peddlers, and well-known conspiracy theorists.”

So what’s the answer to that dilemma? It’s not immediately clear.

Parsons pointed out that Adobe, which maintains a family of generative AI products called Firefly, implements safeguards, like filters, aimed at preventing misuse. And the Content Authenticity Initiative, which Adobe co-founded in 2019 with the New York Times and Twitter, promotes an industry standard for provenance metadata.

But use of the CAI’s standard is completely voluntary, and just because Adobe is implementing safeguards doesn’t mean others will follow suit — or that those safeguards can’t or won’t be bypassed.

The panelists floated watermarking as another useful measure, albeit not a panacea.

A number of organizations are exploring watermarking techniques for generative media, including DeepMind, which recently proposed a standard, SynthID, to mark AI-generated images in a way that’s imperceptible to the human eye but can be easily spotted by a specialized detector. French startup Imatag, launched in 2020, offers a watermarking tool that it claims isn’t affected by resizing, cropping, editing or compressing images, similar to SynthID. Yet another firm, Steg.AI, employs an AI model to apply watermarks that survive resizing and other edits.

Indeed, pointing to some of the watermarking efforts and technologies on the market today, Brandt expressed optimism that “economic incentives” will encourage the companies building generative AI tools to be more thoughtful about how they deploy these tools — and the ways in which they design them to prevent them from being misused.

“With generative AI companies, their content needs to be trustworthy — otherwise, people won’t use it,” she said. “If it continues to hallucinate, if it continues to propagate misinformation, if it continues to not cite sources — that’s going to be less reliable than whatever generative AI company is making efforts to make sure that their content is reliable.”

I’m not so sure — especially as highly capable, safeguard-free open source generative AI models become widely available. As with all things, I suppose, time will tell.

More TechCrunch

The problem is not the media, but the message.

Apple’s ‘Crush’ ad is disgusting

Ever wonder why conversational AI like ChatGPT says “Sorry, I can’t do that” or some other polite refusal? OpenAI is offering a limited look at the reasoning behind its own…

OpenAI offers a peek behind the curtain of its AI’s secret instructions

The federal government agency responsible for granting patents and trademarks is alerting thousands of filers whose private addresses were exposed following a second data spill in as many years. The…

US Patent and Trademark Office confirms another leak of filers’ address data

As part of an investigation into people involved in the pro-independence movement in Catalonia, the Spanish police obtained information from the encrypted services Wire and Proton, which helped the authorities…

Encrypted services Apple, Proton and Wire helped Spanish police identify activist

Match Group, the company that owns several dating apps, including Tinder and Hinge, released its first-quarter earnings report on Tuesday, which shows that Tinder’s paying user base has decreased for…

Match looks to Hinge as Tinder fails

Private social networking is making a comeback. Gratitude Plus, a startup that aims to shift social media in a more positive direction, is expanding its wellness-focused, personal reflections journal to…

Gratitude Plus makes social networking positive, private and personal

With venture totals slipping year-over-year in key markets like the United States, and concern that venture firms themselves are struggling to raise more capital, founders might be worried. After all,…

Can AI help founders fundraise more quickly and easily?

Google has found a way to bring a variation of its clever “Circle to Search” gesture to iPhone users. The new interaction, launched in January, allows Android users to search…

Google brings a variation on ‘Circle to Search’ to iPhone users

A new sculpture going live on Wednesday in the Flatiron South Public Plaza in New York is not your typical artwork. It combines technology, sociology, anthropology and art to let…

Always-on video portal lets people in NYC and Dublin interact in real time

Apple’s iPad event had a lot to like. New iPads with new chips and new sizes, a new Apple Pencil, and even some software updates. If you are a big…

TechCrunch Minute: When did iPads get as expensive as MacBooks?

Autonomous, AI-based players are coming to a gaming experience near you, and a new startup, Altera, is joining the fray to build this new guard of AI agents. The company announced…

Bye-bye bots: Altera’s game-playing AI agents get backing from Eric Schmidt

Google DeepMind has taken the wraps off a new version of AlphaFold, their transformative machine learning model that predicts the shape and behavior of proteins. AlphaFold 3 is not only…

Google DeepMind debuts huge AlphaFold update and free proteomics-as-a-service web app

Uber plans to deliver more perks to Uber One members, like member-exclusive events, in a bid to gain more revenue through subscriptions.  “You will see more member-exclusives coming up where…

Uber promises member exclusives as Uber One passes $1B run-rate

We’ve all seen them. The inspector with a clipboard, walking around a building, ticking off the last time the fire extinguishers were checked, or if all the lights are working.…

Checkfirst raises $1.5M pre-seed to apply AI to remote inspections and audits

Close to a decade ago, brothers Aviv and Matteo Shapira co-founded a company, Replay, that created a video format for 360-degree replays — the sorts of replays that have become…

Controversial drone company Xtend leans into defense with new $40 million round

Usually, when something starts to rot, it gets pitched in the trash. But Joanne Rodriguez wants to turn the concept of rot on its head by growing fungus on trash…

Mycocycle uses mushrooms to upcycle old tires and construction waste

Monzo has raised another £150 million ($190 million), as the challenger bank looks to expand its presence internationally — particularly in the U.S. The new round comes just two months…

UK challenger bank Monzo nabs another $190M as US expansion beckons

iRobot has announced the successor to longtime CEO, Colin Angle. Gary Cohen, who previous held chief executive role at Timex and Qualitor Automotive, will be heading up the company, marking a major…

iRobot names former Timex head Gary Cohen as CEO

Reddit — now a publicly-traded company with more scrutiny on revenue growth — is putting a big focus on boosting its international audience, starting with francophones. In their first-ever earnings…

Reddit tests automatic, whole-site translation into French using LLM-based AI

Mushrooms continue to be a big area for alternative proteins. Canada-based Maia Farms recently raised $1.7 million to develop a blend of mushroom and plant-based protein using biomass fermentation. There’s…

Meati Foods bites into another $100M amid growth to 7,000 retail locations

Cleaning the outside of buildings is a dirty job, and it’s also dangerous. Lucid Bots came on the scene in 2018 with its Sherpa line of drones to clean windows…

Lucid Bots secures $9M for drones to clean more than your windows

High interest rates and financial pressures make it more important than ever for finance teams to have a better handle on their cash flow, and several startups are hoping to…

Israeli startup Panax raises a $10M Series A for its AI-driven cash flow management platform

The European Union has deepened the investigation of Elon Musk-owned social network, X, that it opened back in December under the bloc’s online governance and content moderation rulebook, the Digital Services Act…

EU grills Elon Musk’s X about content moderation and deepfake risks

For the founders of Atlan, a data governance startup, data has always been at the heart of what they do, even before they launched the company. In fact, co-founders Prukalpa…

Atlan scores $105M for its data control plane, as LLMs boost importance of data

It is estimated that about 2 billion people, especially those in lower and middle-income countries, lack access to quality and affordable essential medicines. The situation is exacerbated by low-quality or even killer…

Axmed raises $2M from Founderful to streamline drug supply chains in underserved markets

For decades, the Global Positioning System (GPS) has maintained a de facto monopoly on positioning, navigation and timing, because it’s cheap and already integrated into billions of devices around the…

Xona Space Systems closes $19M Series A to build out ultra-accurate GPS alternative

Bankruptcy lawyers representing customers impacted by the dramatic crash of cryptocurrency exchange FTX 17 months ago say that the vast majority of victims will receive their money back — plus interest. The…

FTX crypto fraud victims to get their money back — plus interest

On Wednesday, Google launched its digital wallet in India with local integrations, nearly two years after the app was relaunched as a digital wallet platform in the U.S. As TechCrunch exclusively reported last month,…

Google Wallet is now available in India

Bluesky has launched a new product roadmap for the coming months. The decentralized social network said on Tuesday that it is planning to introduce direct messages, support for videos, improved…

Bluesky to add DMs, video support and in-app custom feed curation

Samsung Medison, a medical device unit of Samsung Electronics that specializes in developing diagnostic imaging devices, said on Wednesday it plans to acquire Sonio, a Paris-based startup that makes AI-powered software…

Samsung Medison to acquire French AI ultrasound startup Sonio for $92.7M