Women in AI: Sarah Kreps, professor of government at Cornell


Women in AI Sarah Kreps
Image Credits: TechCrunch

To give AI-focused women academics and others their well-deserved — and overdue — time in the spotlight, TechCrunch is launching a series of interviews focusing on remarkable women who’ve contributed to the AI revolution. We’ll publish several pieces throughout the year as the AI boom continues, highlighting key work that often goes unrecognized. Read more profiles here.

Sarah Kreps is a political scientist, U.S. Air Force veteran and analyst who focuses on U.S. foreign and defense policy. She’s a professor of government at Cornell University, adjunct professor of law at Cornell Law School and an adjunct scholar at West Point’s Modern War Institute.

Kreps’ recent research explores both the potential and risks of AI tech such as OpenAI’s GPT-4, specifically in the political sphere. In an opinion column for The Guardian last year, she wrote that, as more money pours into AI, the AI arms race not just across companies but countries will intensify — while the AI policy challenge will become harder.


Briefly, how did you get your start in AI? What attracted you to the field?

I had my start in the area of emerging technologies with national security implications. I had been an Air Force officer at the time the Predator drone was deployed, and had been involved in advanced radar and satellite systems. I had spent four years working in this space, so it was natural that, as a PhD, I would be interested in studying the national security implications of emerging technologies. I first wrote about drones, and the debate in drones was moving toward questions of autonomy, which of course implicates artificial intelligence.

In 2018, I was at an artificial intelligence workshop at a D.C. think tank and OpenAI gave a presentation about this new GPT-2 capability they had developed. We had just gone through the 2016 election and foreign election interference, which had been relatively easy to spot because of little things like grammatical errors of non-native English speakers — the kind of errors that were not surprising given that the interference had come from the Russian-backed Internet Research Agency. As OpenAI gave this presentation, I was immediately preoccupied with the possibility of generating credible disinformation at scale and then, through microtargeting, manipulating the psychology of American voters in far more effective ways than had been possible when these individuals were trying to write content by hand, where scale was always going to be a problem.

I reached out to OpenAI and became one of the early academic collaborators in their staged release strategy. My particular research was aimed at investigating the possible misuse case — whether GPT-2 and later GPT-3 were credible as political content generators. In a series of experiments, I evaluated whether the public would see this content as credible but then also conducted a large field experiment where I generated “constituency letters” that I randomized with actual constituency letters to see whether legislators would respond at the same rates to know whether they could be fooled — whether malicious actors could shape the legislative agenda with a large-scale letter writing campaign.

These questions struck at the heart of what it means to be a sovereign democracy and I concluded unequivocally that these new technologies did represent new threats to our democracy.

What work are you most proud of (in the AI field)?

I’m very proud of the field experiment I conducted. No one had done anything remotely similar and we were the first to show the disruptive potential in a legislative agenda context.

But I’m also proud of tools that unfortunately I never brought to market. I worked with several computer science students at Cornell to develop an application that would process legislative inbound emails and help them respond to constituents in meaningful ways. We were working on this before ChatGPT and using AI to digest the large volume of emails and provide an AI assist for time-pressed staffers communicating with people in their district or state. I thought these tools were important because of constituents’ disaffection from politics but also the increasing demands on the time of legislators. Developing AI in these publicly interested ways seemed like a valuable contribution and interesting interdisciplinary work for political scientists and computer scientists. We conducted a number of experiments to assess the behavioral questions of how people would feel about an AI assist responding to them and concluded that maybe society was not ready for something like this. But then a few months after we pulled the plug, ChatGPT came on the scene and AI is so ubiquitous that I almost wonder how we ever worried about whether this was ethically dubious or legitimate. But I still feel like it’s right that we asked the hard ethical questions about the legitimate use case.

How do you navigate the challenges of the male-dominated tech industry, and, by extension, the male-dominated AI industry?

As a researcher, I have not felt those challenges terribly acutely. I was just out in the Bay Area and it was all dudes literally giving their elevator pitches in the hotel elevator, a cliché that I could see being intimidating. I would recommend that they find mentors (male and female), develop skills and let those skills speak for themselves, take on challenges and stay resilient.

What advice would you give to women seeking to enter the AI field?

I think there are a lot of opportunities for women — they need to develop skills and have confidence and they’ll thrive.

What are some of the most pressing issues facing AI as it evolves?

I worry that the AI community has developed so many research initiatives that focus on things like “superalignment” that obscure the deeper — or actually, the right — questions about whose values or what values we are trying to align AI with. Google Gemini’s problematic rollout showed the caricature that can arise from aligning with a narrow set of developers’ values in ways that actually led to (almost) laughable historical inaccuracies in their outputs. I think those developers’ values were good faith, but revealed the fact that these large language models are being programmed with a particular set of values that will be shaping how people think about politics, social relationships and a variety of sensitive topics. Those issues aren’t of the existential risk variety but do create the fabric of society and confer considerable power into the big firms (e.g. OpenAI, Google, Meta and so on) that are responsible for those models.

What are some issues AI users should be aware of?

As AI becomes ubiquitous, I think we’ve entered a “trust but verify” world. It’s nihilistic not to believe anything but there’s a lot of AI-generated content and users really need to be circumspect in terms of what they instinctively trust. It’s good to look for alternative sources to verify the authenticity before just assuming that everything is accurate. But I think we already learned that with social media and misinformation.

What is the best way to responsibly build AI?

I recently wrote a piece for the Bulletin of the Atomic Scientists, which started out covering nuclear weapons but has adapted to address disruptive technologies like AI. I had been thinking about how scientists could be better public stewards and wanted to connect some of the historical cases I had been looking at for a book project. I not only outline a set of steps I would endorse for responsible development but also speak to why some of the questions that AI developers are asking are wrong, incomplete or misguided.

More TechCrunch

The spam reached Bluesky by first crossing over two other decentralized networks: Mastodon and Nostr.

The ‘vote Trump’ spam that hit Bluesky in May came from decentralized rival Nostr

Welcome to TechCrunch Fintech! This week, we’re looking at the continued fallout from Synapse’s bankruptcy, how Layer wants to disrupt SMB accounting, and much more! To get a roundup of…

There’s a real appetite for a fintech alternative to QuickBooks

The company is hoping to produce electricity at $13 per megawatt hour, which would be more than 50% cheaper than traditional onshore wind.

Bill Gates-backed wind startup AirLoom is raising $12M, filings reveal

Generative AI makes stuff up. It can be biased. Sometimes it spits out toxic text. So can it be “safe”? Rick Caccia, the CEO of WitnessAI, believes it can. “Securing…

WitnessAI is building guardrails for generative AI models

It’s not often that you hear about a seed round above $10 million. H, a startup based in Paris and previously known as Holistic AI, has announced a $220 million…

French AI startup H raises $220M seed round

Hey there, Series A to B startups with $35 million or less in funding — we’ve got an exciting opportunity that’s tailor-made for your growth journey! If you’re looking to…

Boost your startup’s growth with a ScaleUp package at TC Disrupt 2024

TikTok is pulling out all the stops to prevent its impending ban in the United States. Aside from initiating legal action against the U.S. government, that means shaping up its…

As a US ban looms, TikTok announces a $1M program for socially driven creators

Microsoft wants to put its Copilot everywhere. It’s only a matter of time before Microsoft renames its annual Build developer conference to Microsoft Copilot. Hopefully, some of those upcoming events…

Microsoft’s Power Automate no-code platform adds AI flows

Build is Microsoft’s largest developer conference and of course, it’s all about AI this year. So it’s no surprise that GitHub’s Copilot, GitHub’s “AI pair programming tool,” is taking center…

GitHub Copilot gets extensions

Microsoft wants to make its brand of generative AI more useful for teams — specifically teams across corporations and large enterprise organizations. This morning at its annual Build dev conference,…

Microsoft intros a Copilot for teams

Microsoft’s big focus at this year’s Build conference is generative AI. And to that end, the tech giant announced a series of updates to its platforms for building generative AI-powered…

Microsoft upgrades its AI app-building platforms

The U.K.’s data protection watchdog has closed an almost year-long investigation of Snap’s AI chatbot, My AI — saying it’s satisfied the social media firm has addressed concerns about risks…

UK data protection watchdog ends privacy probe of Snap’s GenAI chatbot, but warns industry

U.S. cell carrier Patriot Mobile experienced a data breach that included subscribers’ personal information, including full names, email addresses, home ZIP codes and account PINs, TechCrunch has learned. Patriot Mobile,…

Conservative cell carrier Patriot Mobile hit by data breach

It’s been three years since Spotify acquired live audio startup Betty Labs, and yet the music streaming service isn’t leveraging the technology to its fullest potential — at least not…

Spotify’s ‘Listening Party’ feature falls short of expectations

Alchemist Accelerator has a new pile of AI-forward companies demoing their wares today, if you care to watch, and the program itself is making some international moves into Tokyo and…

Alchemist’s latest batch puts AI to work as accelerator expands to Tokyo, Doha

“Late Pledge” allows campaign creators to continue collecting money even after the campaign has closed.

Kickstarter now lets you pledge after a campaign closes

Stack AI’s co-founders, Antoni Rosinol and Bernardo Aceituno, were PhD students at MIT wrapping up their degrees in 2022 just as large language models were becoming more mainstream. ChatGPT would…

Stack AI wants to make it easier to build AI-fueled workflows

Pinecone, the vector database startup founded by Edo Liberty, the former head of Amazon’s AI Labs, has long been at the forefront of helping businesses augment large language models (LLMs)…

Pinecone launches its serverless vector database out of preview

Young geothermal energy wells can be like budding prodigies, each brimming with potential to outshine their peers. But like people, most decline with age. In California, for example, the amount…

Special mud helps XGS Energy get more power out of geothermal wells

Featured Article

Sonos finally made some headphones

The market play is clear from the outset: The $449 headphones are firmly targeted at an audience that would otherwise be purchasing the Bose QC Ultra or Apple AirPods Max.

6 hours ago
Sonos finally made some headphones

Adobe says the feature is up to the task, regardless of how complex of a background the object is set against.

Adobe brings Firefly AI-powered Generative Remove to Lightroom

All cars suffer when the mercury drops, but electric vehicles suffer more than most as heaters draw more power and batteries charge more slowly as the liquid electrolyte inside thickens.…

Porsche Ventures invests in battery startup South 8 to boost cold-weather EV performance

Scale AI has raised a $1 billion Series F round from a slew of big-name institutional and corporate investors including Amazon and Meta.

Data-labeling startup Scale AI raises $1B as valuation doubles to $13.8B

The new coalition, Tech Against Scams, will work together to find ways to fight back against the tools used by scammers and to better educate the public against financial scams.

Meta, Match, Coinbase and others team up to fight online fraud and crypto scams

It’s a wrap: European Union lawmakers have given the final approval to set up the bloc’s flagship, risk-based regulations for artificial intelligence.

EU Council gives final nod to set up risk-based regulations for AI

London-based fintech Vitesse has closed a $93 million Series C round of funding led by investment giant KKR.

Vitesse, a payments and treasury management platform for insurers, raises $93M to fuel US expansion

Zen Educate, an online marketplace that connects schools with teachers, has raised $37 million in a Series B round of funding. The raise comes amid a growing teacher shortage crisis…

Zen Educate raises $37M and acquires Aquinas Education as it tries to address the teacher shortage

“When I heard the released demo, I was shocked, angered and in disbelief that Mr. Altman would pursue a voice that sounded so eerily similar to mine.”

Scarlett Johansson says that OpenAI approached her to use her voice

A new self-driving truck — manufactured by Volvo and loaded with autonomous vehicle tech developed by Aurora Innovation — could be on public highways as early as this summer.  The…

Aurora and Volvo unveil self-driving truck designed for a driverless future

The European venture capital firm raised its fourth fund as fund as climate tech “comes of age.”

ETF Partners raises €285M for climate startups that will be effective quickly — not 20 years down the road