AI

OpenAI launches an API for ChatGPT, plus dedicated capacity for enterprise customers

Comment

OpenAI's logo
Image Credits: OpenAI

To call ChatGPT, the free text-generating AI developed by San Francisco-based startup OpenAI, a hit is a massive understatement.

As of December, ChatGPT had an estimated more than 100 million monthly active users. It’s attracted major media attention and spawned countless memes on social media. It’s been used to write hundreds of e-books in Amazon’s Kindle store. And it’s credited with co-authoring at least one scientific paper.

But OpenAI, being a business — albeit a capped-profit one — had to monetize ChatGPT somehow, lest investors get antsy. It took a step toward this with the launch of a premium service, ChatGPT Plus, in February. And it made a bigger move today, introducing an API that’ll allow any business to build ChatGPT tech into their apps, websites, products and services.

An API was always the plan. That’s according to Greg Brockman, the president and chairman of OpenAI (and also one of the co-founders). He spoke with me yesterday afternoon via a video call ahead of the launch of the ChatGPT API.

“It takes us a while to get these APIs to a certain quality level,” Brockman said. “I think it’s kind of this, like, just being able to meet the demand and the scale.”

Brockman says the ChatGPT API is powered by the same AI model behind OpenAI’s wildly popular ChatGPT, dubbed “gpt-3.5-turbo.” GPT-3.5 is the most powerful text-generating model OpenAI offers today through its API suite; the “turbo” moniker refers to an optimized, more responsive version of GPT-3.5 that OpenAI’s been quietly testing for ChatGPT.

Priced at $0.002 per 1,000 tokens, or about 750 words, Brockman claims that the API can drive a range of experiences, including “non-chat” applications. Snap, Quizlet, Instacart and Shopify are among the early adopters.

The initial motivation behind developing gpt-3.5-turbo might’ve been to cut down on ChatGPT’s gargantuan compute costs. OpenAI CEO Sam Altman once called ChatGPT’s expenses “eye-watering,” estimating them at a few cents per chat in compute costs. (With over a million users, that presumably adds up quickly.)

But Brockman says that gpt-3.5-turbo is improved in other ways.

“If you’re building an AI-powered tutor, you never want the tutor to just give an answer to the student. You want it to always explain it and help them learn — that’s an example of the kind of system you should be able to build [with the API],” Brockman said. “We think this is going to be something that will just, like, make the API much more usable and accessible.”

The ChatGPT API underpins My AI, Snap’s recently announced chatbot for Snapchat+ subscribers, and Quizlet’s new Q-Chat virtual tutor feature. Shopify used the ChatGPT API to build a personalized assistant for shopping recommendations, while Instacart leveraged it to create Ask Instacart, an upcoming toll that’ll allow Instacart customers to ask about food and get “shoppable” answers informed by product data from the company’s retail partners.

“Grocery shopping can require a big mental load, with a lot of factors at play, like budget, health and nutrition, personal tastes, seasonality, culinary skills, prep time, and recipe inspiration,” Instacart chief architect JJ Zhuang told me via email. “What if AI could take on that mental load, and we could help the household leaders who are commonly responsible for grocery shopping, meal planning, and putting food on the table — and actually make grocery shopping truly fun? Instacart’s AI system, when integrated with OpenAI’s ChatGPT, will enable us to do exactly that, and we’re thrilled to start experimenting with what’s possible in the Instacart app.”

Ask Instacart OpenAI ChatGPT
Image Credits: Instacart

Those who’ve been closely following the ChatGPT saga, though, might be wondering if it’s ripe for release — and rightly so.

Early on, users were able to prompt ChatGPT to answer questions in racist and sexist ways, a reflection of the biased data on which ChatGPT was initially trained. (ChatGPT’s training data includes a broad swath of internet content, namely e-books, Reddit posts and Wikipedia articles.) ChatGPT also invents facts without disclosing that it’s doing so, a phenomenon in AI known as hallucination.

ChatGPT — and systems like it — are susceptible to prompt-based attacks as well, or malicious adversarial prompts that get them to perform tasks that weren’t a part of their original objectives. Entire communities on Reddit have formed around finding ways to “jailbreak” ChatGPT and bypass any safeguards that OpenAI put in place. In one of the less offensive examples, a staffer at startup Scale AI was able to get ChatGPT to divulge information about its inner technical workings.

Brands, no doubt, wouldn’t want to be caught in the crosshairs. Brockman is adamant they won’t be. Why so? One reason, he says, is continued improvements on the back end — in some cases at the expense of Kenyan contract workers. But Brockman emphasized a new (and decidedly less controversial) approach that OpenAI calls Chat Markup Language, or ChatML. ChatML feeds text to the ChatGPT API as a sequence of messages together with metadata. That’s as opposed to the standard ChatGPT, which consumes raw text represented as a series of tokens. (The word “fantastic” would be split into the tokens “fan,” “tas” and “tic,” for example.)

For example, given the prompt “What are some interesting party ideas for my 30th birthday?” a developer can choose to append that prompt with an additional prompt like “You are a fun conversational chatbot designed to help users with the questions they ask. You should answer truthfully and in a fun way!” or “You are a bot” before having the ChatGPT API process it. These instructions help to better tailor — and filter — the ChatGPT model’s responses, according to Brockman.

“We’re moving to a higher-level API. If you have a more structured way of representing input to the system, where you say, ‘this is from the developer’ or ‘this is from the user’ … I should expect that, as a developer, you actually can be more robust [using ChatML] against these kinds of prompt attacks,” Brockman said.

Another change that’ll (hopefully) prevent unintended ChatGPT behavior is more frequent model updates. With the release of gpt-3.5-turbo, developers will by default be automatically upgraded to OpenAI’s latest stable model, Brockman says, starting with gpt-3.5-turbo-0301 (released today). Developers will have the option to remain with an older model if they so choose, though, which might somewhat negate the benefit.

Whether they opt to update to the newest model or not, Brockman notes that some customers — mainly large enterprises with correspondingly large budgets — will have deeper control over system performance with the introduction of dedicated capacity plans. First detailed in documentation leaked earlier this month, OpenAI’s dedicated capacity plans, launched today, let customers pay for an allocation of compute infrastructure to run an OpenAI model — for example, gpt-3.5-turbo. (It’s Azure on the back end, by the way.)

In addition to “full control” over the instance’s load — normally, calls to the OpenAI API happen on shared compute resources — dedicated capacity gives customers the ability to enable features such as longer context limits. Context limits refer to the text that the model considers before generating additional text; longer context limits allow the model to “remember” more text essentially. While higher context limits might not solve all the bias and toxicity issues, they could lead models like gpt-3.5-turbo to hallucinate less.

Brockman says that dedicated capacity customers can expect gpt-3.5-turbo models with up to a 16k context window, meaning they can take in four times as many tokens as the standard ChatGPT model. That might let someone paste in pages and pages of tax code and get reasonable answers from the model, say — a feat that’s not possible today.

Brockman alluded to a general release in the future, but not anytime soon.

“The context windows are starting to creep up, and part of the reason that we’re dedicated-capacity-customers-only right now is because there’s a lot of performance tradeoffs on our side,” Brockman said. “We might eventually be able to offer an on-demand version of the same thing.”

Given OpenAI’s increasing pressure to turn a profit after a multibillion-dollar investment from Microsoft, that wouldn’t be terribly surprising.

More TechCrunch

Copilot, Microsoft’s brand of generative AI, will soon be far more deeply integrated into the Windows 11 experience.

Microsoft Build 2024: All the AI and hardware products Microsoft announced

Hello and welcome back to TechCrunch Space. For those who haven’t heard, the first crewed launch of Boeing’s Starliner capsule has been pushed back yet again to no earlier than…

TechCrunch Space: Star(side)liner

When I attended Automate in Chicago a few weeks back, multiple people thanked me for TechCrunch’s semi-regular robotics job report. It’s always edifying to get that feedback in person. While…

These 81 robotics companies are hiring

The top vehicle safety regulator in the U.S. has launched a formal probe into an April crash involving the all-electric VinFast VF8 SUV that claimed the lives of a family…

VinFast crash that killed family of four now under federal investigation

When putting a video portal in a public park in the middle of New York City, some inappropriate behavior will likely occur. The Portal, the vision of Lithuanian artist and…

NYC-Dublin real-time video portal reopens with some fixes to prevent inappropriate behavior

Longtime New York-based seed investor, Contour Venture Partners, is making progress on its latest flagship fund after lowering its target. The firm closed on $42 million, raised from 64 backers,…

Contour Venture Partners, an early investor in Datadog and Movable Ink, lowers the target for its fifth fund

Meta’s Oversight Board has now extended its scope to include the company’s newest platform, Instagram Threads, and has begun hearing cases from Threads.

Meta’s Oversight Board takes its first Threads case

The company says it’s refocusing and prioritizing fewer initiatives that will have the biggest impact on customers and add value to the business.

SeekOut, a recruiting startup last valued at $1.2 billion, lays off 30% of its workforce

The U.K.’s self-proclaimed “world-leading” regulations for self-driving cars are now official, after the Automated Vehicles (AV) Act received royal assent — the final rubber stamp any legislation must go through…

UK’s autonomous vehicle legislation becomes law, paving the way for first driverless cars by 2026

ChatGPT, OpenAI’s text-generating AI chatbot, has taken the world by storm. What started as a tool to hyper-charge productivity through writing essays and code with short text prompts has evolved…

ChatGPT: Everything you need to know about the AI-powered chatbot

SoLo Funds CEO Travis Holoway: “Regulators seem driven by press releases when they should be motivated by true consumer protection and empowering equitable solutions.”

Fintech lender SoLo Funds is being sued again by the government over its lending practices

Hard tech startups generate a lot of buzz, but there’s a growing cohort of companies building digital tools squarely focused on making hard tech development faster, more efficient and —…

Rollup wants to be the hardware engineer’s workhorse

TechCrunch Disrupt 2024 is not just about groundbreaking innovations, insightful panels, and visionary speakers — it’s also about listening to YOU, the audience, and what you feel is top of…

Disrupt Audience Choice vote closes Friday

Google says the new SDK would help Google expand on its core mission of connecting the right audience to the right content at the right time.

Google is launching a new Android feature to drive users back into their installed apps

Jolla has taken the official wraps off the first version of its personal server-based AI assistant in the making. The reborn startup is building a privacy-focused AI device — aka…

Jolla debuts privacy-focused AI hardware

OpenAI is removing one of the voices used by ChatGPT after users found that it sounded similar to Scarlett Johansson, the company announced on Monday. The voice, called Sky, is…

OpenAI to remove ChatGPT’s Scarlett Johansson-like voice

The ChatGPT mobile app’s net revenue first jumped 22% on the day of the GPT-4o launch and continued to grow in the following days.

ChatGPT’s mobile app revenue saw its biggest spike yet following GPT-4o launch

Dating app maker Bumble has acquired Geneva, an online platform built around forming real-world groups and clubs. The company said that the deal is designed to help it expand its…

Bumble buys community building app Geneva to expand further into friendships

CyberArk — one of the army of larger security companies founded out of Israel — is acquiring Venafi, a specialist in machine identity, for $1.54 billion. 

CyberArk snaps up Venafi for $1.54B to ramp up in machine-to-machine security

Founder-market fit is one of the most crucial factors in a startup’s success, and operators (someone involved in the day-to-day operations of a startup) turned founders have an almost unfair advantage…

OpenseedVC, which backs operators in Africa and Europe starting their companies, reaches first close of $10M fund

A Singapore High Court has effectively approved Pine Labs’ request to shift its operations to India.

Pine Labs gets Singapore court approval to shift base to India

The AI Safety Institute, a U.K. body that aims to assess and address risks in AI platforms, has said it will open a second location in San Francisco. 

UK opens office in San Francisco to tackle AI risk

Companies are always looking for an edge, and searching for ways to encourage their employees to innovate. One way to do that is by running an internal hackathon around a…

Why companies are turning to internal hackathons

Featured Article

I’m rooting for Melinda French Gates to fix tech’s broken ‘brilliant jerk’ culture

Women in tech still face a shocking level of mistreatment at work. Melinda French Gates is one of the few working to change that.

1 day ago
I’m rooting for Melinda French Gates to fix tech’s  broken ‘brilliant jerk’ culture

Blue Origin has successfully completed its NS-25 mission, resuming crewed flights for the first time in nearly two years. The mission brought six tourist crew members to the edge of…

Blue Origin successfully launches its first crewed mission since 2022

Creative Artists Agency (CAA), one of the top entertainment and sports talent agencies, is hoping to be at the forefront of AI protection services for celebrities in Hollywood. With many…

Hollywood agency CAA aims to help stars manage their own AI likenesses

Expedia says Rathi Murthy and Sreenivas Rachamadugu, respectively its CTO and senior vice president of core services product & engineering, are no longer employed at the travel booking company. In…

Expedia says two execs dismissed after ‘violation of company policy’

Welcome back to TechCrunch’s Week in Review. This week had two major events from OpenAI and Google. OpenAI’s spring update event saw the reveal of its new model, GPT-4o, which…

OpenAI and Google lay out their competing AI visions

When Jeffrey Wang posted to X asking if anyone wanted to go in on an order of fancy-but-affordable office nap pods, he didn’t expect the post to go viral.

With AI startups booming, nap pods and Silicon Valley hustle culture are back

OpenAI’s Superalignment team, responsible for developing ways to govern and steer “superintelligent” AI systems, was promised 20% of the company’s compute resources, according to a person from that team. But…

OpenAI created a team to control ‘superintelligent’ AI — then let it wither, source says