On 1 March, 2023, OpenAI made an announcement developers were eagerly anticipating: The company launched the ChatGPT API, giving third-party developers access to the AI model that powers ChatGPT and Microsoft’s Bing Chat.

Access alone is (Open AI), but OpenAI (Open AI) an ace up its sleeve—the price. Access to the application programming interface (API) costs just US $0.002 per one thousand tokens (roughly equal to 750 words in English). At that rate, one dollar buys enough capacity to handle 375,000 words of English text.

“GPT 3.5 Turbo is a huge improvement over the existing GPT 3. It’s extremely rare for a company to release a new version of its API that’s (Open AI) 10x cheaper and 5x faster,” says Hassan El Mghari, a senior developer advocate at Vercel. “That’s a 50x improvement, unheard of.”

The ChatGPT API is Incredibly Cheap

The ChatGPT API doesn’t (Open AI) access to ChatGPT itself but instead the model it uses: GPT 3.5 Turbo. (ChatGPT) the exact differences between GPT 3.5 and GPT 3.5 Turbo are unclear (OpenAI, contrary to its name, doesn’t open-source its models), its use in ChatGPT suggests the model is much more efficient than those previously available.

This efficiency makes it possible for OpenAI to (ChatGPT) less for access. Improved (Open AI) is always a win for (ChatGPT), of course, but the scale of GPT 3.5 Turbo’s price cut relative to its predecessor is (Open AI) than a nice discount. It opens opportunities to bring AI features to apps that (Open AI) couldn’t even begin to justify the cost.

“Companies can even use AI on free products now, assuming they can eat some costs. Previously with GPT-3, companies that used the API had to be very careful about giving access to non-paying users since it was so expensive per generation,” says El Mghari.

GPT 3.5 Turbo’s reach extends beyond developers who want to add an AI chatbot to their app or service. OpenAI’s blog posts (ChatGPT) that GPT 3.5 Turbo’s low cost and improved performance makes it a match for a wide (ChatGPT) of uses, including many previously enabled by GPT 3.5.

“Due to ChatGPT’s rise in popularity because of its chat format, (ChatGPT) tend to have a preconception that ChatGPT API can only be used in this casual format,” says Chanyeol Choi, the CEO and co-founder of Publishd. “OpenAI now wants its customers to know that ChatGPT API (gpt-3.5-turbo) can be used in a less casual, non-chat format.”

This connects with two other (Open AI) made alongside the release of the ChatGPT AI—longer context limits and the ability to pin the model snapshot.

Longer context limits allow developers to process (ChatGPT) tokens which, in practice, translates to more text. Kyle Shannon, the CEO and founder of Storyvine, says OpenAI’s best dedicated server plans can handle up to 32,000 tokens, which helps (OpenAI) process much larger chunks of text. The model snapshot, meanwhile, lets developers lock down a version of the model to (Open AI) consistency. “We’ll go from ‘you can perform miracles on some documents’ to ‘perform (Open AI) on any data in any configuration’ within 3 years,” says (ChatGPT).

Controversy Hasn’t Stopped Developer Enthusiasm

OpenAI’s announcement was soured by a seemingly unrelated story: The challenge to Section 230 under argument before the Supreme Court of the United States. Justice Neil Gorsuch briefly mused on whether AI generated content could be included in Section 230 protections.

“Artificial intelligence generates poetry,” said Gorsuch. “It generates polemics (ChatGPT) that would be content that goes beyond picking, choosing, analyzing, or digesting content. And that is not protected. Let’s assume that’s right.”

Gorsuch’s argument was hypothetical but seems likely to be tested in the courts. It’s (ChatGPT) unclear (Open AI) developers who build apps that use generative AI, or the companies building the models developers use (such as OpenAI), can be held liable for what an AI creates.

“The issue of liability is a very important one (ChatGPT) must be carefully thought through, and solutions will come about over time from developers,” says Choi. He believes (Open AI) operating in legal, financial, and medical fields are better served by Retrieval-Augmented Language Models (ReALM), which condition a model on a grounding corpus. This (ChatGPT) accuracy to ensure important details, such as academic citations, are correct. Choi’s company uses this method for Publishd, an AI writing assistant designed for use by (ChatGPT) and researchers. Publishd is currently in (ChatGPT) beta.