While Apple and Google turn their voice assistants into chatbots, OpenAI is turning its chatbot into a voice assistant.
The San Francisco artificial intelligence start-up on Monday unveiled a new version of its ChatGPT chatbot that can receive and respond to voice commands, images and videos.
The company said the new app, based on an artificial intelligence system called GPT-4o, handles audio, images and video significantly faster than previous versions of the technology. The app will be available starting Monday, free of charge, for both smartphones and desktop computers.
“We are looking at the future of interaction between us and machines,” said Mira Murati, the company's chief technology officer.
The new app is part of a larger effort to combine conversational chatbots like ChatGPT with voice assistants like Google Assistant and Apple's Siri. While Google combines its Gemini chatbot with Google Assistant, Apple is preparing a new, more conversational version of Siri.
OpenAI said it will gradually share the technology with users “over the coming weeks.” This is the first time offering ChatGPT as a desktop application.
The company previously offered similar technologies within various free and paid products. It has now brought them together into a single system available for all its products.
During an event streamed over the Internet, Murati and her colleagues demonstrated the new app as it responded to conversational voice commands, used a live video feed to analyze math problems written on a sheet of paper, and read stories aloud jokes that he had written on the fly
The new app cannot generate videos. But it can generate still images that represent frames of a video.
With the debut of ChatGPT in late 2022, OpenAI has demonstrated that machines can handle requests more like people. In response to textual conversation prompts, he could answer questions, write term papers, and even generate computer code.
ChatGPT was not guided by a set of rules. He learned his skills by analyzing huge amounts of text collected from the Internet, including Wikipedia articles, books and chat logs. Experts have hailed the technology as a possible alternative to search engines like Google and voice assistants like Siri.
Newer versions of the technology have also learned from sounds, images and videos. Researchers call this “multimodal AI.” Essentially, companies like OpenAI have started combining chatbots with AI image, audio, and video generators.
(The New York Times sued OpenAI and its partner, Microsoft, in December, alleging copyright infringement of news content related to artificial intelligence systems.)
As companies combine chatbots with voice assistants, many obstacles remain. Because chatbots learn their skills from Internet data, they are prone to errors. Sometimes they make up information entirely, a phenomenon that AI researchers call “hallucination.” These flaws are migrating to voice assistants.
While chatbots can generate compelling language, they are less adept at taking actions like scheduling a meeting or booking an airline flight. But companies like OpenAI are working to turn them into “AI agents” that can reliably handle such tasks.
OpenAI previously offered a version of ChatGPT that could accept voice commands and respond with voice. But it was a patchwork of three different AI technologies: one that converted speech to text, one that generated a text response, and one that converted this text into a synthetic voice.
The new app is based on a unique AI technology – GPT-4o – capable of accepting and generating text, sounds and images. This means the technology is more efficient and the company can afford to offer it to users for free, Murati said.
“Before there was all this latency that was the result of three models working together,” Murati said in an interview with the Times. “You want to have the experience that we're having, where we can have this very natural dialogue.”