Anthropic: Our Claude 3.5 Model Beats OpenAI’s GPT-4o

Anthropic is releasing a new AI model that promises to outperform OpenAI’s ChatGPT and Google’s Gemini. 

The advancement is arriving as an update to Claude 3 Sonnet, one of three AI models that the company debuted back in March. On Thursday, Anthropic announced Claude 3.5 Sonnet, describing it as “our most intelligent model yet.”

“Sonnet now outperforms competitor models on key evaluations, at twice the speed of Claude 3 Opus (its older flagship model) and one-fifth the cost,” wrote the company, which was founded by former OpenAI employees.

As evidence, Anthropic benchmarked Claude 3.5 against OpenAI’s newest AI model, GPT-4o, which powers ChatGPT. The results show Anthropic’s AI model achieving slightly better results in four of the six benchmarks, which focused on reasoning, coding and math skills. 

(Anthropic)

In addition, Claude 3.5 also bested Google’s Gemini 1.5 across all the tested benchmarks. That said, AI benchmarks have often been criticized as an unreliable indicator due to a lack of standardization and independent oversight. As a result, companies, including Anthropic, can pick and choose which benchmarks to use, and which benchmarks to throw out. 

So the claims about Claude 3.5 should be taken with a grain of salt. Nevertheless, Anthropic says the new model has also made advancements in its writing abilities, along with “grasping nuance, humor, and complex instructions.” This can include translating computer code, “making it particularly effective for updating legacy applications and migrating codebases,” the company said. 

The other major enhancement involves Claude’s ability to process data visually. The “Claude 3.5 Sonnet for vision” feature can understand information from charts and graphs, and also accurately transcribe text from imperfect images. 

To make Claude more useful, Anthropic also added a new feature called “Artifacts,” which will display a second side-by-side window alongside the conversation box. Through the second window, you’ll be able to see and iterate on the content that Claude produces, whether it be computer code, images or text documents.

“This creates a dynamic workspace where they (the user) can see, edit, and build upon Claude’s creations in real-time, seamlessly integrating AI-generated content into their projects and workflows,” the company said. “This preview feature marks Claude’s evolution from a conversational AI to a collaborative work environment.”

Anthropic is offering free access to Claude 3.5 through the website claude.ai and over the Claude iOS app. Existing subscribers to the Claude Pro and Team plan will also receive access to the new model, but with “higher rate limits,” enabling them to send more queries to the chatbot before encountering restrictions. 

The company also plans on upgrading its two other AI models, Claude 3 Haiku and Claude 3 Opus. with the new 3.5 tech later this year.

(Anthropic)