GPT-4.5 release a bit of a dud

[Free]

“It’s a lemon”—OpenAI’s largest AI model ever arrives to mixed reviews

GPT-4.5 offers marginal gains in capability and poor coding performance despite 30x the cost.

arstechnica.com

The verdict is in: OpenAI's newest and most capable traditional AI model, GPT-4.5, is big, expensive, and slow, providing marginally better performance than GPT-4o at 30x the cost for input and 15x the cost for output. The new model seems to prove that longstanding rumors of diminishing returns in training unsupervised-learning LLMs were correct and that the so-called "scaling laws" cited by many for years have possibly met their natural end.

An AI expert who requested anonymity told Ars Technica, "GPT-4.5 is a lemon!" when comparing its reported performance to its dramatically increased price, while frequent OpenAI critic Gary Marcus called the release a "nothing burger" in a blog post (though to be fair, Marcus also seems to think most of what OpenAI does is overrated).

Former OpenAI researcher Andrej Karpathy wrote on X that GPT-4.5 is better than GPT-4o but in ways that are subtle and difficult to express. "Everything is a little bit better and it's awesome," he wrote, "but also not exactly in ways that are trivial to point to."

OpenAI is well aware of these limitations, and it took steps to soften the potential letdown by framing the launch as a relatively low-key "Research Preview" for ChatGPT Pro users and spelling out the model's limitations in a GPT-4.5 release post published Thursday.

Fireship makes it short and sweet:

 

[Free]

Meanwhile, Altman explains that the release of GPT-4.5 has been "staggered" because OpenAI is out of gas GPUs.

OpenAI CEO Sam Altman says the company is 'out of GPUs' | TechCrunch

OpenAI CEO Sam Altman said that the company was forced to stagger the rollout of its newest model, GPT-4.5, because OpenAI is 'out of GPUs.'

techcrunch.com

 

[Free]

OpenAI Admits That Its New Model Still Hallucinates More Than a Third of the Time

If a partner made stuff up a third of the time, it would be a problem — but apparently, it's perfectly fine for OpenAI's new model.

futurism.com

Using SimpleQA, the company's in-house factuality benchmarking tool, OpenAI admitted in its release announcement that its new large language model (LLM) GPT-4.5 hallucinates — which is AI parlance for confidently spewing fabrications and presenting them as fact — 37 percent of the time.

Yes, you read that right: in tests, the latest AI model from a company that's worth hundreds of billions of dollars is telling lies for more than one out of every three answers it gives.

We should have elected it President!

 

[Noodles]

OpenAI Admits That Its New Model Still Hallucinates More Than a Third of the Time

If a partner made stuff up a third of the time, it would be a problem — but apparently, it's perfectly fine for OpenAI's new model.

futurism.com

We should have elected it President!

Literally could not be worse.

 

[Argent Stonecutter]

They're all lemons. They hallucinate 100% of the time because hallucination is how they work. If the hallucinations happen to line up with reality that's just chance.

 

[Casey Pelous]

Former OpenAI researcher and compulsive liar Andrej Karpathy Tommy Flanagan wrote on X that GPT-4.5 is better than GPT-4o but in ways that are subtle and difficult to express. "Everything is a little bit better and it's awesome," he wrote, "but also not exactly in ways that are trivial to point to. Yeah .... that's the ticket!" *giant bong hit* "I'd explain it but, you know, it's like, very, very technical. And, you know what else, I rode a surfboard on a tidal wave from Hawaii all the way to Silicon Valley, never hanging less than five the whole way. Yeah ....I was great .... "

 

[Free]

They're all lemons.

When life hands you lemons, make LSD (and put it in some lemonade).

 

[Noodles]

When life hands you lemons, make LSD (and put it in some lemonade).

Is that why it makes pictures of people with ten fingers? The LSD?

 

[Essence Lumin]