AI's Diminishing Returns: Why Innovation Needs a Reboot

Good morning. Happy WWDC week. Apple’s Developer Conference (WWDC) is in full swing, and the world is watching the silky-smooth keynote and Craig Federighi’s beautiful head of hair yet again.

Just like every year, rumors turn into reality, and reality turns into disappointment for half the crowd.

And because every tech Youtuber will have already covered all details at length, I will spare you these today.

This week, I want to examine a common phenomenon in innovation: Leading AI models are reaching the stage of diminishing returns. We’ll look at the reasons.

Lastly, what’s your currently preferred AI Model? I have a new frontrunner. And if you have not tried Perplexity, you might want to check it out. We’ll examine how they make their product stand out in crowded spaces.

5-minute-read

Thoughts on the latest AI developments

As a product manager, when I evaluate AI solutions for my products, I need to deeply evaluate how well each AI model can solve my use cases. While integration with AI Models in almost any workflow seems promising initially, I am worried about their accuracy. Mistakes would make my product look bad.

Here are some examples where Google’s suggested answers are useless at best, confusing, and bad for your health at worst:

A Reddit user named f*cksmith suggested using glue on pizza 11 years ago. It was so good that it just “stuck” with Gemini.

The current development of large language models reminds me of self-driving cars. Those cars need tons of data to handle very rare situations. Real-world driven miles were scarce, so the industry used simulators to simulate virtually driven miles to train cars. The data produced was great but limited.

See below: a Cruise car in San Francisco drove right into a freshly poured concrete patch. It was more of a self-parking than a self-driving vehicle…

We were shown amazing demos in controlled environments, but real-world performance lagged. Year after year, engineers debated the same unsolved problems at conferences. That’s what I now see with large language models. The initial hype was huge, but progress is slowing down. This is especially true for the leading models.

Why is the improvement on AI Models slowing?

We know that AI models become more powerful the more data they train on.

• Most improvements in current large language models are based on their systems guzzling up every last bit of information they can find on the internet.

• But, existing models have been trained on more or less the entire internet already.

• The internet is too small to train next-generation models.

• AI researchers say data shortage will be a problem for further advancing existing models.

• Researchers mention that models (even the ones that started behind) are closing in on each other and are reaching similar performance levels.

Another problem

• As more people use AI, they will share more data that was created by AI

• As AI models continue to train on data from the internet, they are starting to get trained with synthetic data, meaning with data generated by AI itself.

• Researchers refer to this as a snake eating its own tail - or the models are “collapsing”.

• Imagine everyone would fall back to generate new content for the internet from AI - no more original content…

• AI would then vacuum up the new data to train an “improved model, ” but the risk is that the tails of the original data created by humans would disappear. Source - Mind Matters

Okay, but the AI business is really good, right? They must be making money hand over fist, are they not?

Did someone say bubble? Yes, the revenue growth is staggering. But it is driven with 17x the cost - congratulations Nvidia.

Let’s look at user adoption.

Comparing ChatGPTs app launch in 2023:

To Anthropics app launch (Claude) in May 2024:

We can see that the initial hype is real. ChatGPT, as the first mover, held a stable user growth rate. However, new apps launch with a short hype and cannot stabilize user growth.

Is a devaluation of these models coming?

Over the past few years, many companies have reached unicorn valuation levels—the word unicorn is used for companies valued at $1 billion+.

Given what we discussed above, how realistic can these valuations be?

• Anthropic (founded 2021): Raised $7+ billion over that last year at a $18.4 billion valuation.

• Perplexity (founded 2022): Raised $63 million at a $2.5 billion valuation, current ARR $20 million.

• Mistral AI (founded 2023): Planning to raise $600 million at a $6 billion valuation.

We are seeing a saturation in AI models, a convergence in performance levels, and unsustainable burn rates—meaning losses are eating up all capital.

The beauty of software business models (Cloud, SaaS, etc.) is the high margins that let companies quickly catch up once they hit profitability. However, AI depends heavily on hardware.

Companies that rely on AI hardware have significantly lower margins—between 50% and 60%. That’s still not bad compared to traditional business models, but scaling is more expensive and gated by more factors.

Based on these facts, one should be careful assigning high valuations to new models that are standalone - meaning they don’t directly plug int’s workflow.

How Apple is doing it

AI Models seem to be becoming a commodity, and the value to extract (making money from having great AI models) will be hard for standalone models.

Apple is developing “Apple Intelligence” in-house but has also partnered with ChatGPT to get it in users’ hands faster.

This is a really important step for OpenAI. Without this partnership, I see ChatGPT as just another tool that is not integrated into the users' daily workflow. As we predicted in one of our last posts, OpenAI found a great solution.

The differentiators are how these AI tools are being integrated into existing workflows. 

Craig Federighi explained it like any one of our loyal newsletter readers would:

So, which AI model should you use?

Chatbot Olympics

The Wall Street Journal extensively compared the five leading models in 9 categories. Here is a quick summary for you.

The contenders:

• Chat GPT (Open AI) - Launched November 2022

• Claude (Anthropic) - Launched March 2023

• Copilot (Microsoft) - Launched June 2021 as a coding assistant

• Gemini (Google) - Launched March 2023

• Perplexity - Launched 2023; exact date unclear

These are the results - usefulness compared in different categories:

The surprise is that the newer tool, Perplexity, with the likely smallest user base, wins in most categories.

How did Perplexity beat out all the others?

The differentiators that make Perplexity better.

The differentiators started to attract users, especially for research purposes. They focussed on these use cases early and built a clear advantage in this segment.

And while doing that, they came up with a better Google Search. Yes, I said it. Try it for yourself. It’s very good.

What are they doing better, and what can we learn from them?

Combination of several models

They don’t prioritize just their model. Perplexity provides existing models and lets users select between ChatGPT, Claude, and its own models within their UI.

Real-time web crawling

Perplexity’s crawlers run the Internet in real time, ensuring users get the latest information.

Direct answers with citations

Perplexity also answers questions using direct citations of the sources. So you can quickly screen the answers and understand which source you like best based on the citations. This is a big time saver and helps speed up research projects.

Rephrasing

Perplexity offers follow-up queries that are very helpful especially for people new to prompts. They are also very helpful if you are not sure yet on where your research leads you. I find it helpful for creative iteration on questions I ask it.

Serve more use-cases

Imagine you are looking to buy a new coffee machine. What would you do? Would you even consider ChatGPT? Probably not because 1. You won’t have the most recent data included. and 2. You don’t see the sources.

With Perplexity, you can ask it to show you the most insightful YouTube videos for one of the best machines, and it will list the links.

No more scrolling through endless affiliate marketing pages to get your results.

That’s it for the week. Which model is your frontrunner these days?

Please note the survey below. I would love your feedback—it just takes one click (or tap). Thank you.

Have a great rest of the week,