​​Meta’s LLaMA 4 is wild... but did it cheat?

Newsletter #9 – April 9

Hey clever folks,

Here’s a quick no-fluff update about Meta’s LLaMA 4. 

​​Meta’s LLaMA 4 is wild… but did it cheat?

Meta just dropped LLaMA 4 this weekend —an open-weight, multimodal model family that looks amazing on paper, but (there is always a but with meta) might have some ethical issues. 

What’s New:

• LLaMA 4 understands images and video out of the box 

• It can use up to 10 million tokens of context (that’s not a typo; for context, Gemini uses 2 million)

• It crushed it on LM Arena, beating most models except Gemini 2.5 Pro

But there’s a catch.

The model topping the charts wasn’t the official open release. Meta apparently fine-tuned a special version just to win the LM Arena leaderboard (talk about cheating).

LM Arena called them out, saying the move “didn’t match what we expect from model providers.”

In short, it's a great model with questionable leaderboard ethics (very meta of meta).

2) Now, the 10-million-dollar question: Is LLaMA 4 good or not?

It’s complicated—just like a situationship.

Meta released 3 models:

• Scout (up to 10M context tokens, but hard to run in practice)

• Maverick (1M tokens)

• Behemoth (still in training)

Performance has been underwhelming in real-world tasks, especially on large codebases. Many devs say it doesn’t “pass the vibe check,” despite its benchmark wins. (Yes, vibes are now officially part of AI evaluation.)

There were even rumors that LLaMA 4 was trained on its test benchmarks, but Meta strongly denied it.

Still, the biggest upside? It’s open-weight. That makes it a valuable tool if you’re experimenting or building custom stacks—just don’t expect magic out of the box.

▶️ Watch Fireship’s video here

3) Shopify’s CEO goes full AI-first

An internal memo from Shopify’s CEO just leaked—and it’s a wake-up call for everyone to skill up and level up their AI game:

“Before hiring, prove AI can’t do the job.”

All teams must now justify hiring by proving the task can’t be done by AI. Also, employees are expected to actively learn AI. No opting out.

What does this mean? 
If you’re not building with AI, you’re falling behind.
Still clinging to old workflows? That’s not just holding you back—it’s likely holding your entire company back. Growth waits for no one.

So stay sharp. Stay current. And keep learning. 

Subscribe to newsletters like ours to level up your AI game.

More AI insights coming your way soon.

Till then, signing off.

— Written by Aaron & The Clever Nest Team