🏄 Microsoft's new small AI is beating Google Nano

Phi 2 is peak into the future of mobile, Tesla introduces Optimus Gen 2

Author

Marc Csuzi
December 13, 2023

Hello Surfers🏄!

We talk a lot about large language models aka. LLMs, but there’s a ton of research focusing on their little brothers, small language models or SLMs. These guys will soon run our phones and make chatting with Siri and Alexa way cooler. And guess what? Microsoft just came out with Phi2, and it’s packin’.


Here’s the deal:

THE NEWS

📱 Microsoft's new small AI is beating Google Nano

Yesterday, Microsoft unveiled Phi-2, their latest small language model. Despite its modest size of just 2.7 billion parameters, Phi-2 is a real game-changer, outdoing the larger Mistral and Llama-2 models (7B and 13B parameters respectively) in many key areas. It even surpasses Llama-2-70B, which is 25 times its size, in coding and math tasks.

And to rub salt in Google’s wound Microsoft’s researchers noted that Phi-2 matches or outperforms the recently-announced Google Gemini Nano 2, despite being smaller in size.

So, what's Microsoft's secret?

The usual approach for language models is to feed them more data. But for smaller models like Phi-2, quality trumps quantity. Microsoft's researchers worked on a more refined set of 'textbook' quality data to only train it on important information leaving out all the noise and chatter of the internet. They also cleverly integrated the knowledge from their previous 1.3 billion parameter model, Phi-1.5.

What's really impressive is that even without special training to avoid problematic content, Phi-2 naturally shows better behavior in terms of toxicity and bias compared to other open-source models that underwent specific alignment training.

The researchers couldn’t help but point out that their Phi-2 model could solve the same complex physics problems that Google used to promote its upcoming flagship AI, the GPT-4 competitor Gemini Ultra.

Currently, Phi-2 is only available for research purposes, so commercial applications are still off the table, and in that regard Gemini Nano is still the king of the mobile castle.

But the strides made with Phi-2 hint at the very near future where such models will be local, running on our devices without needing the internet, and responding instantly. Imagine having super-smart voice assistants on our phones – that's going to be a game-changer!

ONE MORE THING

Tesla shared a video of the Optimus Gen 2

The world’s most valuable car company is making good progress on its Optimus robot. In the video it handles eggs delicately and busts out some dance moves. Is Boston Dynamics getting lapped by Tesla? Watch the full video here.

⌚ If You Have One More Minute

  • 👻 Snapchat now lets subscribers share AI-generated snaps

  • 🗺️ A map on Interest in Generative AI by Country

  • 🎭 How Opera's AI bot beats Google at its own game

  • 🔬 New research shows training LLMs on synthetic data is effective

  • 🏆 GPT-4 with a new prompting strategy beats Gemini Ultra on the most important benchmark

AI Art of the Day 🎨

If Monet were painting today. Made with Midjourney by u/Quiksilver737. Check out the whole collection here.

🏄🌊🏄🌊🏄🌊🏄🌊🏄🌊🏄🌊🏄🌊🏄🌊🏄🌊🏄🌊🏄🌊🏄

That's all for today, folks!

If you enjoyed this, please consider sharing this hand-crafted newsletter with a friend.