🏄 Google's AI is beating experts in coding

Google's AlphaCode 2 is a glimpse into the coding future, Adobe's super fast one-step image generation

Author

Marc Csuzi
December 08, 2023

Hello Surfers🏄!

Do you recall my newsletter from yesterday about Google's Gemini? I wrote that it '...is capable of awesome things if we can believe their hands-on demo video.'

I had my doubts, especially about the speed and smoothness shown in the demo. It made Gemini look like it could chat in real-time and react to visual cues from a camera on the spot. That's a big deal compared to the slight delay we're used to with GPT-4.

Well, it was too good to be true… Google admitted that the demo wasn't live. They didn't use spoken prompts in real-time; instead, they used video screenshots and then typed out text prompts for Gemini to respond to. They added the narration later.

So cheap, Google…

However, an overlooked part of yesterday’s demo was their stunning results with AlphaCode 2, that can change coding forever.

Let’s dive into that:

THE NEWS

🤖Google's AI is beating experts in coding

On Wednesday, when Google announced Gemini, they dropped a bunch of videos. Each one featured folks from different corners of Google, all jazzed about what Gemini can do.

In one of these videos two AI researchers talk about the potential of an AI system in coding. They quickly mention Gemini’s superior coding capabilities, compared to earlier Google models, but the real star of the show? AlphaCode 2.

AlphaCode 2 is a system developed to solve competitive programming tasks. It’s powered by Gemini and it’s a beast.

What is competitive programming?

Developers have similar competitions to those my high-school math teachers always nudged me into. Limited time, complex and open-ended problems and all these annoyingly smart kids from your town you never knew about intensely scribbling away.

Except these ones take place online instead of a stuffy classroom.

When AlphaCode 2 entered one of these coding contests, it solved 43% of the problems in just 10 tries. That's better than 85% of the people who try! That’s how tough these contests are.

How did it do that?

It’s in the numbers: The system is built on Gemini 2, but it's fine-tuned with 30 million examples of code written by people to learn how to solve problems.

AlphaCode 2 comes up with a million different code versions to solve each puzzle. The program then tries out all these different solutions, but only keeps the ones that could work, usually 5%. It basically brute forces the problem.

The system then sorts the similar ones into groups and picks the best code from the ten biggest group as final candidates.

Here's something even more awesome: when a person helps AlphaCode 2, they score above 90% of the competitors. And that's just with Gemini Pro. Imagine if it used something even more powerful, like Gemini Ultra!

But let’s not get ahead of ourselves. Despite AlphaCode 2's stellar performance, there's still a long road ahead. Their solution is a bit like throwing spaghetti at the wall to see what sticks – it works but it’s very costly and compute intensive. So we’ll still need those smart kids for now.

But with hardware getting faster, LLMs getting smarter and researchers racing to solve the problem, we might not be far from expert level AI coding capabilities. I can’t even imagine the profound changes that would bring to our life.

ONE MORE THING

Adobe researchers came up with super-fast one-step image generation

In a new research paper Adobe researchers showed off a super fast image generation method. Their model generates images at 20 FPS on modern hardware, which is almost the 24 FPS speed of movies. Real-time video generation is coming.

⌚ If you have one more minute

  • 🎭 Meta just made VR 3D Avatars even more realistic

  • 🖼️ Meta also launched a standalone AI-powered image generator 

  • 🦙 And they announce Purple Llama, an umbrella project featuring open trust and safety tools

  • 📊 Anthropic released a new dataset for measuring discrimination across 70 different potential applications of language models

AI Art of the Day 🎨

This smart answer by Grok is the art of the day! Posted by the great Jim Fan, a leading AI researcher at Nvidia.

🏄🌊🏄🌊🏄🌊🏄🌊🏄🌊🏄🌊🏄🌊🏄🌊🏄🌊🏄🌊🏄🌊🏄

That's all for today, folks!

If you enjoyed this, please consider sharing this hand-crafted newsletter with a friend.