THE NEWS

🎥Stable Video Diffusion shakes up the Video Creation AI race

Stable Video Diffusion was released last week, and honestly, it's blowing my mind how far we've come in making videos from just a bunch of words. Remember when the idea of describing a scene and having it pop up like magic was just wishful thinking? Well, guess what? It's starting to look like that's not so far off anymore.

When Meta released its research paper last September with this example of a “flying dog with a cape” it was an interesting demonstration of the technology, but it wasn't exactly mind-blowing.

The first wave of text-to-video stuff hit us earlier this year, and oh boy, did it give us some memorable goofy content. Like Trump trying to catch an octopus and munching it down with Biden and Obama.

And The Rock eating rocks:

Other than the goofiness, these models were not exactly great for serious storytelling.

But things have been moving fast since then. A bunch of small AI companies are now in the game, pushing the tech forward. Right now, Runway ML and Pika Labs are the big names to watch, and their stuff is way closer to looking real than the early models.

A great example of this is a cosmetics commercial that was recreated with Pika Labs:

Or the Berlin version of Harry Potter:

Stable Video Diffusion enters the race

For the best results, you still start with an image generator like Midjourney or Stable Diffusion and then move to video. So, it's a two-stepper, but merging these state-of-the-art image and video models seems like it's inevitable down the line.

And Stability AI might be the one who will achieve this first. The brains behind Stable Diffusion, just launched Stable Video Diffusion(SVD), and it's a game-changer.

This new video generator is open-source and gives the big players a run for their money. Unlike Pika Labs and Runway ML, which you access through Discord or a website, you can run SVD right on your own computer.

Plus, it's free if you've got a PC with 8GB or more VRAM. And it's totally uncensored – imagine the possibilities! *Porn industry taking notes ferociously

Here's a quick comparison I did: Pika Labs, Runway ML, SVD, top to bottom.

I made the first two myself to see the differences. The SVD one is from X (but I was able to produce similar results). SVD definitely wins this round, I love how it got the flames animated.

The second comparison I found online shows it's a tight race, but SVD is holding its own.

Pictures are coming to life in a much more compelling way than six months ago and open-sourcing it will just accelerate this progress.

Right now, these models can only produce videos that last 2-3 seconds. For these tools to become truly useful, there needs to be significant improvement in their consistency and the ability to generate longer video outputs.

Despite these current limitations, I believe we are likely just 12-18 months away from experiencing enjoyable AI-generated animated videos and short films.

If you want to give SVD a try, you can do so for free here.

ONE MORE THING

Chief AI Scientist at Meta, Yann LeCun, on why we need to open-source AI.

⌚ If you have one more minute

• 🗣️ Watch Elon Musk speak in Joe Rogan’s voice made with Elevenlabs’ new Speech-to-Speech synthesis

• 🤝 US, Britain, other countries ink agreement to make AI 'secure by design'

AI Art of the Day 🎨

Another one of my favorite memes coming to life with SVD.

🏄🌊🏄🌊🏄🌊🏄🌊🏄🌊🏄🌊🏄🌊🏄🌊🏄🌊🏄🌊🏄🌊🏄

That's all for today, folks!

If you enjoyed this, please consider sharing this hand-crafted newsletter with a friend.