🏄Nvidia’s robotics training breakthrough

Star Wars robots soon?

Author

Marc Csuzi
October 25, 2023

Hello Surfers🏄! 

I have to admit, when I first stumbled upon tweets from Nvidia’s Chief Scientist discussing their latest research, it sounded like pure technical mumbo jumbo.

A year ago, this would've meant plunging into the depths of Wikipedia, and embarking on journeys down Google rabbit holes. Only to emerge hours later with a headache, 57 open browser tabs and a vague understanding of the topic.

But today? It's a twenty-minute Q&A with my trusty sidekick, ChatGPT. Sometimes, it still feels like we're living in a sci-fi novel.

Here’s what all the tech enthusiasts are buzzing about, in terms even us regular folks can get:

ONE PIECE OF NEWS

📈Nvidia’s robotics training breakthrough

Here’s how you train an AI when it comes to playing Minecraft, learning to walk in a simulation with real physics or playing football in real life.

You set up rules and give out rewards and penalties:

  • Walk towards that ball? +1 point. (Good robot!)

  • Take a step back? -1 point. (No cookie for you.)

  • Kicking that ball towards the goal? +5 points. (MVP!)

  • Whiff it in the wrong direction? -5 points. (Oopsie daisy.)

  • Make the crowd go wild with a GOAL?! A whopping +100.

This is called reinforcement learning. The goal for the robot is to maximize its total points. Given these rules (called a reward function), the robot will quickly learn to go to the ball, kick it towards the goal, and score. Pretty simple, right?

Now here’s a question for you: How would you design instructions for a robot hand to spin a pen around its fingers?

Tough one, right? Even Nvidia’s researchers were stumped. But then, they had an “Aha!” Moment…. So what did they do? The same as all of us would do if we had a tedious task to solve: Ask ChatGPT.

And guess what? It worked.

They call their system Eureka. While ChatGPT tweaks the rules a.k.a. the reward functions, the inner system runs the reinforcement learning, spinning that pen.

But hold on, it's not what you think. There's no robot hand in an empty lab practicing pen tricks while ChatGPT is frenetically working out the best incentives behind the scenes.

It's more like the training of Neo in "The Matrix" - "I know Kung Fu" style. Eureka can learn in a GPU-accelerated physics simulator that speeds up reality by 1000x. On top of that, the system can simultaneously test different reward functions. Making the progress super fast.

Ready to have your mind blown? Eureka outperformed experts in writing rewards on 83% of tasks, and the tougher the task, the weirder and more brilliant its strategies got. It’s like watching a grandmaster from another planet play chess, with a logic we’ve never seen before.

On top of all that it can take human feedback in natural language to revise and adjust its reward functions, making it the perfect co-pilot for robot engineers to design sophisticated motor behaviours.

Did we just get one step closer to having a Star Wars like world of nifty robots running around, repairing things, and doing tasks on their own? I certainly hope so.

By the way, these pit droids in 'The Mandalorian' are definitely cheating. Look at the two in the front sneakily passing a card.

ONE MORE THING

Big news for companies wanting to keep their data private and run AI on their own servers.

⌚ If you have one more minute

  • 🔬 Research suggests GPT-4 is surprisingly helpful in radiology, on par with task-specific radiology models

  • 📊 Everything Microsoft said about AI on its quarterly earnings call

  • 💵 Google considers subscription business models for its new AI offerings

  • ⚠️ AI risk must be treated as seriously as climate crisis, says Google DeepMind chief

AI Art of the day 🎨

Just a reminder, that AI can churn out photo-realistic images of people. Check out the whole collection of everyday women made with Stable Diffusion by reddit user u/Licovoda.

🏄🌊🏄🌊🏄🌊🏄🌊🏄🌊🏄🌊🏄🌊🏄🌊🏄🌊🏄🌊🏄🌊🏄

That’s it folks!

If you liked it, please share this hand-crafted newsletter with a friend and make this writer happy! ar