- Riding the Wave
- Posts
- 🏄 How to run LLMs on your laptop
🏄 How to run LLMs on your laptop
Next time OpenAI wants to implode, you'll know what to do.
Marc Csuzi
November 24, 2023
Hello Surfers🏄!
Now that everyone has calmed down in the Silicon Valley to focus on celebrating Thanksgiving, I brought you a tutorial about how to run LLMs on your laptop, so next time there’s drama in the OpenAI HQ, you know how to get your own language model running.
Here’s the deal:
THE NEWS
How to Run Language Models on Your Laptop
Have you ever wanted to experiment with language models in total privacy, without someone snooping on your activities?
LM Studio comes to the rescue! It’s a free tool that you can quickly install on Mac, Windows, or even Linux. Just download it, and you're all set.
So, why run models on your laptop instead of just using ChatGPT online?
First off, you can use it offline, anywhere you go. It's also totally private — no companies monitoring your inputs, which means you can work with sensitive data without any risks.
Plus, you get to use uncensored models. This way, you won’t constantly bump into those frustrating 'Sorry, I can’t help with that' messages every few queries.
LM Studio
Once you've got LM Studio up and running, it’s super easy to find and download different open-source models. No need to hop around different websites like Hugging Face or GitHub – it’s all there in the app. Plus, there’s tons of helpful info about each model, including technical papers for the curious minds.
The software itself is pretty user-friendly. You can search for models easily, see which ones work best with your computer setup, and get warnings about models that might be too heavy for your system.
Downloading a model is super easy
They suggest having at least 32 GB of RAM for a smooth experience, but I managed with my 8GB M1 MacBook Air, albeit more slowly.
Pro tip! If you have more than 8 GB, turn on metal support in the settings to get super fast speeds on Macs.
For the tech-savvy folks, there’s a cool local server feature that mimics the OpenAI API, perfect for developing your own AI-powered apps.
So, if you're into AI and language models, LM Studio is definitely worth checking out. It's fun, informative, and a great way to dip your toes into the world of AI right from your laptop.
ONE MORE THING
Researchers taught robots by making GPT-4 Vision watch videos.
This is a great little step toward training robots without programming.
Neat idea for how to program a robot using GPT-4V.
Have GPT-4V watch videos of a human performing a task, then have it come up with a plan of how to replicate it & write the code to make the robot do the actions. arxiv.org/pdf/2311.12015…
— Ethan Mollick (@emollick)
5:41 AM • Nov 23, 2023
⌚ If you have one more minute
🏎️ Formula One trials AI to tackle track limits breaches
🇨🇳 Nvidia delays launch of new China-focused AI chip
📚 An intro to the core ideas and approaches of fine-tuning large language models
AI Art of the Day 🎨
When you post an AI art on Twitter or reddit… made with DALL-E 3 by reddit user u/LizzidPeeple.
LizzidPeeple didn’t share the prompt for the picture though….
🏄🌊🏄🌊🏄🌊🏄🌊🏄🌊🏄🌊🏄🌊🏄🌊🏄🌊🏄🌊🏄🌊🏄
That's all for today, folks!
If you enjoyed this, please consider sharing this hand-crafted newsletter with a friend.