The easy way to run LLMs locally
It’s a misconception that you need the huge processing power of cloud servers to run large language models (LLMs). It’s possible to run LLMs locally on relatively modestly powered PCs, even without an NPU or dedicated graphics.
If you’ve ever tried to download and run an LLM locally, you may have been put off by the need for prerequisites, environments and web UIs to make it all work. However, there is an easier way.
The free LM Studio ( lmstudio.ai) for Windows, Mac or Linux lets you download and run LLMs without any of the faff. You simply choose which LLM you want to run, either by picking from a selection on the home screen or typing its name into the search bar, and it downloads and runs them.
At the time of writing, available LLMs included Google’s recently released Gemma, Qwen from the Alibaba group, and Code Llama from Meta. LM Studio gives handy descriptions of the LLMs featured on the homepage to help you choose, as well as details of how much memory they’ll require to run and the size of the download. Most LLMs are between 2-10GB.
Once you’ve made your choice, click on the Chat icon on the left, select the model you want to interact with and start chatting away. (If you can’t see the USER chat field, close the download pane in the bottom half of the screen to reveal it.)
On the right-hand side, you should be able to access the LLMs settings. Here you can normally enter a system or pre-prompt that tells the LLM how you want to it behave. For example, “Give answers as if explaining to a five-yearold” or “assume technical knowledge”. You can also choose the level of GPU acceleration, prompt overflow settings and more.
Chatting with local LLMs requires a different mindset to using ChatGPT, Copilot or Google Gemini, which have access to live internet data. Its knowledge will be limited to the recency of its training data. For example, Meta’s Llama 2 model told us Boris Johnson was the “current Prime Minister of the United Kingdom”, while Google Gemini replied “As of October 26, 2023, the UK Prime Minister is Rishi Sunak”. Bear in mind it’s at least two weeks since we wrote this copy by the time you’re reading it, so we could be another two PMs down the line by now…
But, as we’ve discussed many times before, using these AI engines as fact checkers/search engines is a bad idea in the first place. They’re more useful for generating text (give it a few bullet points, let the AI put it into prose) or writing code than acting as a pseudo-Google. And these local LLMs are more than capable of these more basic tasks, and won’t cost you a penny to run either.