Which tool to choose
Two free programs, depending on how much you love commands.
- LM Studio: a graphical interface with a catalog of models you choose and download with one click, then chat with as you would with ChatGPT. The choice for those who just want to use the AI without touching the terminal.
- Ollama: operated from the terminal, it's lightweight and has become the standard for local AI. One command downloads and launches the model. Better suited to those who aren't scared off by a line of text.
If it's your first time, start with LM Studio: you see everything on screen and understand what you're doing.
How to do it
With LM Studio (graphical)
- Download LM Studio from the official site (lmstudio.ai) and install it like a normal program.
- Choose a model from the catalog inside the app. Look for a small model to start (the lightweight versions, marked with low numbers like 3B or 7B).
- Download it (it's a file of a few gigabytes) and wait for it to finish.
- Chat. Open the conversation window and type: the AI answers from your computer.
With Ollama (command)
- Download Ollama from the official site (ollama.com) and install it.
- Launch a model from the terminal. The operational syntax:
ollama run llama3.2
This downloads the model (the first time) and immediately opens the chat in the terminal. To try another, change the model name to one from Ollama's catalog.
- Just type. From there you converse as you would with any assistant, but everything stays local.
A concrete example
Giulia has an ordinary laptop, with no powerful graphics card, and wants a private AI for her personal notes. She installs LM Studio, and in the catalog she chooses a 3B model (one of the lightest) because her computer can't handle the large ones. She downloads it in ten minutes.
She opens the chat and pastes in her notes asking for a summary. The AI answers in a few seconds, a bit slower than ChatGPT but entirely sufficient. None of those notes left the laptop, she made no account, and from tomorrow she can use it on the train too, without a connection. The small model was the right move: a large one wouldn't even have started on her computer.
When it does NOT work (and how to fix it)
If it's too slow or the model won't start
Almost always the model is too large for your computer. Download a smaller version (lower number) or a "quantized" variant (compressed, lighter): in LM Studio's catalog these are flagged. A model that runs slowly is better than one that won't start.
If you don't know which model to choose
The catalog is full and confusing. Simple rule: for an ordinary computer, start from a general-purpose 3B or 7B model from a well-known name (Llama, Mistral, Gemma, Qwen, Phi). Try one, and if the answers are enough for you, stay there; if you need more and the computer can handle it, move up in size.
If the download is huge or stalls
Models weigh gigabytes: with a slow connection it takes time. Download a small model first (fewer gigabytes), and do the big downloads when you don't need the line for anything else. If it's interrupted, it usually resumes where it left off; if not, relaunch it.
A tip from someone who actually uses it
Before downloading, check how much memory (RAM) your computer has: that, more than the graphics card, decides which models you can run. As a rough reference, with 8 GB you're on small models, with 16 GB you move up a size, with more you can dare. Knowing your limit saves you from downloading gigabytes of a model that then won't start.
Frequently asked questions
Do I need a powerful graphics card?
It helps a lot for speed, but it isn't mandatory for small models: they run on the processor and memory alone, just more slowly. To get started and figure out whether local AI is for you, an ordinary computer is enough; the powerful card is needed if you want large, fast models.
Can I really use it without the internet?
Yes, after the initial download. The model lives on the disk and runs locally: once downloaded, you can disconnect. It's one of the reasons local AI is chosen, along with privacy.
Is a local AI as capable as ChatGPT?
For everyday practical tasks (summarizing, rewriting, making drafts) a good local model holds its own. On more complex reasoning the best online models remain ahead. Many keep both: the local one for what's private or offline, the online one when the highest quality is needed.