What really changes

The difference is where the "thinking" happens. With an online AI (ChatGPT, Claude, Gemini) your question travels all the way to the company's servers, gets processed there, and the answer comes back. With a local AI all of this happens inside your computer: no trip, no external server.

Hence three concrete consequences: full privacy (the data stays yours), it works offline (even without internet, after the initial download) and zero cost after installation (no subscription). The price to pay: the model runs on your computer's power, so it goes slower than a cloud service, and the free downloadable models are generally a few months behind the best paid ones online.

Which tool you need

Two free programs cover almost all cases.

  • LM Studio: it has a ChatGPT-like interface, you pick the model from a catalog and chat with it. It's the choice for those who don't want to deal with commands.
  • Ollama: it's used from the command line and has become the standard for local LLMs. Lighter and more flexible, a touch more technical.

Both download the model and start it for you; you don't have to set anything up by hand.

A concrete example

A lawyer needs an AI to analyze drafts of legal documents containing confidential client data, and doesn't want them to end up on anyone's servers. He installs LM Studio, downloads a model from the catalog and from then on pastes the texts directly into the program on his computer.

The AI summarizes and reviews the documents for him, exactly as an online one would, but none of those documents ever left the laptop. It goes a bit slower than ChatGPT and the answers are slightly less refined, but for his problem — keeping client data at home — it's the right solution. The confidentiality is worth the half-gear less.

When it does NOT work (and how to fix it)

If the computer can't handle it

The most capable models demand a lot of memory and a good graphics card. If your computer struggles or the model won't start, choose a smaller model (the "light" versions, often indicated with low numbers like 3B or 7B): they run on modest machines, are less brilliant but usable.

If the answers are less brilliant than the cloud

It's normal: free local models are behind the best online ones. For practical tasks (summaries, drafts, rewrites) they're enough; for very complex reasoning, keep an online AI for those cases and use the local one for everything that's confidential.

If you think internet is always needed

It's needed only the first time, to download the model (a file even of several gigabytes). After that, the local AI works completely offline: on a train, on a plane, without a connection. It's one of its advantages, not a limit.

A tip from someone who actually uses it

Start with a small model, even if your computer could handle a big one. A light model downloads fast, starts right away and lets you understand what it's like to work with an AI at home. If you need more quality, you move to a bigger one later. Starting from the most powerful model is the quickest way to fill up the disk and discover that the computer struggles.

Frequently asked questions

Is it really free?

Yes: the programs (Ollama, LM Studio) and the open source models are free. The only "cost" is the space on the disk and your computer's power. There's no subscription or account: you download and use.

Is my data one hundred percent safe?

On the data side yes: with a local AI it doesn't leave the computer, so it can't be collected by a provider. The normal precautions for your PC still apply (backups, antivirus, access control): the local AI removes the cloud risk, it doesn't replace looking after the computer.

Is running the AI locally always the best choice?

Here's the misunderstanding to clear up: no, it depends on what you need. For confidential data and for working offline, local is unbeatable. But if you want the highest quality, the convenience of a ready-made app and you don't have a powerful computer, online is still simpler and more capable. The right choice is often mixed: local for what's sensitive, online for the rest.