How to run local LLMs with Ollama and keep your code private

• What you'll learn: install Ollama, download a model and use it with a programming agent.
• Who it's for: anyone who wants project code to stay on their own computer.
• Outcome: a local model responding and helping you code.
• Prerequisite: a computer with enough memory for the chosen model.

LLM stands for large language model, the type of AI behind programming assistants. Normally it runs in the cloud, on a company's servers.

With Ollama, the model runs on your computer. The code you show the AI does not need to leave your machine.

What you need before you start

• Windows, macOS or Linux.
• VS Code or Antigravity IDE.
• A test project in Git.

Step 1: install Ollama

Download the installer from the official site, ollama.com, and follow the screens. On Linux, the documentation provides an install command for the terminal.

To check if it worked, open the terminal and run:

$ ollama --version

Step 2: download and test a model

Choose a programming model from Ollama's site library. Start with a small one.

$ ollama run qwen2.5-coder

The first use downloads the model. Then it opens a conversation in the terminal. Ask something simple, like "write a function that adds two numbers".

To exit, type /bye.

To see all the models you've already downloaded, run ollama list. This helps you remember the exact name when configuring the agent.

Step 3: connect the model to an agent

Chatting in the terminal is good for testing. To actually code, connect Ollama to an agent that reads and edits the project.

Ollama leaves a local server running on your computer. Agents like Cline, OpenCode and Aider know how to talk to it. In the agent settings, choose Ollama as the provider and select the downloaded model.

See more options in our top 5 open source AI assistants to run locally.

Local or cloud?

Local models tend to be smaller. They work well for simple tasks and for code that cannot leave the computer.

For larger tasks, a cloud agent can help. In Claude Code, for example, you sign in with a Pro or Max subscription, no key needed. The step by step is in our guide on how to install and use Claude Code in VS Code.

Practical workflow: create, review and publish

1. Create

Write the request as if you were talking to a colleague:

Request: "Create a function that validates CPF and write tests for it. Before you change anything, show me the plan."

Read the plan, approve it or ask for adjustments. Then the agent writes the code and runs the project to check.

2. Review

Look at the changes before accepting them. The editor's Source Control tab shows each line changed.

Also ask: "Review what you did and explain it in simple language." If something breaks, paste the error in the chat and ask for a fix.

3. Publish

Git is the project's version history. A pull request, or PR, is a request to merge changes into the main project after a review.

1. Ask: "Create a branch called valida-cpf and make a commit with a clear message."
2. Ask: "Push the branch to GitHub and open the PR."
3. Review the PR on GitHub and merge it.

The same workflow in bridgecode, with separate roles

bridgecode accepts Ollama as a provider. This way you can mix local and cloud models on the same team:

• Planner: reads the request and breaks it into small tasks.
• Executor: writes the code for each task.
• Reviewer: checks everything and points out errors before the PR.

Each agent works on a separate copy of the project, so one does not interfere with the other. You approve the plan at the start and the PR at the end.

For example, the local model handles tasks with sensitive code and a cloud agent reviews only what you release.

Common mistakes and how to avoid them

Downloading a model that's too large

If your computer freezes or slows down, switch to a smaller model.

Forgetting to leave Ollama running

If the agent can't find the model, check that Ollama is open.

Assuming everything stays local

If you mix in cloud agents, what they read goes to the cloud. Keep a clear separation of what can and cannot leave your machine.

To understand why agent environments are growing, see why devs are switching traditional IDEs for agentic environments.