LM Studio: GPU offload for local models
This page may contain Amazon affiliate links. A purchase after a click can earn the site a commission, at no extra cost to you. The advice stays independent. How affiliation works.
LM Studio is an app with a graphical interface for loading GGUF models (GPT-Generated Unified Format, the quantized weight format used for local LLM inference) and adjusting GPU offload (how the model’s layers split between VRAM, the graphics card’s memory, and system RAM). Ollama is often the better fit for a server stack (Docker, Home Assistant). LM Studio is a good way to compare models before you pull them with Ollama.
LM Studio labels, in plain English
| Label in the app | What it means |
|---|---|
| Discover | Browse and download models |
| Local | Models already on disk |
| GPU Layers | Layers that run on the GPU |
| Chat | A test conversation |
Prerequisites
- LM Studio is installed — the beginner LM Studio guide.
- An NVIDIA or AMD card with current drivers, or Apple Silicon (unified memory — see Mac Mini M4).
- A sense of your VRAM: VRAM and which model.
A rough table: VRAM and GPU layers (7B–8B Q4)
These are rough ranges as of October 2026. Adjust GPU Layers (GPU layers; see the label guide) for your card. Use this table while you set GPU layers (step 3 below). On a phone, a note points back to this same table instead of repeating it.
| VRAM available, about | 7B–8B Q4_K_M model | GPU layers (rough) |
|---|---|---|
| 6 GB | tight | 0–15 (often CPU) |
| 8 GB | usable (a comfortable minimum for 7B Q4) | 20–28 |
| 12 GB | comfortable | 30–35 (almost everything on the GPU) |
| 16 GB or more | plenty of room | max, or close to it |
If generation crashes or suddenly slows down, lower the layer count by 4 and test again.
Read VRAM while you adjust (Windows)
- Open Task Manager → Performance → GPU → the Dedicated GPU memory column.
- Load the model in LM Studio with GPU Layers at 0. Note the idle VRAM after the load.
- Raise the count in steps of 8. As soon as the bar nears 95% during generation, drop 4 layers.
On Linux, the nvidia-smi command (NVIDIA’s command-line tool) shows memory.used / memory.total. That is useful over SSH on a mini PC with no desktop. If you do not have NVIDIA, use the CPU and RAM table in the VRAM guide and accept a 7B that runs mostly on the processor.
Instruct models versus base models
In Discover (label guide), prefer instruct cards (models trained to talk and follow instructions) over base cards (raw weights, a weaker fit for chat). The Q4_K_M quantization (4-bit, a common size-versus-quality compromise as of October 2026) is still the household default for a 7B. The same idea is in choosing a first model.
Step by step
- Open LM Studio → the Discover tab (browse and download; see the label guide) or Local (models on disk, same guide) → download an instruct Q4 model (for example Mistral 7B Instruct).
- Open the Chat tab (a test conversation, label guide) → select the model → in the bottom-right corner of the chat window, click the CPU/GPU icon. The Hardware panel opens, with the GPU Layers slider and a CPU/GPU load preview.
- Set GPU Layers. Start at 0, then raise the count in steps of 8 while you watch VRAM (Task Manager on Windows, or
nvidia-smion Linux). Check the VRAM table before you go too high. The note under this procedure points at the same table on a phone. - Ask the same instruction three times (“Summarize this in 3 bullets…”) and compare latency and quality.
- Write down the stable setting so you can reproduce it later in Ollama if you move — choosing a first model.
The same VRAM table (step 3) — while you adjust GPU Layers, keep the VRAM and GPU-layer table at the top of the article in view (it is not copied here: one source of truth). If generation crashes, drop 4 layers and test again.
Mac with Apple Silicon
There is no discrete VRAM. Unified memory is shared. Be less aggressive if you have 16 GB unified. 24 to 32 GB makes 8B models comfortable. Detail: Mac Mini M4.
Next: integrations and local code
Once the GPU setting is stable in LM Studio, most households switch to Ollama for family chat (Open WebUI, a private assistant). For development in VS Code (Visual Studio Code, Microsoft’s editor), the Continue extension is a sidebar that sends your open files to an LLM to complete code. In practice it targets the Ollama API at http://127.0.0.1:11434, not LM Studio, unless you use an advanced setup. Dedicated guide: a local coding assistant with Continue.
Other useful links:
- GPU, iGPU, and eGPU over OCuLink if you are adding an RTX or an external enclosure.
- VRAM and which model before you buy a card.
- An electricity budget if the tower stays on for LM Studio and Ollama.
- An SSD for models: GGUF files downloaded through Discover take tens of gigabytes.
Hardware if VRAM blocks a 7B Q4 — compare your row in the table first (8 GB is the comfortable minimum for a 7B Q4, about 20–28 layers). If you are below that (6 GB, or fewer stable layers), a card with 8 GB of VRAM can be the next step. This page does not list a fixed price or one exact model: open a search and read the spec sheet (VRAM, card length, power). A rough RTX 4060 search (often 8 GB), October 2026: RTX 4060 on amazon.ca (affiliate link — affiliation).
Stability: signs the GPU setting is too high
- Generation stops mid-sentence.
- A memory error, or a black screen from the NVIDIA driver.
- VRAM sits at 100% as soon as the model loads.
What to do: lower the GPU layers by 4, restart LM Studio, and test again.
Move the model to Ollama
Once the model checks out in LM Studio (for example Mistral 7B Instruct Q4_K_M, tried through Discover), open the Mistral page in the Ollama library, pick a tag close to your LM Studio quantization, and pull it:
ollama pull mistral:7b-instruct-q4_K_M
If the command replies success (or the equivalent in your Ollama version), the same kind of model is ready for Docker and Home Assistant. Q4 and Q5 tags are detailed in installing Ollama.
Security
LM Studio sometimes listens on a local API if you turn that on. Keep the firewall enabled, and leave the port off the public internet unless you have authentication.
Common GPU-offload errors
| Symptom | What to try |
|---|---|
| VRAM stays at 0% even with GPU Layers set | An outdated NVIDIA driver, or the wrong backend selected |
| Quality looks the same on CPU and GPU | GPU layers are too low. Raise them in steps of 8 |
| A Mac with 16 GB unified memory is full | Close the browser and other apps. Test a 7B Q4 model only |
| The model is missing after Discover | Check LM Studio’s models folder. Cross-check the SSD guide |
Where to go next
- Install LM StudioA first look at Discover and a local chat.
- First steps with WhisperTranscription that does not go through Ollama.
- ComfyUI and FLUXAnother VRAM user. Run one heavy job at a time.
- Cost over 3 yearsLM Studio alone does not replace a cloud subscription.
LM Studio or Ollama: one decision table
| Situation | LM Studio | Ollama | What to do next |
|---|---|---|---|
| Try or compare GGUF files (a visual chat) | yes — GPU Layers | possible from the CLI | Write down the stable setting, then ollama pull the equivalent |
| Fine control of GPU offload | yes | partial offload, depending on the version | Keep LM Studio for the VRAM calibration |
| Less time in a terminal at the start | a guided interface (Discover, Chat) | CLI, plus integrations to explore | Install LM Studio, then Ollama |
| A stable local API (scripts, Docker) | limited | yes | Docker for Ollama and Open WebUI |
| A browser chat for the whole family | through Ollama | yes, plus Open WebUI | LM Studio stays useful for one-off tests |
| Home automation or workflows | a poor fit on its own | yes | Home Assistant, n8n |
| Code help (VS Code) | calibrate VRAM first | the Ollama API for Continue | A local coding assistant with Continue |
In practice: use LM Studio to calibrate GPU Layers on an RTX or a Mac, then use Ollama to serve that same kind of model to the whole house. Measure power use if the machine stays on (electricity).
First-session checklist (15–30 min)
- Install LM Studio — the beginner guide.
- Download a 7B instruct Q4 through Discover (label guide).
- Set GPU Layers in steps of 8 until the first memory error, then back off by 4.
- Write the stable setting in a text file (
gpu-layers-mistral-7b.txt). - Find the Ollama equivalent and run
ollama pullfor Open WebUI. - If the tower heats the living room, compare a mini PC dedicated to chat.
This checklist keeps you from staying stuck in LM Studio when the household goal is a stable Ollama API for the whole family. Need a reminder of Ollama versus LM Studio? Use the decision table just above.
Common questions
LM Studio or Ollama for a household?
LM Studio is strong for trying models in a visual interface. Ollama fits APIs, Docker, and integrations such as Home Assistant and n8n.
The GPU does not show up in LM Studio
Update the NVIDIA or AMD drivers. On a Mac, check that the Metal backend is on. Restart LM Studio after the driver install.
How many GPU layers should I set?
Raise them until VRAM is nearly full, then drop 4 layers if the app crashes or generation stops. That matches the step-by-step procedure.
Sources
Narrow it down with the hardware selector · French version of this section