Can You Still Run Large Models on 2GB of VRAM? I Deployed Qwen3.5 2BB on an Old MX250 Laptop

Preface

In Which local LLM performs best? An LM Studio benchmark on the 8845HS (780M iGPU), I already compared how different local models perform on the 8845HS iGPU. This time it's more extreme: I'm taking a first-gen 2019 MateBook 14 and deploying on it, to see whether this seven-year-old laptop can still be put to use.

I bought this MateBook 14 in 2019 during college. My Flying Fortress had broken down at the time. I had a desktop, but it wasn't convenient for lab classes, so I splurged on my first ultrabook. I have to say, after getting used to gaming laptops (I even spent a stretch commuting on the subway with a 3kg+ Flying Fortress on my back), switching to an ultrabook was a huge jump in experience—so nice. I could just tuck it under my arm for class; it was basically weightless 😄

Unfortunately, not long after I bought it, the next-gen MateBook 14 came out—more specs for the same price, with RAM bumped to 16GB 😭

After graduation, this laptop gathered dust at home for a long time, and its performance couldn't keep up anymore. I thought about selling it second-hand, but I was too lazy to ever do it. Only recently did I pull it out, reinstall Linux, and start using it to deploy some services—not bad as a home server.

This time I'm going a step further and deploying a local LLM. Later I can also hook it up to something like OpenClaw or a Hermes Agent, and see whether this MX250 GPU can actually contribute something.

First, a Look at This Old Machine's Specs

Let's start with the hardware.

This MateBook 14 is the 2019 first-gen model. It only has 8GB of RAM, a discrete NVIDIA MX250 GPU, and just 2GB of VRAM. (See the appendix at the end for full specs.)

Back in 2019, the MX250 still counted as a fairly common entry-level discrete GPU on ultrabooks—good enough, barely, for light games like League of Legends.

But in today's era of 7B, 14B, even tens or hundreds of billions of parameters, 2GB of VRAM looks a bit ridiculous.

Especially since when I was tinkering with local models on that 8845HS machine, I was still looking at models around 4B and 9B, like Qwen and Gemma.

With the MX250, the whole approach has to change.

The focus is figuring out which models can fit entirely into this 2GB of VRAM.

Why Can't You Just Copy Today's 7B / 9B Playbook?

When people talk about local LLMs now, 7B–9B is basically already considered a “small model.”

If you have 16GB or 32GB of RAM, or a halfway decent GPU, running a 7B/8B Q4 quantized model isn't anything special.

But the MX250 only has 2GB of VRAM.

And there's a common misconception here: just because the model file fits in VRAM doesn't mean the model can actually run.

At runtime, besides the weights themselves, you also need extra space for the KV Cache, runtime buffers, context, and so on. The longer the Context Length, the more the KV Cache typically occupies.

So on a GPU with only 2GB of VRAM, there's already barely any room to maneuver.

7B and 9B are basically off the table.

Even with CPU + GPU hybrid offloading—in theory putting some layers on the GPU and some in system RAM—it doesn't mean much for a machine with only 8GB of RAM.

You'd likely end up with a very awkward result: an output speed of 1 token/s 😂

Since my goal is to use this computer as a small server that runs long-term, I care more about stability, low resource usage, and actual usability.

Sticking with LM Studio

There are plenty of local model runtimes now—Ollama, llama.cpp, vLLM, and so on—but this time I'm sticking with LM Studio, which I'm already familiar with.

On one hand, its underlying inference Runtime is already mature; on the other, it's not just a GUI app.

LM Studio comes with a very handy CLI:

lms

So even if this MateBook mainly serves as a server going forward, I can still download, load, and unload models and manage the API Server entirely from the terminal.

After installing LM Studio and running it once, you can check:

lms --help

List downloaded models:

lms ls

See which models are currently loaded into memory:

lms ps

To load a model:

lms load <model>

At that point, the experience is already pretty close to Ollama.

The GUI can handle finding models and tuning parameters up front; once the config is stable, you can mostly just use the CLI after that.

The Model Can Only Go Smaller

At first, I still instinctively looked for models the way people do with local LLMs today.

But I quickly realized that in the face of 2GB of VRAM, a lot of so-called “small models” aren't small at all.

Forget 7B / 9B—even the 4B class would squeeze VRAM and system RAM uncomfortably hard.

And for this machine, I didn't want to force a bunch of CPU Offload just so the model could “start.”

If most of the compute still lands on the CPU, then there's no point in the MX250 existing.

So the idea gradually became clear: the model itself should be small enough to fit as completely as possible into the MX250's 2GB of VRAM.

In the end, the target landed around 2B.

It used to be that 1B and 2B models were basically written off as “toys”; but in the past few years, model architecture, training data, and post-training capabilities have all improved very quickly. For simple text processing, structured tasks, classification, summarization, even lightweight Agent scenarios, 2B is no longer something completely unusable.

My goal was never to have the MX250 write large projects here anyway.

Having it take on some simple, fixed, predictable local tasks already accomplishes the goal.

Settling on Qwen3.5 2B

After all the tinkering, I ended up choosing: Qwen3.5 2B

In LM Studio, that's:

qwen/qwen3.5-2b

Here I think the way you choose a model matters more than which specific model you pick.

On a machine with very limited resources, you should first establish a resource budget, not chase whichever model has the highest Benchmark.

For the MX250, that budget is brutally simple: 2GB VRAM.

Only then do you look for a model that can be fully—or as fully as possible—GPU Offloaded.

If you chase a 7B model only to have most of the compute fall back to the CPU, waiting forever for every few characters of output, then the so-called stronger model capability doesn't mean much in practice.

A slightly weaker 2B model that actually runs smoothly is much more reasonable for this kind of old machine.

Using lms load to Control Model Load Parameters

Once the model was decided, resources were so tight that I couldn't just load it blindly—I had to tweak some parameters.

The command I ended up using:

lms load qwen/qwen3.5-2b \
  --context-length 4096 \
  --identifier qwen-commit

Load result:

Model loaded successfully in 3.70s.
(1.81 GiB)

To use the model in the API/SDK,
use the identifier "qwen-commit".

Two parameters here are fairly important.

• --context-length 4096
• --identifier qwen-commit

identifier is straightforward: it's like giving the currently loaded model a name for API calls.

Later, when other programs call LM Studio, you don't need to care about the specific GGUF filename or model version—just use qwen-commit.

Even if I swap the underlying model later, as long as the identifier stays the same, the upper-layer apps don't need config changes.

Why Only 4096 Context?

With only 2GB of VRAM, Context is not a “bigger is better” parameter.

A longer Context means the model can remember more, but at the same time, runtime resource usage like the KV Cache also increases.

With only 2GB of VRAM, you have to compromise among context capability, VRAM usage, and inference performance.

And if you think about it, the tasks this machine is actually suited for later on won't need particularly long context anyway.

For example, simple summarization, text classification, information extraction, format conversion, simple tool calls—4K Context is already enough to handle a lot.

If you really need tens of K of Context, just use a cloud model honestly. No need to torture the MX250 😂

2GB of VRAM Is Really Tight

After the model loaded successfully, I checked nvidia-smi.

GPU memory usage: 1928 MiB / 2048 MiB

Basically hitting the ceiling.

LM Studio showed the loaded model size as about 1.81 GiB.

The MX250's 2GB of VRAM was almost completely squeezed dry, with only a little over 100 MiB left.

So can you still run today's LLMs on 2GB of VRAM? It's been verified: yes, you can.

Next we can look at how far you can actually push such limited resources.

How Fast Is It in Practice?

Of course, loading successfully is only the first step.

Once you actually use it, the gap between the MX250 and modern GPUs is still very obvious.

Once you hook it up to an Agent like Hermes, the Agent itself also brings extra content like the System Prompt, tool descriptions, and context. What actually gets sent to the model in one request isn't just the “hello” you see on screen.

So plain chat is still acceptable, but as soon as you enter an Agent scenario, speed and experience drop immediately.

While experimenting, I also realized that a full Agent Framework like Hermes may not actually be the best use for this machine.

A 2B model's reasoning ability is limited to begin with, and Agent is precisely a scenario that heavily tests instruction following, Tool Calling, planning, and long-context capability.

If you have it do simple text tasks, Qwen3.5 2B can work.

But if you expect it to autonomously plan and call tools in sequence like Claude Code, Codex, or today's large-model Agents, you're clearly asking too much of it.

So my positioning of this machine shifted a bit later on: a lightweight local inference node that can stay online long-term.

You Don't Have to Use It Directly on the MateBook

Once the model is running, there's another issue.

This MateBook itself is used as a Home Server; I don't usually sit in front of it to operate it.

So how do other computers use the model?

LM Studio itself can start a local Server compatible with the OpenAI API.

For example:

lms server start --bind 0.0.0.0 --port 1234

Then other devices on the same LAN can access it via this MateBook's IP address:

http://192.168.x.x:1234/v1

That way my desktop, laptop, even other servers can treat it as an OpenAI Compatible API.

For example, listing models:

curl http://192.168.x.x:1234/v1/models

Or calling the qwen-commit we just set up:

curl http://192.168.x.x:1234/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen-commit",
    "messages": [
      {
        "role": "user",
        "content": "你好"
      }
    ]
  }'

At this point the MateBook starts to feel a bit like an “AI Server.”

The client devices don't need to install the model at all.

Virtual Networking and LM Link

I also previously tinkered with LM Studio's built-in LM Link:

lms link enable

And kept getting:

LM Link enabled. Connecting...
LM Link enabled but could not connect.
Use lms link status for details.

Then it suddenly hit me: why do I have to use LM Link?

I already have a virtual network set up—can't I just call it via the internal IP?

For example:

http://100.x.x.x:1234/v1

The overall architecture:

台式机 / 笔记本 / 平板
        │
        │ 组网
        ▼
2019 MateBook 14
        │
        ▼
    LM Studio
        │
        ▼
  Qwen3.5 2B
        │
        ▼
     MX250

One thing to note: once the LM Studio API is bound to a non-localhost address, other devices on the network may also be able to access it.

In a LAN environment, it's best to confirm firewall rules; if you need to expose it to more devices

MateBook 14 Detailed Specs

Core specs

• Processor (CPU): optional 8th-gen Intel Core i5-8265U or i7-8565U.
• Graphics (GPU): Intel UHD 620 integrated graphics, or NVIDIA GeForce MX250 (2GB GDDR5) discrete GPU.
• Memory (RAM): 8GB LPDDR3 2133MHz.
• Storage: 512GB PCIe NVMe SSD.

Display

• Size and aspect ratio: 14-inch 3:2 productivity golden-ratio full-screen display.
• Resolution: 2160 × 1440 (185 PPI), 100% sRGB color gamut, peak brightness 300 nits; some versions support multi-touch.

Design and portability

• Dimensions: 307.5 mm × 223.8 mm × 15.9 mm.
• Weight: about 1.49 kg.
• Material: metal chassis.

Ports and features

• Ports: 1× USB-C (data and charging), 1× USB 3.0, 1× USB 2.0, 1× HDMI, 3.5mm combo audio jack.
• Features: fingerprint reader integrated into the power button, hidden pop-up webcam (located between F6/F7 on the keyboard), supports Huawei Huawei Share OneHop.