On-Device AI in the Browser: What Gemini Nano Can and Can’t Do

“AI-powered” on a browser extension usually means your data is being posted to a server somewhere. Increasingly it doesn’t have to: Chrome ships a small language model that runs on your own machine, and an extension can use it without a single network request.

This is a practical look at on-device AI in Chrome — what Gemini Nano is genuinely good at, where it falls over, and why the availability question matters more than the capability question.

What “on-device” actually means#

The model weights are downloaded to your computer once, and inference runs locally using your CPU or GPU. When an extension summarises a thread, the text goes to a model in the browser, not over the internet.

Three consequences that matter:

• Nothing is transmitted. No request body containing your email, no provider logs, no retention policy to read.
• No API key, no per-token cost. Which changes what an extension can afford to offer for free.
• It works offline, once the model is downloaded.

The trade is that you’re running a model small enough to sit on a laptop, and that is a real constraint rather than a detail.

Chrome’s built-in AI#

Chrome exposes several JavaScript APIs backed by a local model — a general prompt interface plus purpose-built ones for summarising, writing, rewriting, translating and language detection. The underlying model is Gemini Nano, Google’s smallest model, designed for exactly this.

The important architectural detail: the model is shared. It’s downloaded once by Chrome, not bundled into each extension. So an extension using it adds kilobytes rather than gigabytes — but it also can’t guarantee the model is there, because that’s the browser’s business, not the extension’s.

This surface has moved during Chrome’s rollout — API names, availability and requirements have all changed across versions. Anything you read about it, including this, is worth checking against Chrome’s current documentation before you build on it.

What it’s good at#

Short, bounded, structural tasks over text you hand it:

• Summarising a few paragraphs into a sentence or a bullet list.
• Extraction. Pulling a date, a name, a commitment or an action item out of a message. This is where it’s strongest, and it’s the workhorse for a tool that reads email.
• Classification. Is this thread waiting on me or on them? Two or three categories, reliably.
• Rewriting. Shortening, changing tone, tidying a draft.
• Short drafts from an explicit brief.

The pattern: tasks where the input is a paragraph or two, the output is short, and the answer is largely present in the input rather than requiring outside knowledge.

What it isn’t good at#

Being straightforward about this matters, because oversold on-device AI is how people conclude the whole category is a gimmick:

• Long context. A forty-message thread with attachments is not its territory. Frontier models handle that; a small local model doesn’t.
• Multi-step reasoning. Anything that needs to hold several constraints together and work through them.
• Factual knowledge. It doesn’t know your industry, recent events, or anything outside what you gave it. Don’t ask it questions; give it text and ask it to do something with that text.
• Nuance in short messages. “Let me think about it” is ambiguous to humans too, and a small model will not resolve it reliably.
• Speed on modest hardware. It’s local, which means it competes with everything else your machine is doing.

The honest framing: it’s a capable text tool, not an assistant. Used for extraction and classification it’s genuinely useful. Used as a substitute for a frontier model it will disappoint.

The availability problem#

This is the part that decides whether you can actually ship something on it.

The model needs a supported desktop platform, a fair amount of free disk space, sufficient memory, and adequate graphics hardware. Chrome downloads it in the background when conditions are met. Mobile Chrome is not part of the picture.

Which means, from an extension’s point of view, there are several possible states at any moment: available; downloadable but not yet downloaded; downloading; or unavailable on this device entirely. A feature built on it has to handle all four gracefully — and “gracefully” cannot mean an error message, because the user did nothing wrong.

In practice this is the single biggest constraint. Not what the model can do, but that a meaningful share of your users won’t have it.

Bring your own key as the escape hatch#

The pragmatic answer is to make on-device the default and let people supply their own API key when they want more.

The user pastes a key for Claude, OpenAI or Gemini; the extension calls that provider directly from the browser; the developer never sees the key or the traffic. It’s a good pattern for three reasons:

• The user chooses the trade. They know a cloud call means their text leaves the machine, and they opted in explicitly.
• No middleman. Requests go browser → provider. The extension author isn’t proxying, storing or paying for anything.
• Capability when it’s needed. Long threads and harder reasoning get a bigger model, without forcing everyone through the cloud.

If you use an extension with a BYOK option, it’s worth checking that the key is stored locally and that requests go directly to the provider rather than through the developer’s server. Those are very different arrangements wearing the same label — much like the difference between OAuth and a content script.

Why this matters beyond privacy#

The privacy argument is the obvious one and not the only one.

An extension that calls a cloud model pays per request, which means it must either charge you, limit you, or find another way to make the numbers work. On-device inference costs the developer nothing per use. That changes what a free tier can honestly contain, and it removes the incentive to route your data somewhere it can be monetised.

It also removes an entire failure mode: an extension whose intelligence lives in someone’s server stops working the day that server does. Local inference degrades to “the model isn’t available on this device”, which is a much better worst case.

CommitLatch is built this way — Gemini Nano on-device by default for extracting commitments and summarising threads, with an optional bring-your-own-key path to Claude, OpenAI or Gemini for anything heavier. In the default configuration nothing reaches our servers, because there is no request to make. For a tool reading a work inbox, that seemed like the right default rather than a feature.

On-device AI FAQ#

What does on-device AI in Chrome mean?

The model weights are downloaded to your computer once and inference runs locally using your own hardware. When an extension summarises something, the text goes to a model inside the browser rather than over the internet — so nothing is transmitted, there is no per-token cost, and it works offline once the model is present.

What is Gemini Nano good at?

Short, bounded tasks over text you hand it: summarising a few paragraphs, extracting a date or a commitment, classifying a message into two or three categories, rewriting for tone or length, and short drafts from an explicit brief. The pattern is that the answer is largely present in the input rather than requiring outside knowledge.

What is it not good at?

Long context such as a forty-message thread, multi-step reasoning, and anything requiring factual knowledge it was not given. It is a capable text tool rather than an assistant — used for extraction and classification it is genuinely useful, used as a substitute for a frontier model it will disappoint.

Why is the model not available on my machine?

It needs a supported desktop platform, enough free disk space, sufficient memory and adequate graphics hardware, and Chrome downloads it in the background only when those conditions are met. Mobile Chrome is not part of the picture. Requirements have changed across Chrome versions, so check the current documentation.

What is bring your own key, and is it safe?

You supply your own API key for a provider and the extension calls that provider directly from your browser, so the developer never sees the key or the traffic. It is a good pattern when you need more capability than a small local model offers. Check that the key is stored locally and that requests go straight to the provider rather than through the developer’s server.

Why does on-device inference matter beyond privacy?

Because it costs the developer nothing per use, which changes what a free tier can honestly contain and removes the incentive to route your data somewhere it can be monetised. It also removes a failure mode: a tool whose intelligence lives on someone’s server stops working the day that server does.