The Prompt API

Thomas Steiner

Published: November 11, 2024, Last updated: May 20, 2025

Explainer Web Extensions Chrome Status Intent
GitHub Experimental In EPP Chrome beta Chrome 137 beta View Intent to Experiment

With the Prompt API, you can send natural language requests to Gemini Nano in the browser.

There are many ways you could use the Prompt API in Chrome Extensions. For example:

  • Instant calendar events. Develop a Chrome Extension that automatically extracts event details from web pages, so users can create calendar entries in just a few steps.
  • Seamless contact extraction. Build an extension that extracts contact information from websites, making it easier for users to contact a business or add details to their list of contacts.
  • Dynamic content filtering. Create a Chrome Extension that analyzes news articles and automatically blurs or hides content based on user-defined topics.

These are just a few possibilities, but we're excited to see what you create.

Use the Prompt API in Extensions

There are two extension functions available to you in the LanguageModel namespace:

  • availability() to check what the model is capable of and if it's available.
  • create() to start a language model session.

Model download

The Prompt API uses the Gemini Nano model in Chrome. While the API is built into Chrome, the model is downloaded separately the first time an extension uses the API.

To determine if the model is ready to use, call the asynchronous LanguageModel.availability() function. This should return one of the following responses:

  • 'no': The browser supports the Prompt API, but it can't be used at the moment. This could be for a number of reasons, such as insufficient available disk space available to download the model.
  • 'readily': The browser supports the Prompt API, and it can be used right away.
  • 'after-download': The browser supports the Prompt API, but it needs to download the model first.

To trigger the model download and create the language model session, call the asynchronous LanguageModel.availability() function. If the response to availability() was 'after-download', it's best practice to listen for download progress. This way, you can inform the user in case the download takes time.

const session = await LanguageModel.create({
  monitor(m) {
    m.addEventListener("downloadprogress", (e) => {
      console.log(`Downloaded ${e.loaded} of ${e.total} bytes.`);
    });
  },
});

Model capabilities

The availability() function also informs you of the language model's capabilities. Apart from available, the object also has the following fields:

  • defaultTopK: The default top-K value (default: 3).
  • maxTopK: The maximum top-K value (8).
  • defaultTemperature: The default temperature (1.0). The temperature value must be between 0.0 and 2.0.
await LanguageModel.availability();
// {available: 'readily', defaultTopK: 3, maxTopK: 8, defaultTemperature: 1}

Create a session

Once the Prompt API can run, you create a session with the create() function. You can prompt the model with either the prompt() or the promptStreaming() functions.

Customize your session

Each session can be customized with topK and temperature using an optional options object. The default values for these parameters are returned from LanguageModel.availability().

const capabilities = await LanguageModel.availability();
// Initializing a new session must either specify both `topK` and
// `temperature` or neither of them.
const slightlyHighTemperatureSession = await LanguageModel.create({
  temperature: Math.max(availability.defaultTemperature * 1.2, 2.0),
  topK: capabilities.defaultTopK,
});

The create() function's optional options object also takes a signal field, which lets you pass an AbortSignal to destroy the session.

const controller = new AbortController();
stopButton.onclick = () => controller.abort();

const session = await LanguageModel.create({
  signal: controller.signal,
})

Initial prompts

With initial prompts, you can provide the language model with context about previous interactions, for example, to allow the user to resume a stored session after a browser restart.

const session = await LanguageModel.create({
  initialPrompts: [
    { role: 'system', content: 'You are a helpful and friendly assistant.' },
    { role: 'user', content: 'What is the capital of Italy?' },
    { role: 'assistant', content: 'The capital of Italy is Rome.'},
    { role: 'user', content: 'What language is spoken there?' },
    { role: 'assistant', content: 'The official language of Italy is Italian. [...]' }
  ]
});

Session limits

A given language model session has a maximum number of tokens it can process. You can check usage and progress toward that limit by using the following properties on the session object:

console.log(`${session.tokensSoFar}/${session.maxTokens}
(${session.tokensLeft} left)`);

Session persistence

Each session keeps track of the context of the conversation. Previous interactions are taken into account for future interactions until the session's context window is full.

const session = await LanguageModel.create({
  initialPrompts: [{
    role: "system",
    content: "You are a friendly, helpful assistant specialized in clothing choices."
  }]
});

const result1 = await session.prompt(
  "What should I wear today? It is sunny. I am unsure between a t-shirt and a polo."
);
console.log(result1);

const result2 = await session.prompt(
  "That sounds great, but oh no, it is actually going to rain! New advice?"
);
console.log(result2);

Clone a session

To preserve resources, you can clone an existing session with the clone() function. The conversation context is reset, but the initial prompt remains intact. The clone() function takes an optional options object with a signal field, which lets you pass an AbortSignal to destroy the cloned session.

const controller = new AbortController();
stopButton.onclick = () => controller.abort();

const clonedSession = await session.clone({
  signal: controller.signal,
});

Prompt the model

You can prompt the model with either the prompt() or the promptStreaming() functions.

Non-streaming output

If you expect a short result, you can use the prompt() function that returns the response once it's available.

// Start by checking if it's possible to create a session based on the
// availability of the model, and the characteristics of the device.
const {available, defaultTemperature, defaultTopK, maxTopK } =
  await LanguageModel.availability();

if (available !== 'no') {
  const session = await LanguageModel.create();

  // Prompt the model and wait for the whole result to come back.
  const result = await session.prompt("Write me a poem!");
  console.log(result);
}

Streaming output

If you expect a longer response, you should use the promptStreaming() function which lets you show partial results as they come in from the model.

const {available, defaultTemperature, defaultTopK, maxTopK } =
  await LanguageModel.availability();

if (available !== 'no') {
  const session = await LanguageModel.create();

  // Prompt the model and stream the result:
  const stream = session.promptStreaming('Write me an extra-long poem!');
  for await (const chunk of stream) {
    console.log(chunk);
  }
}

promptStreaming() returns a ReadableStream whose chunks successively build on each other. For example, "Hello,", "Hello world,", "Hello world I am,", "Hello world I am an AI.". This isn't the intended behavior. We intend to align with other streaming APIs on the platform, where the chunks are successive pieces of a single long stream. This means the output would be a sequence like "Hello", " world", " I am", " an AI".

For now, to achieve the intended behavior, you can implement the following. This works with both the standard and the non-standard behavior.

let result = '';
let previousChunk = '';

for await (const chunk of stream) {
  const newChunk = chunk.startsWith(previousChunk)
      ? chunk.slice(previousChunk.length) : chunk;
  console.log(newChunk);
  result += newChunk;
  previousChunk = chunk;
}
console.log(result);

Stop running a prompt

Both prompt() and promptStreaming() accept an optional second parameter with a signal field, which lets you stop running prompts.

const controller = new AbortController();
stopButton.onclick = () => controller.abort();

const result = await session.prompt(
  'Write me a poem!',
  { signal: controller.signal }
);

Terminate a session

Call destroy() to free resources if you no longer need a session. When a session is destroyed, it can no longer be used, and any ongoing execution is aborted. You may want to keep the session around if you intend to prompt the model often since creating a session can take some time.

await session.prompt(
  "You are a friendly, helpful assistant specialized in clothing choices."
);

session.destroy();

// The promise is rejected with an error explaining that
// the session is destroyed.
await session.prompt(
  "What should I wear today? It is sunny, and I am unsure between a
  t-shirt and a polo."
);

Demo

To test the Prompt API in Chrome Extensions, install the demo extension. The extension source code is available on GitHub.

Demo interface for the Prompt API

Participate and share feedback

Your input can directly impact how we build and implement future versions of this API, and all built-in AI APIs.