Chrome Canary exposes an experimental Prompt API that allows user to prompt a query to the internal Gemini Nano to generate answer. The default temperature and topK are 1 and 3, respectively, and the maximum topK is 8. In this blog post, I show how to configure the temperature and topK of a prompt session so that the model can generate creative answers with larger temperature and topK.
Install Gemini Nano on Chrome
Update the Chrome Dev/Canary to the latest version. As of this writing, the newest version of Chrome Canary is 134.
Please refer to this section to sign up for the early preview program of Chrome Built-in AI.
https://developer.chrome.com/docs/ai/built-in#get_an_early_preview
Please refer to this section to enable Gemini Nano on Chrome and download the model. https://developer.chrome.com/docs/ai/get-started#use_apis_on_localhost
Disable text safety classifier on Chrome
- (Local Development) Go to chrome://flags/#text-safety-classifier.
- (Local Development) Select Disabled
- Click Relaunch or restart Chrome.
Scaffold an Angular Application
ng new prompt-api-demo
Install dependencies
npm i -save-exact -save-dev @types/dom-chromium-ai
This dependency provides the TypeScript typing of all the Chrome Built-in APIs. Therefore, developers can write elegant codes to build AI applications in TypeScript.
In main.ts, add a reference tag to point to the package's typing definition file.
// main.ts
/// <reference path="../../../node_modules/@types/dom-chromium-ai/index.d.ts" />
Bootstrap the language model
import { InjectionToken } from '@angular/core';
export const AI_PROMPT_API_TOKEN = new InjectionToken<AILanguageModelFactory | undefined>('AI_PROMPT_API_TOKEN');
export function provideLanguageModel(): EnvironmentProviders {
return makeEnvironmentProviders([
{
provide: AI_PROMPT_API_TOKEN,
useFactory: () => {
const platformId = inject(PLATFORM_ID);
const objWindow = isPlatformBrowser(platformId) ? window : undefined;
return objWindow?.ai?.languageModel;
},
}
]);
}
I define environment providers to return the languageModel in the window.ai namespace. When the codes inject the AI_LANGUAGE_PROMPT_API_TOKEN token, they can access the Prompt API to call its’ methods to submit queries to the Gemini Nano.
// app.config.ts
export const appConfig: ApplicationConfig = {
providers: [provideLanguageModel()]
};
In the application config, provideLanguageModel is imported into the providers array.
Validate browser version and API availability
Chrome built-in AI is in experimental status, and the Prompt API is supported in Chrome version 131 and later. Therefore, I implement validation logic to ensure the API is available before displaying the user interface so users can enter texts.
The validation rules include:
- Browser is Chrome
- Browser version is at least 131
- ai Object is in the window namespace
- Prompt API’s status is readily
export async function checkChromeBuiltInAI(): Promise<string> {
if (!isChromeBrowser()) {
throw new Error(ERROR_CODES.NOT_CHROME_BROWSER);
}
if (getChromVersion() < CHROME_VERSION) {
throw new Error(ERROR_CODES.OLD_BROWSER);
}
if (!('ai' in globalThis)) {
throw new Error(ERROR_CODES.NO_PROMPT_API);
}
const assistant = inject(AI_PROMPT_API_TOKEN);
const status = (await assistant?.capabilities())?.available;
if (!status) {
throw new Error(ERROR_CODES.API_NOT_READY);
} else if (status === 'after-download') {
throw new Error(ERROR_CODES.AFTER_DOWNLOAD);
} else if (status === 'no') {
throw new Error(ERROR_CODES.NO_LARGE_LANGUAGE_MODEL);
}
return '';
}
The checkChromeBuiltInAI function ensures the Prompt API is defined and ready to use. If checking fails, the function throws an error. Otherwise, it returns an empty string.
export function isPromptAPISupported(): Observable<string> {
return from(checkChromeBuiltInAI()).pipe(
catchError(
(e) => {
console.error(e);
return of(e instanceof Error ? e.message : 'unknown');
}
)
);
}
The isPromptApiSupported function catches the error and returns an Observable of error message.
Display the AI components
@Component({
selector: 'app-detect-ai',
imports: [PromptShowcaseComponent],
template: `
<div>
@let error = hasCapability();
@if (!error) {
<app-promt-showcase />
} @else if (error !== 'unknown') {
{{ error }}
}
</div>
`
})
export class DetectAIComponent {
hasCapability = toSignal(isPromptAPISupported(), { initialValue: '' });
}
The DetectAIComponent renders the PromptShowcaseComponent where there is no error. Otherwise, it displays the error message in the error signal.
// prompt-showcase.component.ts
@Component({
selector: 'app-prompt-showcase',
imports: [NgComponentOutlet],
template: `
@let outlet = componentOutlet();
<ng-container [ngComponentOutlet]="outlet.component" [ngComponentOutletInputs]="outlet.inputs" />
`,
changeDetection: ChangeDetectionStrategy.OnPush
})
export class PromptShowcaseComponent {
promptService = inject(ZeroPromptService);
componentOutlet = computed(() => {
return {
component: ZeroPromptComponent,
inputs: { isPerssion: true }
}
});
}
The PromptShowcaserComponent renders the ZeroPromptComponent dynamically.
Prompt Response Component
const transform = (value: TemplateRef<any> | undefined) => typeof value === 'undefined' ? null : value;
@Component({
selector: 'app-prompt-response',
imports: [TokenizationComponent, FormsModule, LineBreakPipe, NgTemplateOutlet],
template: `
@let responseState = state();
@if (perSessionTemplate()) {
<ng-container *ngTemplateOutlet="perSessionTemplate(); context: perSessionTemplateContext()" />
}
<div>
<span class="label" for="input">Prompt: </span>
<textarea id="input" name="input" [(ngModel)]="query" rows="3"></textarea>
</div>
<button (click)="submitPrompt.emit()">Submit</button>
<div>
<span class="label">Response: </span>
<p [innerHTML]="responseState.response | lineBreak"></p>
</div>
`,
changeDetection: ChangeDetectionStrategy.OnPush
})
export class PromptResponseComponent {
state = input.required<PromptResponse>();
query = model.required<string>();
perSessionTemplate = input(null, { transform });
perSessionTemplateContext = input<any | undefined>(undefined);
submitPrompt = output();
}
The PromptResponseComponent displays a text area where users can enter a query. Then, they click the button to submit the query to the internal Gemini Nano, which generates a text answer. The submitPrompt output function notifies the ZeroPromptComponent component that a user query has been submitted. Finally, the LineBreakPipe pipe cleanses the response before displaying it.
Zero Prompt Component
// zero-prompt.component.ts
@Component({
selector: 'app-zero-prompt',
imports: [FormsModule, PromptResponseComponent],
template: `
<div class="session">
<app-prompt-response [state]="responseState()" [(query)]="query"
(submitPrompt)="submitPrompt()"
[perSessionTemplate]="isPerSession() ? session : undefined"
[perSessionTemplateContext]="templateContext()"
/>
<ng-template #session let-capabilities="capabilities">
<div>
<div>
<span class="label" for="temp">Temperature: </span>
<input type="number" id="temp" name="temp" class="per-session" [(ngModel)]="capabilities.temperature" max="3" />
<span class="label"> (Max temperature: 3) </span>
<span class="label" for="topK">TopK: </span>
<input type="number" id="topK" name="topK" class="per-session" [(ngModel)]="capabilities.topK" max="8" />
</div>
<div>
<span class="label" for="temp">Per Session: </span>
<span>{{ capabilities.description }}</span>
</div>
</div>
</ng-template>
</div>
`,
providers: [
{
provide: AbstractPromptService,
useClass: ZeroPromptService,
}
],
changeDetection: ChangeDetectionStrategy.OnPush
})
export class ZeroPromptComponent extends BasePromptComponent {
isPerSession = input(false);
zeroPromptService = this.promptService as ZeroPromptService;
responseState = computed<PromptResponse>(() => ({
response: this.response(),
}));
temperature = this.zeroPromptService.temperature;
topK = this.zeroPromptService.topK;
templateContext = computed(() => this.isPerSession() ? {
capabilities: {
temperature: this.temperature,
topK: this.topK,
configValues: this.zeroPromptService.configValues(),
}
} : undefined
);
}
The session ngTemplate displays two number fields to enter the temperature and topK. The templateContext context holds the temperature and topK signals to bind to the number fields and the string representation of the configurations. The ZeroPromptService will configure the prompt session using these signal values. The PromptResponseComponent component allows users to enter a query and displays the results. When the isPerSession input is true, the ZeroPromptComponent renders the session template to allow users to provide the temperature and topK.
const cmpCapabilties = (a: Capability, b: Capability) => a.temperature === b.temperature && a.topK === b.topK;
capabilities = computed(() => ({
temperature: this.temperature(),
topK: this.topK(),
}));
constructor() {
toObservable(this.capabilities)
.pipe(
debounceTime(300),
distinctUntilChanged(cmpCapabilities),
switchMap(async (data) => {
await this.zeroPromptService.resetConfigs(data);
await this.zeroPromptService.createSessionIfNotExists();
}),
takeUntilDestroyed(),
).subscribe();
When users change the temperature or topK, the capabilities computed signal recomputes the value. The toObservable function tracks the capabilities signal and calls the ZeroPromptService to destroy the old session and create a new one.
Base Component
@Directive({
standalone: false
})
export abstract class BasePromptComponent {
promptService = inject(AbstractPromptService);
session = this.promptService.session;
query = signal('Tell me about the job responsibility of an A.I. engineer, maximum 500 words.');
response = signal('');
async submitPrompt() {
this.response.set('');
const answer = await this.promptService.prompt(this.query());
this.response.set(answer);
}
}
The BasePromptComponent provides the submit functionality and signals to hold the query, response, and view states.
The submitPrompt method submits the query to Gemini Nano to generate texts and assign them to the response signal.
Define a service layer over the Prompt API
The ZeroPromptService service encapsulates the logic of the Prompt API.
When users provide a new temperature and topK, the resetConfigs method updates the values in the signals and destroys the old session. The createPromptSession creates the session with temperature and topK. When the service is destroyed, the ngOnDestroy method destroys the session to avoid memory leaks.
@Injectable({
providedIn: 'root'
})
export class ZeroPromptService extends AbstractPromptService implements OnDestroy {
#controller = new AbortController();
topK = signal(3);
temperature = signal(1);
configValues = computed(() =>
`\{topK: ${this.topK()}, temperature: ${this.temperature()}\}`
);
override async createPromptSession(options?: PromptOptions): Promise<AILanguageModel | undefined> {
const createOptions = {
signal: this.#controller.signal,
temperature: this.temperature(),
topK: this.topK()
};
return this.promptApi?.create({ ...options, ...createOptions });
}
async resetConfigs(config?: { temperature: number, topK: number }) {
const capabilities = await this.promptApi?.capabilities();
const defaultTemperature = capabilities?.defaultTemperature || 1;
const defaultTopK = capabilities?.defaultTopK || 3;
const { temperature = defaultTemperature, topK = defaultTopK } = config || {};
this.destroySession();
this.temperature.set(temperature);
this.topK.set(topK);
}
ngOnDestroy(): void {
this.destroySession();
}
}
The AbtractPromptService defines standard methods other prompt services can inherit.
The createSessionIfNotExists method creates a session and keeps it in the #session signal for reuse. A session is recreated when the old one has very few tokens remaining (< 500 tokens).
export abstract class AbstractPromptService {
promptApi = inject(AI_PROMPT_API_TOKEN);
#session = signal<AILanguageModel | undefined>(undefined);
#tokenContext = signal<Tokenization | null>(null);
#options = signal<PromptOptions | undefined>(undefined);
resetSession(newSession: AILanguageModel | undefined) {
this.#session.set(newSession);
this.#tokenContext.set(null);
}
shouldCreateSession() {
const session = this.#session();
const context = this.#tokenContext();
return !session || (context && context.tokensLeft < 500);
}
setPromptOptions(options?: PromptOptions) {
this.#options.set(options);
}
async createSessionIfNotExists(): Promise<void> {
if (this.shouldCreateSession()) {
this.destroySession();
const newSession = await this.createPromptSession(this.#options());
if (!newSession) {
throw new Error('Prompt API failed to create a session.');
}
this.resetSession(newSession);
}
}
}
The abstract createPromptSession method allows concrete services to implement their own sessions. A session can have zero prompt, a system prompt, or an array of initial prompts.
abstract createPromptSession(options?: PromptOptions): Promise<AILanguageModel | undefined>;
The prompt method creates a session when one does not exist. The session then accepts a query to generate and return the texts.
async prompt(query: string): Promise<string> {
if (!this.promptApi) {
throw new Error(ERROR_CODES.NO_PROMPT_API);
}
await this.createSessionIfNotExists();
const session = this.#session();
if (!session) {
throw new Error('Session does not exist.');
}
const answer = await session.prompt(query);
return answer;
}
The destroySession method destroys the session and resets the signals in the service.
destroySession() {
const session = this.session();
if (session) {
session.destroy();
console.log('Destroy the prompt session.');
this.resetSession(undefined);
}
}
In conclusion, software engineers can create Web AI applications without setting up a backend server or accumulating the costs of LLM on the cloud.
Top comments (0)