Tag: AI

The web platform team in Chrome is working on built-in AI features, where the browser provides AI models, including large language models (LLMs), to enable on-device AI for browser features and web platform APIs. This is a game changer and a huge opportunity for WordPress to democratize AI-assisted publishing. Let me tell you why.

If WordPress does not want to fall behind its competitors, it must seamlessly provide typical AI features users nowadays expect from a publishing platform. There are already various AI WordPress plugins, but they all come at a cost—both literally and metaphorically. These plugins all rely on third-party server-side solutions, which impacts both your privacy and your wallet.

Web AI has several key benefits over a server-side approach. It brings models to the browser, protecting sensitive data and improving latency.

Web AI is the overarching term for the ability to run AI solutions in the browser using JavaScript, WebAssembly, and WebGPU. This space was pioneered by libraries such as TensorFlow.js and Transformers.js. Using those tools, websites can download models and run tasks of their choice directly in the browser.

Chrome’s built-in API

If everyone has to download and update these models all the time, this doesn’t really scale and isn’t really sustainable. That’s where Chrome’s built-in AI steps in. It is just one form of client-side AI or Web AI.

With the built-in AI, your site or web app will be able to run various AI tasks against foundation and expert models without having to worry about deploying and managing said models. Chrome achieves this by making Gemini Nano available through dedicated web platform APIs, running locally on most modern computers.

At the moment, the Chrome team expects the built-in AI to be beneficial for both content creation and content consumption. During creation, this could include use cases such as writing assistance, proofreading, or rephrasing. On the consumption side, typical examples are summarization, translation, categorization, or answering questions about some content.

Early preview program participants will receive more detailed information about the following APIs:

• Prompt API
• Summarization API
• Language detection API
• Writer and rewriter APIs

To give you an example, using the prompt API is pretty straightforward:

const session = await ai.languageModel.create({
  systemPrompt: "You are a friendly, helpful assistant specialized in clothing choices."
});

const result = await session.prompt(`
  What should I wear today? It's sunny and I'm unsure between a t-shirt and a polo.
`);

console.log(result);

const result2 = await session.prompt(`
  That sounds great, but oh no, it's actually going to rain! New advice??
`);Code language: JavaScript (javascript)

The other APIs are similarly straightforward to use. By the way, if you are an avid TypeScript user, there are already type definitions which I helped write.

Web AI Advantages

• The browser takes care of model distribution and updates for you, significantly reducing the complexity and overhead.
• Everything is processed locally on-device, without sending your sensitive data elsewhere, keeping it safe and private.
• No server round trips means you can get near-instant results.
• You can access AI features even if you’re offline or have bad connectivity.
• Save money by not having to use expensive third-party services or sending large models over a network.

This is not the first time where doing things client-side is better than on the server.¹ You could of course consider a hybrid approach where you handle most use cases on-device and then leverage a server-side implementation for the more complex use cases.

Using built-in AI in WordPress

If WordPress core wants to offer AI capabilities to each and everyone of its users, it can’t jeopardize user’s privacy by relying on expensive third-party services. The project also does not have the resources or infrastructure to maintain its own API and running AI inference on a shared hosting provider in PHP is also not really something that works.

That’s why Web AI and particularly Chrome’s built-in AI are a perfect match for WordPress. With it, users benefit from a powerful Gemini Nano model that helps them accomplish everyday tasks.

The modern and extensible architecture of the WordPress post editor (Gutenberg) makes it a breeze to leverage Chrome’s built-in AI. To demonstrate this, I actually built several AI experiments. They enhance the user experience and accessibility for both editors and readers.

Here’s an overview:

Provide tl;dr to visitors

Uses Chrome’s built-in summarization API to provide readers a short summary of the post content. The UI is powered by WordPress’ new Interactivity API.

Writing meta descriptions based on the content

Using the dedicated summarizer API, the content can be easily summarized in only a few sentences.

“Help me write”

Options for rewriting individual paragraphs à la Google Doc, like rephrasing, shortening or elaborating.

Generate image captions / alternative text

Uses Transformers.js and Florence-2 to generate image captions and alternative text for images directly in the editor. Also integrated into Media Experiments, which supports video captioning too.

Generate a memorable quote

A slight variation on the summarization use case, this extracts a memorable quote from the article and gives it some visual emphasis.

Assigning tags & categories to blog posts

Suggest matching tags/categories based on the content, grabs a list of existing terms from the site and passes it to the prompt together with the post content.

Sentiment analysis for content / comments

Using a simple prompt to say whether the text is positive or negative. Could be used to suggest rephrasing the text à la Grammarly, or identify negative comments.

Summary

From summarizing content to more complex examples like automatically categorizing posts, the currently explored use cases peek at what’s possible with Web AI in WordPress. Their main advantage lies in their combined strength and the deep integration into WordPress, making for a seamless UX. Chrome’s built-in AI also makes this functionality ubiquitous, as every WordPress user could leverage it without any browser extension, plugin, or API. This is just the beginning. In the future, more complex AI features could push the boundaries of interacting with WordPress even further.

1. I’m of course talking about client-side media processing. ↩︎