DeepMind’s AI-enabled pointer: Gemini in Chrome by pointing

Start here

In everyday words

Instead of pasting text into an AI tool, you point at the thing you mean and speak. The system uses that on-screen context to answer or take the next step.

Need a meaning?

Quick Sip

What you need to know

Who is affected: knowledge workers, developers building AI UX, product teams
What changed: On May 12, 2026, Google DeepMind described an AI-enabled pointer prototype and said Gemini in Chrome can answer questions about the specific part of a webpage you point to.
Why it matters: Most AI interfaces still make users copy context into a chat window. Pointing-and-speaking can keep work inside the app you are already using, reducing prompt friction and making “show me this” requests more reliable.
What to watch next: Watch what data is captured from the screen, how permissions and privacy controls work across apps, and whether “point-to-context” interactions hold up in messy real-world pages.

Four useful details

DeepMind outlined four interaction principles: maintain the flow, show and tell, use “this/that” shorthand, and turn pixels into actionable entities.
The post shows demos like summarizing a PDF into an email, turning a table into a chart, and using pointing plus speech for tasks in Google AI Studio.
DeepMind says these ideas are being integrated into products, including Gemini in Chrome for page-specific questions and a planned “Magic Pointer” rollout in its new laptop experience.

OpenAI · Official AnnouncementOffering Zero Data Retention for frontier models

Adds source-backed context on ai safety from OpenAI.