Google is pushing its AI assistant one step further on the Mac. The Gemini app now includes a "computer use" feature that can click buttons, type, navigate menus and fill out forms in desktop applications and web sites—all from an isolated background window. Your cursor and main screen stay untouched, letting you continue working while Gemini handles repetitive chores.
Key takeaways
- Gemini can control apps, files and websites on Apple Silicon Macs running macOS Sequoia 15.0 or later.
- The AI operates in a separate background window with a live preview in the chat, so you can monitor its actions.
- Feature is limited to US users with a Google AI Plus, Pro, or Ultra subscription (plans start at $4.99/month).
- You must explicitly add @Computer use to your prompt and approve any "key actions" Gemini wants to take.
- Apple’s upcoming macOS 27.3 will add clearer permission prompts and transparency indicators for AI‑driven app control.
How computer use works
When you ask Gemini to perform a multi‑step task—such as batch‑editing photos, generating a presentation from data across several sites, or turning a PDF shipping quote into a spreadsheet—you include the @Computer use tag in your request. Gemini then opens an isolated background window, executes the required clicks and keystrokes, and streams a live preview back to the chat. You can stop the process with a single click, and Gemini will pause for logins, payments or any other sensitive actions, awaiting your confirmation.
What you can do with the new feature
- Batch photo edits: Select a folder of images, tell Gemini to apply a preset, and let it run while you work elsewhere.
- Data‑driven presentations: Ask Gemini to pull metrics from several web dashboards, paste them into Keynote, and format slides automatically.
- E‑commerce listings: Provide product details and let Gemini fill out resale platform forms, upload images and set prices.
- 3D modeling assistance: In a demo, Gemini used Blender to turn a sketch into an animated model, then launched an AR web app to view it.
Getting started
To try computer use, follow these steps:
- Confirm you have an Apple Silicon Mac (M1, M2, etc.) running macOS Sequoia 15.0 or newer.
- Subscribe to Google AI Plus, Pro, or Ultra (US‑only, $4.99+/month).
- Download the Gemini app from Google’s website.
- Open the app, go to Customize > Connectors, and enable the new "Desktop" apps you want Gemini to access.
- In the chat, type @Computer use followed by a clear description of the task.
- Watch the live preview, approve any key actions, and stop the process if needed.
Safety and privacy
Google emphasizes that Gemini operates in an isolated window and never hijacks your cursor. You choose which apps it can access, and the AI asks for confirmation before performing actions that involve logins, payments or other sensitive data. Apple’s upcoming macOS 27.3 will add a transparency indicator whenever an app controls another app, and users will be able to limit access to specific folders and apps on a per‑request basis.
Subscription and pricing
| Plan | Monthly price (US) | Access |
|---|---|---|
| AI Plus | $4.99 | Computer use + standard Gemini features |
| AI Pro | Varies (starts at $4.99) | Enhanced performance, higher usage limits |
| AI Ultra | Varies (starts at $4.99) | Top‑tier speed, priority support, unlimited computer use |
What this means for you
If you regularly juggle repetitive Mac tasks—like formatting spreadsheets, uploading product listings, or stitching together media—Gemini’s computer use can free up minutes or even hours each week. Because the AI works in the background, you keep full control of your primary workspace. The feature also opens the door for developers to build custom "Desktop" connectors that let Gemini interact with niche software such as CAD tools or video editors.
What happens next
Google is rolling the feature out today in English for US subscribers, and it will likely expand to other regions and languages later in 2026. Apple’s macOS 27.3, slated for release before the end of the year, will tighten permission prompts, making it easier to see when an AI agent is controlling an app. Keep an eye on future Gemini updates, which may add more granular controls, broader app support and integration with Google Workspace tools.
Conclusion
Google’s Gemini computer use bridges the gap between file access and actual app interaction on the Mac. By keeping the AI’s actions visible and under user control, it offers a practical way to automate tedious workflows without surrendering your screen. For power users with an Apple Silicon Mac, a modest AI subscription could translate into tangible productivity gains.
Frequently asked questions
What Mac models support Gemini computer use?
Only Apple Silicon Macs (M1, M2, etc.) running macOS Sequoia 15.0 or newer.
Do I need a paid Google AI plan?
Yes, the feature is limited to Google AI Plus, Pro or Ultra subscribers in the US.
How do I tell Gemini to use computer use?
Add the tag @Computer use to your prompt and describe the task.
Will Gemini take over my mouse cursor?
No, it works in an isolated background window with a live preview, leaving your screen free.
Is there a way to stop Gemini mid‑task?
Yes, you can stop the process with a single click in the Gemini interface.
