Vibe Coding XR: Google’s New Workflow for Building XR Apps with AI in Under a Minute

I’ve been playing with XR prototyping tools for years, and the friction has always been the same: you need to stitch together perception pipelines, game engines, and sensor integrations before you can even test a simple idea. It’s a lot of setup for something you might throw away. Google Research’s new Vibe Coding XR workflow aims to change that by letting you describe what you want in plain English and get a working, physics-aware XR app in under a minute.

The idea is straightforward: combine Gemini Canvas (their LLM-powered coding environment) with the open-source XR Blocks framework. You type or speak a prompt like “create a beautiful dandelion,” and Gemini handles the spatial logic, scene configuration, and interaction setup automatically. The result is a WebXR app that runs on Android XR headsets like the Galaxy XR, complete with hand tracking and physics. If you don’t have a headset handy, you can test the same app in a simulated reality environment on desktop Chrome.

This isn’t just a demo gimmick. The team behind it — led by Ruofei Du and Benjamin Hersh — has been iterating on the workflow for over a year. The key insight is that vibe coding, where LLMs translate human intent directly into code, works well for 2D and 3D web development but has lagged in XR because of the extra complexity. By feeding Gemini curated code templates and specialized system prompts, they’ve made it handle spatial reasoning without the user needing to know anything about XR APIs or depth sensing.

One thing I appreciate is the rapid iteration loop. After Gemini generates the app, you can pinch a button on the headset to enter XR mode and see the result instantly. If the dandelion doesn’t blow away the way you imagined, you tweak the prompt and regenerate. The “Share” button also creates a public link, which is handy for sending prototypes to collaborators without asking them to install anything.

The technical report goes deeper into how they handle physics and hand interactions, but the practical takeaway is this: Vibe Coding XR lowers the barrier for experienced developers to test spatial UIs, 3D interactions, and educational experiences without spending days on scaffolding. It’s not going to replace full game engines for production work, but for rapid prototyping it’s a solid step forward.

They’re demoing this at ACM CHI 2026, and the live demo is already available online. If you’ve ever wanted to build an XR app but got bogged down by the setup, this is worth a look.