Naturally, you can work in a mixed reality environment with a connected Bluetooth keyboard and mouse, and you can place yourself in an immersive environment if you want to focus, or leave transparent mode on to make sure your coworkers aren’t snapping photos and laughing. while wearing ridiculous headphones to get things done. It was unclear whether it would be possible to connect the headset to a laptop to bring your work into mixed reality, a feature available in Apple Vision Pro.
Gemini in XR
A tap on the side of the earbuds opens an app launcher, and this is where you can activate the Gemini if you want it to stay persistently “on.” Once turned on, there is an icon at the top of the virtual space to let you know that everything you say and look at is being recorded by Gemini.
In transparent mode, you can approach an object and ask Gemini about it: a Google employee who was demoing the headphones (before I tried them) approached another person wearing an FC Barcelona shirt and He asked Gemini to find the “classification of this object.” team.” Gemini quickly registered the team name and got search results with league standings and recent match scores.
You can ask Gemini anything like this and it will respond with visual results displayed on the headset. I asked him to “take me to Peru” and he opened a 3D version of Google Maps. I was able to move around and focus on Lima, and in cities where Maps already has many 3D models, you can explore areas in greater detail. You can still talk to Gemini in these experiences, so I asked him questions like when would be the best time to visit and got a prompt response.
In another example, I took a look inside a restaurant in New York City for a virtual tour of the space. Google says it can use AI to stitch together images of the interior of a place and display them to make it look like you’re there. He did a pretty good job and I asked Gemini if the place takes reservations, without having to specifically say the name, because I was looking at the name of the restaurant. He does I accept reservations, but Gemini couldn’t make one for me. (That integration could come later.)
Next, I watched some videos on YouTube, where the 2D content looks sharp and colorful. The stereoscopic content was even better; My senses felt surrounded. I saw some hikers walking along a trail and I asked Gemini where all this was and he said, “New Zealand.” I couldn’t verify that, but it seemed like the right answer. I saw more spatial playback of 2D videos as the virtual player added depth and layers to make them feel 3D. I jumped into the Google TV app and enabled “Cinema mode” to start a virtual theater to watch movies and shows, like on other VR devices.
Circle to Search, the feature Google introduced earlier this year on Android phones, is also available on Android XR. Simply walk up to a physical object near you, press the top button on the headset, and then pinch and draw a circle around what you want to know more about. You will get a Google search page with results.
smart glasses
Project Moohan feels like Google and Samsung are playing catch-up with the rest of the VR market, although the Gemini integration gives their efforts a unique layer. However, I have to admit that I was much more excited to try the smart glasses, where Gemini feels they could be even more useful. They did not disappoint. I walked into another room and there were several pairs of glasses in front of me. Some were sunglasses, others had clear lenses. Like headphones, you can charge them with your prescription. Google did not provide a name for the glasses prototypes.