“Mariner is our exploration, at this point a research prototype, of how the user interface is reimagined with AI,” says Hassabis.
Google launched Gemini in December 2023 as part of an effort to catch up with OpenAI, the startup behind the popular chatbot ChatGPT. Despite having invested heavily in AI and contributing to key research advances, Google saw OpenAI praised as the new leader in AI and its chatbot even touted as perhaps a better way to search the web. With its Gemini models, Google now offers a chatbot as capable as ChatGPT. It has also added generative AI to search and other products.
When Hassabis first revealed Gemini in December 2023, he told WIRED that the way it had been trained to understand audio and video would eventually prove transformative.
Google today also offered a glimpse of how this could happen with a new version of an experimental project called Astra. This allows Gemini 2 to make sense of its surroundings, seen through the camera of a smartphone or other device, and converse naturally with a human voice about what it sees.
WIRED tested Gemini 2 at Google DeepMind’s offices and found it to be an impressive new type of personal assistant. In a room decorated to look like a bar, Gemini 2 quickly evaluated several bottles of wine on view, providing geographic information, details of taste characteristics and prices obtained from the web.
“One of the things I want Astra to do is be the ultimate recommendation system,” says Hassabis. “It could be very exciting. There may be connections between the books you like to read and the food you like to eat. “There probably are and we just haven’t discovered them.”
Through Astra, Gemini 2 can not only search the web for relevant information about a user’s environment and use Google Lens and Maps. It can also remember what you’ve seen and heard (although Google says users could delete data), providing the ability to learn about a user’s tastes and interests.
In a simulated gallery, Gemini 2 offered a wealth of historical information about the paintings on the walls. The model quickly read several books as WIRED flipped through the pages, instantly translating poetry from Spanish to English and describing recurring themes.
“There are obvious business model opportunities, for advertising or recommendations,” says Hassabis when asked if companies could pay for Astra to highlight their products.
Although the demos were carefully selected and the Gemini 2 will inevitably make mistakes in real-world use, the model withstood efforts to trip it up reasonably well. He adapted to the interruptions and, like WIRED, suddenly changed his view of the phone, improvising as a person would.
At one point, his correspondent showed Gemini 2 an iPhone and said it had been stolen. Gemini 2 said that it was wrong to steal and that the phone had to be returned. However, when pressed, he agreed that it would be fine to use the device to make an emergency phone call.
Hassabis acknowledges that bringing AI to the physical world could lead to unexpected behaviors. “I think we need to learn how people are going to use these systems,” he says. “What is it useful for; but also the privacy and security aspect, we have to think about that very seriously from the beginning.”