My favorite Google AI features from the Pixel 10 launch

The Made By Google event was not only a showcase of Google’s latest Pixel hardware, but a launchpad for many new AI features. I’m typically skeptical of the current generation of AI, but as I checked out the new software across various demo sessions, I found myself more and more intrigued. It seems like Google, along with Apple and Samsung, has been working on making these AI-powered updates more helpful in a way that might actually make our lives easier or simply more fun.

There wasn’t enough time to write up every single one of them, so I’ve put a few of my favorites in this story to give you a better sense of what to expect when the Pixel 10 series hits retail shelves later this month. Spoiler alert: Many of these have to do with voice and calls — an area Google has historically excelled at.

The Recorder app can generate backing music

I have long been enamored with Google’s Recorder app. It started with the on-device transcription that made getting quotes from my interviews easy and relatively secure. But when Apple introduced a multi-track recording function to its Voice Memos app, I quickly jumped ship. While the iOS recorder has inferior transcription in terms of accuracy and readability, the fact that I could basically record a duet with myself seriously appealed to the musical theater geek in me. I played both Elphaba and Glinda, crooning their parts from “For Good” into my iPhone.

But when Google’s senior director of product management for Pixel software Shenaz Zack told me the Pixel 10’s recorder app would add AI-generated music to your singing, I went silent in slight disbelief. I spent much of my youth ripping karaoke tracks from YouTube videos, looking up “minus one” or “backing tracks” or “instrumentals only” on various download platforms. My friends and I were aspiring performers, looking to mix our own covers of popular songs, and a tool that would generate backing music to our voice tracks would have been a dream come true. Honestly it kind of still is.

Zack walked me through the process twice — on my first try I sang a verse and part of the chorus of “Golden” from the Kpop Demon Hunters soundtrack. I giggled self-consciously at the end, before Zack hit stop. As it recorded, the app actually showed a tag that indicated it knew I was singing, and when we selected the recording after, a chip appeared saying “Create and add music.”

Tapping that brought up a panel titled “Choose a vibe to create music” with two sections: Featured vibes and Your vibes. Under the first one, the options were “Chill beats,” “Cozy,” “Dance party,” “Rainy day blues,” “Romantic” and “Surprise me.” On my second attempt, when I rushed through a rendition of the all-time banger “Mary Had a Little Lamb,” the app displayed a warning at the bottom that said “The beat might not match well if the recording is short.”

I chose Dance Party, hit next, and waited a minute or so while Recorder went to work. The animation at the top said the system was analyzing the audio, identifying the rhythm, locking onto the beat and harmonizing the track before delivering the result.

I don’t quite know what I was expecting, but I can say that those who were at all concerned about digital rights management have nothing to worry about. The music that Google generated for “Golden” sounded nothing like the original, and while it did make my voice sound less lonely and made for a more complete track, I felt like I needed a few more adjustments to feel satisfied with it. As for “Mary Had a Little Lamb,” the result was as generic as expected for an AI-generated soundtrack to a very basic nursery rhyme.

To Google’s credit, what came out seemed to be in the right key and rhythm, and I certainly will need much more time playing around with this to see if tweaking the settings will help. I also wanted to point out that the generated music also stopped as my singing stopped, so the giggling I mentioned earlier was not scored.

Although this feature did not live up to my (admittedly unrealistic) fantasy, I do think it’s a fun use of AI and seems harmless. It’s not going to be a mainstay of most people’s daily routines, although Zack did say that a large percent of people actually used Recorder for singing. This update could certainly make for a nice little dose of musical creativity.

Voice Translate made it sound like I was speaking German

I had more concerns around the Voice Translate feature that was supposed to make you or your caller sound like you were speaking in a different language. According to Google, the goal is to “break down language barriers during phone calls.” When I asked Zack why the company felt the need to make the voice resemble the caller’s, she said it was about personal connection.

Zack explained that her parents live in India, and though they speak English, they’re not very fluent. That makes for some difficulty when they call Zack’s kids. Simply adding a robotic voice that’s translating between the grandparents and the children wouldn’t feel right, either. I was initially skeptical that fully replacing the caller’s original voice with a translated version would help, but after a few demos, I am certainly swayed.

To be clear, the person placing the call has to do so from a Pixel phone for Voice Translate to work. Once you choose Voice Translate from the Call Assist submenu, you’ll have to choose a language. When the call is connected, the system will say to both parties that the “Call is translated by Google AI in each speaker’s voice. Audio is not saved.”

I tried this out a few times with a Google representative who spoke German, whom we will refer to as “Uncle Tim” to make it easier for me to describe this demo. Each time he spoke, I could hear a couple seconds of his voice in German, before a chime played and the version in the original language became softer. What sounded like a dubbed actor playing Uncle Tim came on and conversed in English, complete with realistic replications of pitch, rhythm and expression.

I also could hear feedback when I talked on the call, so I heard myself speaking German on the other end. It was truly strange, because it sort of did sound like me. One of my closest friends lives in Germany, and has had to put up with my attempts to learn German for more than 10 years. I immediately wanted to try Voice Translate on her to see if she would believe I had suddenly become fluent (but of course, I’d have to figure out how to get her to ignore the warnings that Google AI was at work).

I’ll be honest, the experience wasn’t perfect. Not only were the translations sometimes off (some of what Uncle Tim said in English didn’t make sense), the generated voices seemed less like a complete replication of the caller and more like a novice dubbing artist. That’s not a bad thing, since I was very concerned about impersonation being a problem.

To that end, Zack said Google was deliberate about the implementation. She reminded me of the “ducking” that was in place, which is when the original speech is still audible in the first few seconds and then softer throughout. Like the original audio is ducking below the dubbed voice — get it? And I remembered that while the AI voice might sound sort of like me, it isn’t designed to simply make up things I’m saying — it’s just translating the content. I’m the one that decides whether to go off and curse out a relative and have that conveyed in their native tongue, for example.

Of course, there may still be bugs and quirks to work out. I was amused by the various accents that came through in the English-speaking version of Uncle Tim. At first he sounded American, but in subsequent conversations he took on an Australian accent.

All this is powered by the Pixel 10’s Tensor G5 chip and processed on-device using “a new codec and semantic understanding,” according to Zack, to understand the speaker’s vocal expressions. For now, I see what Google is going for and cannot wait to call my friend in Frankfurt.

At launch, Voice Translate will support translating to or from English with Spanish, German, Japanese, French, Hindi, Italian, Portuguese, Swedish, Russian and Indonesian.

Google/Screenshot by Engadget

Magic Cue surfacing your flight info when you call your airline is helpful

The recorder app, translation and expressive-sounding AI are areas Google has long proven expertise in. And lest we forget, the company has also been a pioneer in suggesting actions from your emails and adding events to your calendar by scanning your inbox. With the Pixel 10’s Magic Cue feature, Google is basically bringing this functionality to your texts and calls.

While Magic Cue can helpfully show shortcuts within the Messages app to help you answer questions about reservations or send photos from recent trips, I’m most into one specific aspect. When you call an airline to make changes to a flight, for instance, the Pixel 10 can pull up your reservation information and display it within the call, so you won’t have to open your email, and search for the booking confirmation to have your reference number ready. Sure, it might only save you seconds, but it’s so much easier, and Google already does a version of this in your inbox.

I would love to see this particular feature expand and cover other types of appointments so you can quickly get codes or other identifying information during calls to, say, your plumber, doctor, insurance provider and more.

Camera and photo features continue to improve

Google continues to improve upon areas it’s led the way in, and photography remains a strength of Pixel phones. The company was one of the first major players to use its algorithmic prowess to dramatically improve the quality of low light photos and with the Pixel 10 Pro it again uses computational processing to deliver superior images.

Pro Res Zoom on the new phone did manage to produce some surprisingly clean pictures of faraway buildings, at least in my demo at Google’s Manhattan office. I was impressed by how clear the lines on the underside of a skyscraper that we zoomed to a 100x level on looked. Google was also careful to clarify that Pro Res Zoom won’t work on people, and that distant text may look odd. 

"We've tuned Pro Res Zoom to minimize hallucinations, however they may still occur — especially with faraway text. Additionally, when Pro Res Zoom detects a person in the scene, we use a different enhancement algorithm that prevents inaccurate representations," according to Google.

in those situations, the algorithm will drop to Super Res Zoom quality. Depending on which Pixel phone you’re using, Super Res Zoom delivers up to either 20x or 30x zoom.

In the results I saw, people standing on a deck at the top of a tower just seemed a bit pixelated compared to the building’s facade, and the effect wasn’t jarring or even really noticeable until I zoomed in. But that might be because they were a tiny part of the picture — I imagine things would look different if a person was the main subject in a scene.

As someone who enjoys composing pictures, I didn’t think the Camera Coach feature would do anything for me. But I was pleasantly surprised that I actually liked some of the AI’s proposed framing options. I still don’t think I’ll use this much in the real world, but it might help other people who want tips on photography.

I was initially nonplussed about the new Photos feature that lets you tell the AI how to edit your pictures, but after a brief demo I came around. Simply telling Gemini to “turn that red dress blue” or “get rid of the people in the background” was not only easier, but suprrisingly effective. I also want to point out that Google also made tweaks to the Guided Frame feature in its camera app that helps those who are blind or visually impaired know what is in the scene. It now uses Gemini models, which should help with object recognition.

Finally, it’s worth calling out the support for C2PA content authenticity initiative. Google is building this into the Photos app, where metadata will show whether or not AI was used in a picture. The Pixel 10 phones will be the first to implement the new industry-standard Content Credentials (CR) within its native camera app, and companies like Adobe, Amazon, Google, Meta, Microsoft, OpenAI are all part of the initiative. 

An assortment of other updates worthy of mention

Those were just a slice of the new AI-related features I was impressed by at my recent demos ahead of Google’s event this week. But there are quite a few more I found promising, like visual overlays in Gemini Live and the new Pixel Journal app. I didn’t spend as much time with either, but they worked in my brief demos. So did the “take a message” feature that will send transcriptions of voicemails to you, which seems like a much better way to be alerted to a missed call than a hidden section of the Phone app.

I’m not yet sold on the Daily Hub, which is basically an updated version of the existing pages that sit to the left of the home page showing relevant actions and articles you might want to explore. I’m fairly intentional when it comes to looking for things to consume, and have specific apps I prefer for doomscrolling (Reddit over everything), so I’m not sure Daily Hub will suit me.

Still, the fact that I liked the bulk of the new AI features coming to the Pixel 10 series is pretty significant. Of course, I will still reserve judgement until I can spend more time with them in the real world, and hope to write reviews of some of them. But it’s clear from my time with demos of the Pixel 10 that Google has been pretty thoughtful about how it imbues its hardware with AI, and I hope its competitors take notes.

This article originally appeared on Engadget at https://www.engadget.com/ai/my-favorite-google-ai-features-from-the-pixel-10-launch-130023979.html?src=rss https://www.engadget.com/ai/my-favorite-google-ai-features-from-the-pixel-10-launch-130023979.html?src=rss