Android’s latest round of AI features improve accessibility, file sharing and more

If you’re an Android user, today is your lucky day; Google has announced a swath of new AI features for the entire ecosystem. Broadly speaking, the features make Android devices more accessible, but there’s something here for everyone.

For instance, one of the new enhancements, Expressive Captions, automatically generates subtitles that attempt to capture the emotion and intensity of what’s being said. So, let’s say you’re video chatting with a friend who groans after you make a lame dad joke. The feature will not only transcribe what they said, but it will also include “[groaning]” in the transcription. This works for other subtleties of human speech, too, such as when someone gasps or whispers something, and is compatible across Android, including streaming and social media apps. Per Google, Expressive Captions are available on Pixel 6 and newer Pixel phones, as well as “other compatible” Android devices.

Separately, Google has enhanced Android’s Image Q&A in Lookout feature. The latest version of the tool makes use of the company’s Gemini 1.5 Pro model to provide more helpful image descriptions. Image Q&A is primarily designed to assist blind and low-vision users, but in reality, anyone can use the feature to get Android to describe a picture in a natural-sounding voice.

Speaking of Gemini, Google is supercharging the AI agent with new extensions that provide better integration with some of the most popular Android apps. For instance, a new Spotify plugin allows Gemini to play your favorite songs for you and find playlists that suit your current mood. In the future, the company is promising tighter integration with Google Maps and even smart home devices that are linked to your Google account.

Additionally, Gemini now features the capability to remember things about you so that it can provide more personalized responses. For example, you can tell Gemini you’re a vegetarian, and the agent will remember that about you the next time you ask it to recommend a new dinner recipe. Google notes it has made it easy to view, edit and delete any personal information you’ve told Gemini to remember.

Another more practical update comes in the form of a Google Drive feature called auto-enhancements. The next time you upload a scanned document to the service, it will automatically edit the image to optimize the contrast and adjust the white balance, as well as remove any shadows and blurring.

And if you want to share a file with someone, Google has made that easier, too, with an improvement to Android’s Quick Share functionality. There’s a new feature that allows you to transfer pictures, videos and documents by displaying a QR code on your phone. Using this tool, you don’t need to add the recipient as a contact or fiddle with your Quick Share settings.

Lastly, if you’re a Pixel user, you can look forward to all of the above features and more. Most notably, there are improvements to the Pixel Screenshots app. For one, now you can save things you find with Circle to Search directly to the software. Google suggests this feature will be handy for holiday gift ideas. Pixel Screenshots will now also automatically categorize your snaps for you and provide suggestions, such as calendar invites and directions, based on the information you saved.

As with most Android updates, it can take some time for Google to roll out all the new features it announced today to every user, so be patient if you don’t see them on your device right away.