On Tuesday, Apple rolled out a potentially controversial set of Audio Intelligence features into testing with the first developer beta versions of iOS 27.3 and watchOS 27.3. The betas bring four new AI features that allow Apple Watch Series 12 wearers to detect important sounds, identify songs, catch up on conversations, and take notes. After just a day with Apple’s Siri Recaps feature on my Apple Watch, I can already see how its value may ultimately outweigh any creepiness factor that comes from the AI-powered audio recording and summarization it provides.
However, I also see areas where AI-assisted note-taking can go too far, capturing things that aren’t worth remembering, and where there’s room for improvement. The first two, sound and song detection, don’t represent potential privacy concerns; they are designed to detect either important sounds — like an alarm or doorbell — or identify a song using Shazam’s technology. The former helps deaf and hard-of-hearing users navigate the world, and the latter is something we’ve already done with our iPhone for nearly two decades now.
It’s the two other features, Live Rewind and Siri Recaps, that have raised concerns. The features record and then transcribe or summarize the conversations happening with and around the Apple Watch’s owner. Live Rewind lets you go back to see what was said in the past 15 seconds, while Siri Recaps summarize audio throughout your day, saving notes of the important things that were said.
Apple has taken care to ensure the privacy of these AI features by processing the audio and running AI models on-device, when possible, and by not identifying speakers, storing raw audio files, and redacting private information, like government ID numbers or personal account information. Still, some are put off by the idea that we’re now normalizing the idea that technology is always listening. I find myself sympathizing with both camps.
I often lament that we’ve allowed technology to become so pervasive in our daily lives, but I still see the value in what Apple has built. I haven’t yet had an appropriate time to test Live Rewind, nor do I think this will be a feature I use as often. I’m more likely to say “sorry, what?” or “can you repeat that?” if I missed something that was said, before turning to my Apple Watch for help.
But with Siri Recaps, I immediately saw how this could become a feature I rely on regularly to save important notes for later. I was able to test the feature in a real-world scenario yesterday, when we were dropping off my teen’s car for repairs and picking up a rental. At both places, employees shared a lot of information about time frames, prices, and policies that were delivered verbally.
Extract — continue reading at the source.
