Apple’s AI-powered “Audio Intelligence” technology for the Apple Watch has entered beta testing through the iOS 27.3 update, following its announcement at the company’s “Surprise and Shine” hardware event in September. This suite of four opt-in features empowers Apple Watch Series 12 wearers to detect important sounds, identify songs, catch up on previous conversations, and take notes.
The sound recognition feature is designed to assist deaf and hard-of-hearing users by alerting them to critical sounds such as sirens, alarms, or doorbells. Additionally, the song recognition feature utilizes Apple’s Shazam app to detect background music and identify the song title and artist.
However, the most discussed aspects involve ambient listening technologies: Live Rewind and Siri Recaps. In response to the backlash against AI recording devices like Meta’s AI glasses, Apple aims to balance utility with privacy for its users.
Live Rewind enables users to go back in time to review what was just said. For instance, if someone provides verbal directions or reads a menu, double-pressing the digital crown displays a text snippet of the last 15 seconds on the watch face. This snippet will vanish permanently after 30 seconds unless saved to the standalone Siri app, which can be useful for remembering names or jotting down quick notes. Users can also tap an “Ask Siri” option to seek clarification on specific terms mentioned.
When activated, the Watch plays an audible chime and displays an animation. This combination of gesture, sound, and visual display serves as a signal to those nearby that a recording is in progress.
The Siri Recaps feature extends this ambient listening by generating high-level summaries of conversations. Crucially, Apple notes that the feature does not identify speakers, save raw audio, or transcribe conversations word-for-word. Instead, it creates summaries with bullet points focusing on key details like dates, deadlines, and pricing. This mimics natural note-taking by stripping away emotion and tone, making it useful for work meetings or parent-teacher conferences, provided the user is not wearing noise-canceling headphones or listening to music.
Users are prompted to review their saved Siri Recaps; otherwise, they are automatically deleted within seven days. A morning unlock prompt also reminds users of existing recaps. These notes can be saved to the Notes app or exported to the Journal app. Users can enable the feature manually or schedule it, such as restricting recapping to work hours, and can disable it instantly via the Control Center.
Apple emphasizes that Siri Recaps is designed to omit sensitive information, including government IDs, financial data, authentication details, and self-harm language. The technology relies on the S11 chip, microphone, and Apple Intelligence models running on the iPhone, alongside Private Cloud Compute.
The process begins with the S11 chip on the Watch detecting a conversation. Audio is processed through a secure enclave on the Watch, transferring condensed text to the paired iPhone. The iPhone then encrypts this text and sends it to Private Cloud Compute for summarization, which is deleted immediately after use. Recaps remain encrypted when synced to other devices and are inaccessible to Apple.
Similarly, Live Rewind uses this secure pipeline to create 15-second text snippets.
Apple asserts that these features will not negatively impact battery life. They are currently available to developers via the iOS 27.3 update and will later be rolled out to the public beta before a full release.
The Editorial Staff at AIChief is a team of professional content writers with extensive experience in AI and marketing. Founded in 2025, AIChief has quickly grown into the largest free AI resource hub in the industry.