Hands-on with Ring’s new Smart Video Search

Ring’s new Smart Video Search lets you search through recorded footage to find everything from a lost cat to a red sweater. According to Ring, the AI-powered feature can locate specific events in your Ring video history based on natural language searches, such as “a black cat at night,” “a red truck in the rain,” and “a kid riding a bike yesterday.” Compared to Ring’s current smart notifications, which can alert you to doorbell rings, people, or packages, Smart Video Search opens up a whole new level of historical insight.

The feature is launching today in beta for users in the US with the Ring Protect Pro plan ($19.99 a month). At launch, Ring says it can search for queries related to animals, vehicles, packages, and people, which can be qualified by location, time, and weather — for example, “raccoons in the backyard last night.” It can also identify various actions, including jumping, running, playing, and riding.

Ring has long lagged behind competitors like Google Nest, Arlo, and Wyze by not offering alerts for pets and vehicles on its cameras. While the new search feature won’t send you real-time alerts for any of those activities, since it can only see in the past, it will at least make it possible to easily find those things in your recorded footage.

I’ve been playing with Smart Video Search for a few days, and it’s largely worked well. It showed me where my cat Smokey had been so I knew where to look for him, showed when the UPS truck rather than the FedEx truck had been at my door, and identified a possum in my backyard. But despite receiving several Amazon deliveries (which my Ring video doorbell alerted me to), the search pulled up my rabbit hutch when I typed in “Amazon truck.” I eventually got it to find the delivery events by searching for “Amazon delivery person,” where it appeared to identify them by their blue vest.

The search feature is opt in, and I could choose which of my Ring cameras it was enabled on. I had to agree to allow Ring to process the videos “securely” and to use my anonymous searches to improve the feature (in accordance with Ring’s privacy policy). The app then spent a few minutes analyzing my events, but I could start searching immediately via a new search box in the app’s history tab, accessed from the menu bar.

I have an indoor / outdoor cat, and I’m always wondering where he is during the day. (I’ve written about how I use security cameras as a digital cat flap.) Typing in “cat today” pulled up several videos. By default, they’re sorted by the most relevant, but I could also choose to sort by date, which puts the most recent activity first. This showed me Smokey had been in the backyard about 10 minutes ago.

The feature can search through your entire history — up to 180 days — with some limitations I get into below. I tried searching for possums and raccoons, as I have a lot of late-night backyard visitors. This was less successful than my cat search. It mostly surfaced videos of my cat and dog and a lot of squirrels. But a few blurry images of a possum did show up (from a Ring floodlight camera mounted up high — hence the blur).

In an ideal world, the Ring app would only notify me when I need to know about something — when my cat is in the backyard or when there’s a raccoon near my chicken coop — to cut down on notification fatigue. But I can see plenty of use for the search function. Beyond the obvious one of looking for specific things after an emergency or incident, I could see myself using it to check whether my son had his tennis bag with him when he left for school, for example.

Security cameras are also useful for finding lost things. Yes, I have scrolled back through doorbell footage to see if I was carrying my purse when I left the house or when I couldn’t find my red sweater. This feature will save me a lot of scrolling time.

Typing “red sweater front door” brought up several images on the Ring Doorbell showing various visitors wearing red. There’s no person recognition here, though, which is a feature offered by competitors. It just sees generic people. Adding “red sweater woman” did narrow the results down. It also serves up “possibly related” videos when it runs out of relevant matches.

The search uses what the company calls Ring IQ, which Eric Kuhn, GM of Ring experiences and subscriptions, explains is a combination of Ring AI technology and “Visual Language Modeling to match text to images and quickly deliver results.”

In an interview with The Verge, Kuhn said the feature searches live view and recorded motion events. It only analyzes video, not audio. When Ring’s 24/7 recording launches next month, it won’t search through all that footage; according to Kuhn, it will be limited to event-triggered recordings and live views. Kuhn also said there are guardrails that limit the use of derogatory terms or explicit language.

Maciek Tegi, principal product manager at Ring, explained to me how the search differs from Ring’s existing video intelligence, saying that the models for person and package detection are classifier-type models developed to determine whether there is a package or a person in the field of view. The AI behind Smart Video Search matches video against text. “We process the video and text separately using two models. We look at the text and create a numerical representation of the text, and then look at the frames in a video and create numerical representation for those as well, then match the numbers,” said Tegi.

Ring isn’t the first consumer security camera company to announce an AI-powered video search feature, but it is the first to offer it to all its users publicly. Smart Video Search is available starting today in a public beta, whereas Wyze’s AI video search feature is in a pilot program you need to request access for. In August, Google announced that an AI-powered search for its camera history would be coming to its Google Home Public Preview program, but it hasn’t rolled it out yet.

Ring announced new subscription plans earlier this month, effective November 5th. Smart Video Search will be part of the new Ring Home Premium subscription for $19.99 a month, which also includes the new 24/7 continuous recording feature.

Many Ring customers are upset that the new plans, while costing the same, have removed Ring’s professional home monitoring service for its Ring Alarm. (This feature is now an optional add-on for $10 a month.) Whether new features like Smart Video Search and 24/7 recording will be enough to encourage Ring users to stick with the company remains to be seen. But the search feature certainly feels like it will be useful, and I’m looking forward to testing continuous recording, which is something Ring users have wanted for a long time.