How AI is being used in images and video?

Image and video recognition, or computer vision, has become a broad category encompassing many different use cases

gear head artificial intelligence robot aumented reality collaboration

Thanks to user-generated content (UGC), our use of video and photography has dramatically increased, resulting in a more challenging task of tracking and categorizing all this unstructured data. Image and video recognition, or computer vision, has become a broad category encompassing many different use cases.

Helping you search

Imagine if you were looking for a particular type of shoe or piece of furniture, but you just couldn’t find the right words to express it in a text search box. As an alternative to typing in your search criteria, we are now getting accustomed to voice search via Alexa, Siri, Cortana, Google Home, etc. However, at the end of the prompt, the voice is really just fulfilling a text query.

Now, imagine if you could take a couple pictures of what you have in mind and send that to the search engine to help you find what you are looking for. Using image-based AI, the search would break out the different components of the image and allow you to choose which aspects of it are most important. As a woman, maybe you already own a nice dress and have a matching purse but want to find a pair of shoes that incorporate a little of both—does such a shoe exist? With an image-based search, you could find out quickly. Maybe it was a little black bow on the purse that you’d like to incorporate—the image AI picks up on the brand of the purse you have, not because you stated what it is, but because the database can recognize which brand it is, so it has an idea of the type of brand and budget you want in your new shoe. Quite a powerful result that goes beyond simple search today.

More targeted ads

Imagine going through some Instagram feeds of your favorite extreme athletes and then seeing ads featuring the shoes they wear. That subtle, subconscious recognition that you’ve seen those shoes before helps advertisers increase brand awareness and engagement. Startups in the sports marketing and social media world are using computer vision to provide more personalized ads to consumers while increasing the effectiveness of ad spending for sponsors.

In other cases, advertisers are able to customize their ads in real time to the audience. For example, if the advertiser makes several different products then it can adjust its ad to the viewers on the fly as they are browsing different types of content. Perhaps you are on a website looking at interior design ideas for the home and are on a page looking at interior design ideas for living rooms. A furniture maker trying to place the ad can promote its latest sofa in real time. The next visitor on the website might be looking at dining room ideas and the advertiser promotes cutlery that can match the dining room and style based on the image AI knowing that this person is looking at “wood, contemporary” designs.

Helping coaches with sports game reviews

An early stage startup is using computer vision to help sports coaches review playbacks of their games to help with coaching players. The manual process of reviewing videos can now be automated by allowing the AI service to jump through the entire game, identify specific players, and send them a custom video containing all the clips relevant to them. This is a game-changer, literally, for coaches around the world and their teams.

Easier home automation

Another startup is using AI and computer vision to help home automation finally become easier to use, in hopes of wider adoption. Home automation has existed for a long time since the days of X10, timed lights, and the promise of smart homes from several manufacturers, but it saw little in the form of mass adoption. Google Home, Alexa, and other devices have made lighting and home entry an easier project for homeowners. These smart assistants are allowing people to use their voice to simplify interaction with their home, such as lighting, heating, and even entry.

With AI, adding computer vision can take the spoken or app-based interaction out of the equation for even more ease of use. For example, the home could see you pulling into the garage and automatically turn up the heat, or turn it off as it senses you leaving. Taking this AI vision into businesses, the HVAC system could save companies more money as it sees more people entering a room or in a certain part of the office and only cooling those areas or rooms. And, as it senses (“sees”) more people entering a conference room, it could automatically turn up the cooling to offset the increased body heat.

Use of video and AI

Image and video-based AI has many use cases as you can see in just a handful of examples above. With video, it is the ability to extract information from a video, or to be able to insert content intelligently into a video. There are numerous additional examples that can be used in public safety or by government agencies as well.

Advertising in videos

With more product placement taking place, advertisers can insert their product into “placeholders” dynamically. Instead of being limited to a specific advertiser, the video content producer can leave areas in their video that easily incorporate an inserted image. Depending on the geography, language, and demographics of the viewer, AI can dynamically insert an ad into a video that has already been produced. Or, it can combine multiple videos on the fly depending on the audience. This enables a very powerful, localized, and personalized approach to providing more native-based advertising.

Videos on computer vision

Want to learn more? Here are a few videos I came across that can get you up to speed on image and video recognition:

Fei Fei Li, professor at Stanford University, in a TED Talk:

A University of Washington graduate, working on a darknet neuro-network:

This article is published as part of the IDG Contributor Network. Want to Join?