In an announcement about a set of new discovery tools, Amazon-owned video game streaming site Twitch also unveiled that it recently acquired Palo Alto-based video indexing platform ClipMine. The startup’s technology is now being put to use to translate visual information in videos – like objects, text, logos and scenes – into metadata that can help people more easily find the streams they want to watch.
Launched back in 2015, ClipMine had originally introduced a platform designed for crowdsourced tagging and annotations. The idea then was to offer a technology that could sit over top videos on the web – like those on YouTube, Vimeo or DailyMotion – that allowed users to add their own annotations. This, in turn, would help other viewers find the part of the video they wanted to watch, while also helping video publishers learn more about which sections were getting clicked on the most.
The company later pivoted to focus on the e-sports industry, with tools that could extract information from game videos like player names, game type, number of gaming sessions per stream, and more. It also shifted away from the idea of crowdsourcing to take advantage of other technologies, like computer vision and machine learning.
Twitch’s interest in the company’s deep learning-based video indexing platform was due to its ability to analyze video content – like gamer’s streams – to identify what’s taking place in those streams, who’s playing and other variables.
It has now put this technology to use with the launch of new stream discovery tools for Blizzard Entertainment’s “Overwatch” and “Hearthstone.”
In the directories for those game streams, viewers are able to filter channels by a number of factors, including by hero on “Overwatch” streams, by game mode, player rank, number of wins, and by hero class on “Hearthstone” streams.
This is not the first time Twitch has introduced metadata filtering for game streams on its site. In April, the company added a feature that allowed viewers to filter “League of Legends” streams by player rank and champions played. This proved to be a fairly popular addition, the company says.
“When we recently introduced metadata filtering options for one of our top e-sports games, we not only received positive reception from its fans, but demand from the greater community to have access to similar discovery tools for other titles,” said JT Gleason, Director, Integration Success, at Twitch, in a statement.
Bringing a similar filtering option to “Overwatch” and “Hearthstone” made sense because they were two of Twitch’s most popular competitive games, and players often watch others in order to help improve their own skills.
“Our new metadata filters now make it easy to find more granular aspects of gameplay that previously required a lot more searching,” Gleason added.
Traditionally, online video content is indexed much like the web – using metadata like titles, tags, descriptions, and captions. But Twitch’s streams are live, and don’t have as much metadata to index. That’s where a technology like ClipMine can help. Streamers don’t have to do anything differently than usual to have their videos indexed, instead, ClipMine will analyze and categorize the content in real-time.
“Video game streams have a very rich structure that has been difficult to exploit for the purpose of enabling content discovery,” said Zia Syed, an ex-Goolger and ClipMine’s founder, who’s now director of engineering at Twitch. “By employing computer vision and machine learning developed at ClipMine, we have been able to recreate that structure in a reliable, scalable and cost-effective way. This enables us to match creators and viewers in a very precise manner opening up new ways of content discovery on Twitch,” he added.
According to Syed’s LinkedIn, he joined Twitch in November of last year, so Twitch’s characterization of the ClipMine acquisition as “recent” is a slight stretch.
The new filtering feature began rolling out on Thursday. Twitch hints that more discovery tools are in the works, noting in a blog post that users should “keep your eyes peeled.”
ClipMine was backed by $2.6 million funding, says Crunchbase. Investors included Google board member Ram Shriram, former Facebook vice president Greg Badros, entrepreneur Amarjit Gill and Cadence CEO Lip-Bu Tam.