Using AI to automate eSports highlight creation

Hearthstone wallpaper
Neal Romanek
Post Production
September 19th 2017 at 2:26PM : By Neal Romanek

Elastic Media is using AI to allow gamers to easily and quickly publish their own highlight reels online

EVE, the Elastic Video Engine, is an AI developed by UK-Israeli startup Elastic Media. The platform uses AI and deep learning to 'watch' live streams and broadcasts and automatically separate them into segments.

The clips are tagged with metadata that can be used to filter the level of action, enabling content producers to tailor what they offer viewers, saving substantial hours of work for editors and costs for broadcasters.

The EVE AI has been aimed at broadcasters, particularly for sports and news, but recently the Elastic team, lead by CEO and founder Ronen Shoval, has looked at the benefits the platform has for eSports.

We talked to Elastic Media’s VP of business development, Gavin Goodvach:

 

How does EVE work?

We use artificial intelligence that can understand where the semantic units are in a piece of content. We can teach the computer how to isolate those semantic units, and we use a bunch of different filters for that. In for example, a sports broadcast, different camera angles mean different things. Different in-buffers and out-buffers mean different things. We can even teach the AI to understand the inflections of voice.

What applications does the EVE AI technology have?

We have 19 patents pending on our technology. One of them is what we call Live Catch-up. Say you come in 30 minutes late to a live football match. How do you catch up on the highlights of what’s happened without missing the rest of the game? Live Catch-up let’s you press a button on the bottom of the screen; the live game continues to play and you can swipe through the highlights of the game, which have been selected automatically by the EVE AI.

How did you develop the AI technology for gaming?

We are developing deals with CNN and Bloomberg and other big broadcasters, but we thought: What other verticals where we can go to market quickly. And that’s when we started looking at eSports. Esports is probably the fastest growing content vertical there is. If you’re a good player in the United States, you can get a full ride scholarship at some universities.

So we took a look at what their pain point is and it’s very clear. A gamer will play six, seven, eight hours of gameplay a day. At the end of the day they have eight hours of video and nothing they can do with it. Who’s going to edit an eight hour video and who’s going to watch it? Also, there are 17 thousand registered streamers on Twitch and they are all generating these huge video assets that they can’t do anything with.

One thing I’ve learned in the last six months is the key to any product succeeding in this space is to have contact with the community

 

So we started with Hearthstone, which is one of the top ten games. And we taught the computer to automatically edit Hearthstone. We call it a “clickbot” – it’s the first clickbot in the world. What it does is allow the streamer to take the URL for the video of his gameplay, stick it into our website and at five times the speed we process that video and cut it up into individual games with metadata attached – how long the game was, what characters were play – so at the end of the day the player has ten or twenty assets that he can use to monetise or publicise his brand. And all this metadata is created solely by analysing the video.

The next level will be to create an online destination where anyone who wants to play a certain level of a certain game can search any part of a game via our metadata. And the response we’ve had from the gaming community has been surprisingly overwhelming.

What are Elastic's plans for the technology around eSports?

We plan to scale horizontally. We’ll have bots for League Of Legends and other games, and we’ll try to get as many games on the platform as possible. And we’re going to develop more and more metadata on each of the games.

We talk to the gamers and players and we ask them: What do you need? What do you want? What are the important elements of metadata that we can add? For Hearthstone there are specific types of metadata that are useful for the gamers, where on, say, League Of Legends it could be a very different kind of thing.

One thing I’ve learned in the last six months is the key to any product succeeding in this space is to have contact with the community. The most important thing is that we enable that conversation. And we enable it through social media, we enable it through our portal which has places where the community can feedback to us.

 

This article is part of our 2017 eSports Special. To read the entire eSports special in online magazine, click here