Ubisoft
Article about Ubisoft

Published 2 weeks, 1 day ago

Games Ubisoft Published by J. Doe

Ubisoft La Forge – Pushing State-Of-The-Art AI In Games To Create The Next Generation Of NPCs

Ubisoft La Forge is an open research and development initiative that brings together scholars and Ubisoft experts with the driving objective to bridge the gap between academic research and videogame innovations. Experimenting with the latest technologies and techniques in videogame production, they are at the forefront of the academic world, with dedicated teams investigating uses for the latest technology, such as artificial intelligence, to make games more realistic, fun, and efficient to develop.

To unravel some of its mysteries and find out how it helps create more realistic NPCs, help them navigate complex game worlds, and create more human-like reactions, we spoke with Joshua Romoff, a data scientist at Ubisoft La Forge and a Montreal native, who took his love of videogames and turned it into a data science PhD. Now researching the different applications of machine learning in games, he recently gave a talk at the Artificial Intelligence and Interactive Digital Entertainment (AIIDE) 2021 conference to present the breakthrough he and his team have been working on to improve NPCs’ pathfinding and navigation using machine learning.

What is deep reinforcement learning, and how does it work?

I like to think of the player as kind of an extension of a gamepad, and every input players put through the gamepad results in an action. Let’s focus on the reinforcement learning part: It’s the idea that you’re trying to reinforce some kind of behavior, similar to the classic Pavlov’s dogs experiment where a researcher rings a bell at the dogs’ feeding time and the dogs learn to associate the bell with a reward. You’re trying to encourage or discourage certain kinds of outcomes with rewards and penalties. My job is to design the tests and define when we give or take away rewards, and the goal of the AI is to get the highest score it can with the actions available.

Are there any games that particularly inspired you in your study of deep reinforcement learning?

I’ve always been into open-world games where you get to run around and interact with NPCs, stuff like Far Cry, for example. One thing that always stands out to me in those kinds of games is how players interact with the AIs of the NPCs, and it’s a core factor of the experience to me. You’re interacting with the world, and the NPCs are a big part of that interaction. I always liked messing with NPCs and trying to bug out the AI as a kind of challenge, seeing how I can manipulate them.

As an R&D scientist, what’s your day-to-day like?

Another big part of my role is working with master’s and PhD students who are pursuing their degrees. All our students are paid, but I work with them and their professors to define projects for them, and we’ll usually have a bunch of student projects going at the same time, which helps the students, but also helps push what we’re doing forward. Once we have a working prototype, we put the tech inside a sandbox environment, which is basically a simplified version of an actual game engine, and we can see the results of the work we’ve been doing. If a project works out, it’s a chance for the students’ work to appear in the games we develop and for them to get some experience of what it’s like to work on games, so we always try to make sure that the projects we’re working on result in something that game teams can use in their productions.

In your AIIDE talk, you went over how you did some tests in games like Hyper Scape to create more “player-like” bots. Can you talk us through it? The thing that is really cool about Hyper Scape is that the 3D environment is quite complex to navigate and has a lot of verticality to it. Players have a lot of tools available to them as well, things like jump pads that propel you straight up in the air, and double jumps, and you can use those to navigate to the tops of buildings. You can combine those things, too, so it’s really interesting for a game developer or tester to know that the map they have created allows players to navigate the whole thing.

Traditionally, games use what’s called a navmesh, kind of a 2D map of all the traversable areas in a world, and that data allows bots to define where they go and how they get there. But it was really hard to do tests with that method, because when you have all these crazy actions like jump pads and double jumps, plus vertical floors which aren’t always connected by stairs or ramps, the combinations make the possibilities explode in number.

We understand you saw some interesting results in some of your tests with other games. Can you tell us about those?

JR: One example was a bot we trained in For Honor, actually. We wanted the agent to defend more, so we gave it a bonus reward for doing that. It’s really funny, because one of the main challenges of training agents with this process is that, whatever setup you give it and action you’re trying to incentivize, it’s in theory going to learn how to do that as best as it possibly can. That wouldn’t be fun, so you want to incentivize other types of behaviors, like defending, that add some variability to its actions.

The other reason you might give these little bonus rewards is because it can speed up the training process, so it’s easy to just give it a little bonus here for defending and there for attacking – but it’s not obvious how all of these bonuses will combine, and you can end up with these really funny behaviors.

Are those kinds of results still valuable in the process?

If it’s to test the game, as our experiments were, these results are very useful, because you’ll see what the optimal behavior is based on the rewards you give. You might notice these things it’s learning and figure out that the behavior is actually helping the agent achieve its goal, which could point to something you weren’t aware of, allowing you to debug and know if your code is working as expected.

Original article

Jan 11, 2022 at 08:36

MEDIA CONTENT

Image related to: Ubisoft La Forge – Pushing State-Of-The-Art AI In Games To Create The Next Generation Of NPCs.

AMOUNT OF IMAGES

3 items

COPYRIGHT

Ubisoft

MORE ARTICLES