This is the second part in my three part series on AWS Deep Racer. If you’ve arrived here directly the first part in the series can be viewed here, and mainly covers setting up AWS Deep Racer in a single AWS account using multi-user mode. I’d recommend you read that first, then return to read about the virtual racing.
Once your participants have access to AWS Deep Racer you’re going to want to create a race for them to enter to ramp up the gamification side. To host a virtual race you’ll need someone with the administrator privileges to go into the Deep Racer Console and navigate to ‘Racing League’ and then ‘Community Races’ on the left hand side. You should then have the option to ‘Create Race’ on the right hand side, this will then present you with various options for your virtual race format.
Out of the options available I would recommend thinking about whether this is part of a series, is going to include an in person race, and the level of previous experience your racers have with Deep Racer. For example our virtual race was designed for the majority of participants who’d never used Deep Racer before, and was a ‘qualifying round’ for an in person finals. We chose to run a ‘Classic race’ because people could set times and then refine models to set their fastest possible lap, giving them a chance of having some really good models for the in person event. If your race was a one off with no follow up you might prefer a ‘Live Race’, allowing you to arrange a video conference and all watching the race together in a ‘shoot out’. We chose ‘time trial’ as the simplest race for new competitors, and because it aligned with our plans for the in person race, but if you’re running an event for people with significant experience you might want to choose an alternative. You can then choose the time period your virtual event runs, although it’s worth noting people can’t pre-join the race ahead of opening time, they’re only allowed to join once the race time period is open.
Choose your track, which if you’re planning to run a follow up in person event should be the same track as will be used there, and then choose how you’ll rank times, number of laps, and the penalty for going off track. Once you’ve created it you should have a hyperlink you can use to share with your participants to join your virtual race.
I would strongly recommend running an event kick-off. For newbies an introduction to Machine Learning and reinforcement learning is essential to help understand the concepts, and once they’re put in simple terms with some analogies the high level concepts of reinforcement learning are soon understood. Our participants had a great overview from AWS on Machine Learning in general, reinforcement learning, reward functions and hyperparameters within Deep Racer. As a dog owner I think this was my favourite slide.
AWS Deep Racer model behaviour is influenced by a reward function and hyper-parameters.
The reward function is written in python, examples of which can be found here. Reward functions can take a number of input parameters from the car, they can be found here. They include parameters like the current speed of the vehicle, it’s position on the track, the amount of progress made around the track and more. These can be supplemented by importing python modules, for example time, math and others might be useful for more complex models. I’ve seen some extremely complicated models do poorly, and some extremely simple (in terms of number of lines of code) do extremely well, and vice versa! The important thing to note is you’re coding something to reward an outcome, rather than coding a precise action, this way the model is able to solve a complex problem like driving, without thousands of lines of code.
Hyperparameters influence how the models train. This post isn’t supposed to teach you the inner workings of machine learning, and I’m not a PhD mathematician, so if you want to really understand the inner workings I’d recommend taking a course. An AWS focussed example would be the A Cloud Guru AWS Machine Learning Speciality course, which because the certification curriculum covers machine learning theory as well as AWS service implementation gives a great overview. Generally speaking you’re tuning these hyperparameters to either favour exploration (in our case explore the track, try and find a way to maximise the reward), or exploitation (refine and make incremental gains on a path around the track that is working). There are instructions in the ‘info’ section of the page where you configure them.
It’s important to communicate what car and track you want participants to train with, particularly if you want to follow up with an in person race. As the car is learning a track during the training it won’t perform well if you move it to another track for the race. Also it’d be no good training with a dual camera / lidar car, if your race car is single camera / no lidar.
It’s also important to communicate to people how to enter a model into your virtual race. Although you will have provided a hyperlink to participants as I described earlier which they can use to register against, they’ll still need to submit their individual models in to your race. Unfortunately at the present time the Deep Racer console doesn’t provide the most intuitive way to do that, in two different places! First when creating the model itself users have the opportunity to ‘Automatically Submit to the Deep Racer League’, this is the first place participants can submit their model to the wrong race, you’ll want them to deselect that option, otherwise they’re entering a public race most likely on a completely different track!
The second place participants can enter the wrong race is after the model has trained. There’s a nice button entitled ‘Submit to virtual race’, unfortunately once again that takes the racer down a path of entering a public race most likely on a completely different track!
To ensure participants enter your race it is best to advise them, once they’ve registered via the hyperlink you provide, to navigate to Racing League -> Community races and then scroll down to find the correct race.
the screen should then show the event with a drop down list for them to select the appropriate model.
It’s important during a virtual race to think about how to communicate and engage with participants. As we’re Office 365 users I arranged a Microsoft Team with channels for announcements and Q&A, so participants had somewhere to go for reference information or recordings of calls we’d done like the kick-off event. I kept posting updates to keep up interest. Number of days to go, along with a summary of the day and a video of the current leading lap, which can be downloaded from the leader board screen (right click and save when the video is running) were my frequent posts. I arranged a number of drop in ‘surgeries’ at various points for the duration of our virtual race so participants could come along for help on getting started, troubleshooting issues or advice on how to tune their models. It was during these surgeries that the issues I’ve describe above like submitting a model to the public race came to light. In addition people had issues with permissions after about one hour, to do with session timeout. Logging out and back in via the ‘Your racer profile’ or completely out of the AWS Console usually did the trick, although one unlucky participant lost a complicated reward function they were authoring in the console at the time, so if you’re creating something complex it may be worth creating it outside the Deep Racer reward function screen and pasting it in.
Overall our virtual race was a success and has set us up nicely for the in person event. We had over 1,100 hours of training across 85 participants and we had great feedback that participants learnt something and had fun whilst doing it. Three people went sub 9 seconds, with 15 people going sub 10 seconds. Only 2.2 seconds separated our top 30 qualifiers who have been invited to our in person event. Our fastest qualifier came in at a rapid 8.394 seconds, video of their lap below!
We now progress onto the physical race, and my write up on how to set that up and and how it went will conclude this Deep Racer trilogy of blog posts… The third part of this series can be viewed here.
