I’m Mark Ross, Lead Cloud Architect at Atos, specialising in AWS. I’ve been an AWS Ambassador since 2021 and joined the AWS Community Builder program in 2022. I’ve set-up an AWS focussed community interested in training, certification and working with AWS technologies with our customers, our Atos AWS Coaching Hub. Our most recent event is an AWS Deep Racer competition, to allow colleagues to get hands on and learn some reinforcement learning in a fun and gamified way.
AWS Deep Racer provides the opportunity to train a reinforcement model in the AWS Cloud to race around a track. You develop a model based on a reward function, which encourages the car to undertake behaviour that gives it a greater reward, and discourages it from undertaking behaviour that will give it less / no reward. You’re not defining the output, but more the guiding principles the model should use to explore with. So reward for keeping on the track, going faster, not turning too sharply might all be useful strategies. I’m sure anyone with a pet or a child will relate I’m sure! The other major area to influence the model is the hyperparameters. Hyperparameters influence how quickly a model will train, which is in contention with how accurate the model may be, as there’s is often a trade-off between optimisation and exploration. Model optimisation is not dissimilar to how the ‘lean’ principles work, you might make incremental gains to optimise an existing process, but it’s not going to lead to an amazing new discover which might radically alter your ways of working or industry.
In order to host your own Deep Racer event you’re going to need to provide an environment for people to use. Fortunately capabilities for event hosting have been greatly improved with the release of AWS Deep Racer Multi-User Mode, which eliminates the need to issue individual AWS Accounts to people and all the headache that provides with set-up and close down afterwards, as even within an AWS Organization the limit on the % of accounts that can be closed in a month is a pain.
To make use of multi-user mode only a single AWS Account is required. Within that account each user requires the ability to access that account, with appropriate permissions. Options include an IAM user within that account, or if you’re operating this within an enterprise environment you could assume a role within the account, for example via using AWS Identity Centre (formerly known as Single Sign-On). Permissions required to use AWS Deep Racer are provided via an AWS managed policy (AWSDeepRacerDefaultMultiUserAccess), but if you’re using IAM users you’ll probably also want to add another AWS managed policy to allow users to manage their passwords (IAMUserChangePassword). As our event is a hybrid event with a round of virtual use, followed by a live in-person event, I also wanted to give people an easy way to provide their models to us ahead of time, to make it easier on the day to schedule everyone’s laps. To that end I created another policy that granted permissions to a shared S3 bucket I created, but only to a subfolder of the user’s IAM user name. The policy makes use of ${aws:username}, so there’s no need to create them per user.
To Enable multi-user mode you need to navigate to the AWS Deep Racer console in us-east-1 (the service only exists there at the time of writing), and on the right hand side there’s a ‘multi-user management’ section. You must be using an account with sufficient AWS permissions to set this up, an AWS managed policy can provide this (AWSDeepRacerAccountAdminAccess). If you open setup there’s an intuitive screen that will navigate you through enabling the feature within the AWS account, as well as sample email invitations you can share with participants. Once enabled you can also look in the ‘Monitor usage’ section to see how many training hours have been used and how much has been spent. This section is also where you can control the level of ‘sponsorship’ you provide to participants, in terms of how many hours of training they can undertake (currently costs $3.5/hour) and how many models they can store (currently $0.023 per GB-Month). One thing to note — you cannot set sponsorship levels until someone has registered, nor can you set your own customer default levels. Each participant starts with 5 hours and 3 models, and periodically you’ll want to go in and set to whatever you want to provide as annoyingly there’s no API for Deep Racer so you can’t do it automatically based on an event.
I decided it’d be sensible to automate the set-up as much as possible. The Deep Racer elements can’t be automated due to lack of API, but everything else can. I used a Microsoft Form (we’re Office365 users) to capture email addresses of interested participants. I’ve then created infrastructure as code and open sourced it in my GitHub repository here to create everything except the Deep Racer config automatically. I took the email addresses and used them as the IAM username (adding them to the variables file in my repo will allow you to deploy them). Running the code creates everything required permission wise and the S3 bucket for model storage, it then dumps out IAM usernames and passwords for you to issue to participants. I used the CSV output, converted it to an Excel table that could then ultimately be used as a data source for a mail merge (I was tempted to use AWS SES but my participants weren’t in domains my account had verified) to issue those details along with instructions for our event, along with some billing alerts so your event doesn’t get out of hand cost wise.
Once your participants have been issued with their credentials, or you’ve allowed them to assume a role into your account the user experience is pretty simple. They should log-in to the AWS account (or assume role if not using the code I’ve supplied) and navigate to the Deep Racer console in us-east-1. Because multi-user is enabled and they’re assigned the AWSDeepRacerDefaultMultiUserAccess policy they’ll see a prompt to sign-in to Deep Racer. Note — this is a different set of permissions to their IAM credentials, and are specific to Deep Racer (they can be taken elsewhere and used in other AWS Accounts later). These accounts can’t be pre-created due to aforementioned lack of API, so you need to get participants to create their own account via the link below the ‘Sign-in’ button. A quick registration process and code sent to their email address to verify should result in access being granted.
Once logged in participants will be able to use Deep Racer, however their permissions will limit their visibility to only their own models, as well as prevent them from altering multi-user settings like changing the amount of sponsored usage. They will be able to create cars, models, run evaluations and submit models to virtual races.
Join me next time where I’ll give an overview on setting up a virtual race, along with details on how our competition is going! The second part of this series can be viewed here.
