Deploy to AWS manually¶
The Compute and Data Layer of Valohai can be deployed to your AWS Account. This enables you to:
Use your own EC2 instances to run machine learning jobs.
Use your own S3 Buckets for storing training artefacts such as trained models, preprocessed datasets, visualizations, etc.
Access databases and data warehouses directly from the workers, which are inside your network.
Valohai doesn’t have direct access to the EC2 instances that execute the machine learning jobs. Instead it communicates with a static EC2 instance in your AWS subscription that’s responsible for storing the job queue, job states, and short-term logs.

Warning
Before running the template you’ll need the following information from Valohai:
valohai_assume_user
is the ARN of the user Valohai will use to assume a role in your AWS subscription to manage EC2 instances.queue_address
will be assigned for the queue in your subscription.
Configure the IAM roles¶
The 4 roles listed below are required for a Valohai installation.
ValohaiQueueRole
will be attached to the Valohai Queue instance, and allows it to fetch the generated password from your AWS Secrets Manager. Access is restricted to secrets that are tagged valohai:1.ValohaiWorkerRole
is attached to all autoscaled EC2 instances that are launched for machine learning jobs.ValohaiS3MultipartRole
will be used to upload files larger than 5GB.ValohaiMaster
is the role that the Valohai service will use to manage autoscaling and EC2 resources. The role is also used to manage the newly provisioned valohai-data-* S3 Bucket.
Create the 4 policies listed below by going to your AWS Console > IAM > Policies.
Warning
Update the following placeholders in ValohaiMaster
and ValohaiS3MultipartPolicy
before you save the policies:
valohai-data-<my-company-name>
with your company name (valohai-data-companya
).<AWS-ACCOUNT-ID>
with your AWS Account ID.
ValohaiQueuePolicy
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "0",
"Effect": "Allow",
"Action": [
"secretsmanager:GetResourcePolicy",
"secretsmanager:GetSecretValue",
"secretsmanager:DescribeSecret",
"secretsmanager:ListSecretVersionIds"
],
"Resource": "*",
"Condition": {
"StringEquals": {
"secretsmanager:ResourceTag/valohai": "1"
}
}
},
{
"Sid": "1",
"Effect": "Allow",
"Action": "secretsmanager:GetRandomPassword",
"Resource": "*"
}
]
}
ValohaiWorkerPolicy
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "autoscaling:SetInstanceProtection",
"Resource": "*",
"Effect": "Allow",
"Sid": "1"
},
{
"Action": "ec2:DescribeInstances",
"Resource": "*",
"Effect": "Allow",
"Sid": "2"
}
]
}
ValohaiS3MultipartPolicy
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "MultipartAccess",
"Effect": "Allow",
"Action": [
"s3:AbortMultipartUpload",
"s3:GetBucketLocation",
"s3:GetObject",
"s3:ListBucket",
"s3:ListBucketMultipartUploads",
"s3:ListBucketVersions",
"s3:ListMultipartUploadParts",
"s3:PutObject"
],
"Resource": [
"arn:aws:s3:::valohai-data-<my-company-name>",
"arn:aws:s3:::valohai-data-<my-company-name>/*"
]
}
]
}
ValohaiMasterPolicy
{
"Version" : "2012-10-17",
"Statement" : [
{
"Sid" : "2",
"Effect" : "Allow",
"Action" : [
"ec2:DescribeInstances",
"ec2:DescribeVpcs",
"ec2:DescribeKeyPairs",
"ec2:DescribeImages",
"ec2:DescribeSecurityGroups",
"ec2:DescribeSubnets",
"ec2:DescribeInstanceTypes",
"ec2:DescribeLaunchTemplates",
"ec2:DescribeLaunchTemplateVersions",
"ec2:CreateTags",
"autoscaling:DescribeAutoScalingGroups",
"autoscaling:DescribeScalingActivities"
],
"Resource" : "*"
},
{
"Sid" : "AllowUpdatingSpotLaunchTemplates",
"Effect" : "Allow",
"Action" : [
"ec2:CreateLaunchTemplate",
"ec2:CreateLaunchTemplateVersion",
"ec2:ModifyLaunchTemplate",
"ec2:RunInstances",
"ec2:RebootInstances",
"autoscaling:UpdateAutoScalingGroup",
"autoscaling:CreateOrUpdateTags",
"autoscaling:SetDesiredCapacity",
"autoscaling:CreateAutoScalingGroup"
],
"Resource" : "*",
"Condition" : {
"ForAllValues:StringEquals" : {
"aws:ResourceTag/valohai" : "1"
}
}
},
{
"Sid" : "ServiceLinkedRole",
"Effect" : "Allow",
"Action" : "iam:CreateServiceLinkedRole",
"Resource" : "arn:aws:iam::*:role/aws-service-role/autoscaling.amazonaws.com/AWSServiceRoleForAutoScaling"
},
{
"Sid" : "4",
"Effect" : "Allow",
"Action" : [
"iam:PassRole",
"iam:GetRole"
],
"Resource" : "arn:aws:iam::<ACCOUNT-ID>:role/ValohaiWorkerRole"
},
{
"Sid" : "0",
"Effect" : "Allow",
"Condition" : {
"StringEquals" : {
"secretsmanager:ResourceTag/valohai" : "1"
}
},
"Action" : [
"secretsmanager:GetResourcePolicy",
"secretsmanager:GetSecretValue",
"secretsmanager:DescribeSecret",
"secretsmanager:ListSecretVersionIds"
],
"Resource" : "*"
},
{
"Action" : "secretsmanager:GetRandomPassword",
"Resource" : "*",
"Effect" : "Allow",
"Sid" : "1"
},
{
"Effect" : "Allow",
"Action" : "s3:*",
"Resource" : [
"arn:aws:s3:::valohai-data-<my-company-name>",
"arn:aws:s3:::valohai-data-<my-company-name>/*"
]
}
]
}
Next create the roles listed below by going to your AWS Console > IAM > Roles.
Name |
Use Case |
Policy |
---|---|---|
ValohaiQueueRole |
EC2 |
|
ValohaiWorkerRole |
EC2 |
|
ValohaiS3MultipartRole |
Another AWS Account.
|
|
ValohaiMaster |
Another AWS Account
|
|
Important
Note ValohaiQueueRole
and ValohaiWorkerRole
need to also have instance profiles They will get automatically created when you create a role through the console.
If you’re creating the resources with the AWS CLI, follow the AWS documentation to learn how to create and attach an instance profile to a role.
Deploying resources¶
We recommend setting up the Valohai resources in a fresh AWS Account. If you prefer to use an existing account and VPC, you’ll need to adopt the instructions below to your environment.
Reach out to support@valohai.com if you have any questions.
Warning
Before deploying your resources, you’ll need to get the following information from Valohai:
valohai_assume_user
is the ARN of the user Valohai will use to assume a role in your AWS subscription to manage EC2 instances.queue_address
will be assigned for your job queue.
Resource |
Name |
Details |
---|---|---|
VPC |
|
|
Subnets |
|
|
Internet Gateway |
|
|
Routing Table |
Rename the default route table of |
|
Security Group |
|
|
Security Group |
|
|
Secrets Manager Secret |
|
|
Elastic IP |
|
|
EC2 Instance |
IMPORTANT: Replace the |
|
S3 Bucket |
|
|
Next steps¶
You’ll need to share the ARN of the ValohaiMaster Role
with your Valohai contact, so they can finish the setup on app.valohai.com and enable your organization’s environments on the platform.