The Compute and Data Layer of Valohai can be deployed to your GCP project. This enables you to:
- Use your own Virtual Machines instances to run machine learning jobs.
- Use your own Google Storage Bucket for storing training artifacts such as trained models, preprocessed datasets, visualizations, etc.
- Access databases and data warehouses directly from the workers, which are inside your network.
Valohai doesn’t have direct access to the virtual machine instances that execute the machine learning jobs. Instead, it communicates with a static virtual machine in your GCP project that’s responsible for storing the job queue, job states, and short-term logs.
Requirements
Before you can deploy Valohai to your environment, you’ll need to get the valohai_email
and queue_address
from Valohai support.
Configure the IAM Resources
Start by creating a new role in your GCP Project IAM & Admin -> Roles.
Property | Value |
---|---|
Title | ValohaiMaster |
Description | A role used by app.valohai.com to manage Valohai related resources |
ID | ValohaiMaster |
Role launch stage | General Availability |
Permissions |
|
Next, create two new Service Accounts.
Property | Value |
---|---|
Name | valohai-sa-master |
Description | Used to manage Valohai related VM resources in the project |
Role |
ValohaiMaster
|
Grant access | Add the valohai_email you received a Service account token creator |
Property | Value |
---|---|
Name | valohai-sa-queue |
Description | Service account used by the Valohai queue virtual machine |
Roles |
|
Deploying Resources
Secret Manager
You’ll need to upload two secrets for Valohai:
valohai_redis_password
as the password that will be set on the job queue machine. Your workers and app.valohai.com will need this to be able to access the job queue.valohai_master_sa
will be used by the Valohai autoscaler to create and delete virtual machine resources for your Valohai machine learning jobs.
Start by going to the IAM -> Service Accounts page and opening valohai-sa-master
.
Go to the KEYS
tab and create a new JSON key. The key will be downloaded to your workstation.
Next, go to the Security -> Secret Manager and create the two secrets:
Name | Secret value |
---|---|
valohai_redis_password |
Generate a random password, that includes lowercase and capital letters and numbers. |
valohai_master_sa |
Paste the JSON contents of the key file you just downloaded |
VPC
Start by creating a new VPC in your GCP Project VPC -> VPC networks.
Property | Value |
---|---|
VPC | valohai-vpc |
Subnet creation mode | Automatic |
Firewall Rules
Next, open the VPC -> Firewall and create two firewall rules.
Property | Value |
---|---|
Rule name | valohai-fr-queue-redis |
Description | Allows connection to the queue from Valohai services and valohai workers from this project |
Network | valohai-vpc |
Direction | Ingress |
Action on match | Allow |
Target tags | valohai-queue |
Source IP Ranges |
|
Second source filter / Source tags | valohai-worker |
Specified protocols and ports | TCP on port 63790 |
Property | Value |
---|---|
Rule name | valohai-fr-queue-http |
Description | Allows connections on port 80 for the Let’s Encrypt HTTP challenge |
Network | valohai-vpc |
Direction | Ingress |
Action on match | Allow |
Target tags | valohai-queue |
Source IP Ranges | 0.0.0.0/0 |
Specified protocols and ports | TCP on port 80 |
Virtual Machine
Property | Value |
---|---|
Name | valohai-queue |
Region | Choose your region and zone |
Type | e2-medium |
Boot disk |
|
Identity and API access | Service Account: valohai-sa-queue |
Networking |
|
Management | Startup script: IMPORTANT: Replace the <queue_address> with the value you got from Valohai.
|
Next steps
You’ll need to share the following details with your Valohai contact, so they can finish the setup on app.valohai.com and enable your organization’s environments on the platform:
- External IP of the
valohai-queue
virtual machine - Project ID
Note
Make sure you have enough quota for both vCPUs and GPUs on your GCP account. You can read more about quotas on GCP’s documentation here.