Snowflake Connector

Run SQL queries on Snowflake and save results to your data store.

Why use this connector?

Query directly from Valohai: No need to export data manually. Write SQL, run execution, get CSV output.

Version your queries: Every query is saved with the execution. Reproduce results months later by checking which query ran when.

Feed downstream jobs: Query outputs get datum URLs. Use them as inputs in other executions or pipelines.

Requirements

• Organization and account in Snowflake

• Existing database with data

• Firewall settings allow Valohai workers to access the database

• User with permissions to query the database

Authentication

Snowflake uses username and password authentication.

Set these environment variables:

• SNOWSQL_ACCOUNT: Your Snowflake account in format <orgname>-<account_name>

• SNOWSQL_WAREHOUSE: Warehouse name

• SNOWSQL_DATABASE: Database name

• SNOWSQL_SCHEMA: (Optional) Schema name, default is PUBLIC

• SNOWSQL_USER: Snowflake username

• SNOWSQL_PWD: Snowflake password (mark as a secret)

Add environment variables

Environment variables can be added:

• Project-wide: Project Settings → Environment Variables

• Organization-wide: Admin users can create environment variable groups that can be passed to several projects.

• Per-execution: Set when creating the execution

We recommend project or organization settings for credentials.

Run a query

1. Open your project

2. Click Create Execution

3. Expand valohai-ecosystem → Select snowflake-query

4. Configure parameters:

   • query: Your SQL query

   • output-path: (Optional) Output filename, default is results.csv

   • datum-alias: (Optional) Alias for easy reference, e.g., latest-sales

5. Verify environment variables are set

6. Click Create Execution

Example query

SELECT 
    region,
    product_line,
    SUM(revenue) as total_revenue,
    AVG(profit_margin) as avg_margin
FROM sales_data
WHERE sale_date >= '2025-01-01'
GROUP BY region, product_line
ORDER BY total_revenue DESC
LIMIT 500

Results are saved as results.csv (or your custom output path) and uploaded to your data store.

Use query results

The output gets a datum URL. Reference it in other executions:

- step:
    name: analyze-sales
    image: python:3.11
    command:
      - python analyze.py
    inputs:
      - name: sales-data
        default: datum://latest-sales

Or use it in a pipeline by passing the execution output to the next node.

Troubleshooting

Connection fails

Check:

• SNOWSQL_ACCOUNT format is correct: <orgname>-<account_name>

• Firewall rules allow Valohai workers to connect

• Warehouse is running (not suspended)

Authentication fails

Check:

• SNOWSQL_USER and SNOWSQL_PWD are correct

• User has access to the specified database and warehouse

• Password is marked as a secret in Valohai

Query fails with permission error

Check:

• User has SELECT permission on the tables

• Schema name is correct (default is PUBLIC)

• Database and warehouse names match your Snowflake setup

Next steps

Other database connectors:

• BigQuery

• AWS Redshift

Build your own:

• Create custom library steps