Model Artifacts & Versioning

Advanced model versioning patterns, automated deployment workflows, and integration with production systems.

---

Overview

This guide covers:

• Versioning strategies for different ML workflows

• Automated deployment on approval

• Model artifact management

• Production deployment patterns

• Governance through Model Hub

---

Versioning Strategies

Development vs. Production Versions

Use tags and states to separate development from production:

# Development/experiment version
metadata = {
    "model.pkl": {
        "valohai.model-versions": [{
            "model_uri": "model://churn-model/",
            "model_version_tags": ["experiment", "feature-test", "dev"],
            "model_release_note": "Testing new feature engineering"
        }],
        "experiment_id": "exp-042",
        "status": "experimental"
    }
}

# Production candidate version
metadata = {
    "model.pkl": {
        "valohai.model-versions": [{
            "model_uri": "model://churn-model/",
            "model_version_tags": ["production-candidate", "validated"],
            "model_release_note": "Ready for staging deployment - passed all quality gates"
        }],
        "validation_passed": True,
        "quality_score": 0.95
    }
}

Workflow:

1. Create development versions with experiment tag (stay in Pending)

2. Best experiment → Retag as production-candidate

3. Validate → Approve

4. Deploy to production

---

Semantic Versioning Pattern

Organize versions with semantic meaning:

# Major version: Model architecture change
metadata = {
    "model.pkl": {
        "valohai.model-versions": [{
            "model_uri": "model://recommendation/",
            "model_version_tags": ["v2.0.0", "architecture-change", "transformer"],
            "model_release_note": "Major update: Switch from collaborative filtering to transformer-based model"
        }],
        "architecture": "transformer",
        "major_version": 2
    }
}

# Minor version: Feature improvements
metadata = {
    "model.pkl": {
        "valohai.model-versions": [{
            "model_uri": "model://recommendation/",
            "model_version_tags": ["v2.1.0", "feature-update"],
            "model_release_note": "Added user engagement features, improved accuracy by 3%"
        }],
        "architecture": "transformer",
        "major_version": 2,
        "minor_version": 1
    }
}

# Patch version: Bug fixes or retraining
metadata = {
    "model.pkl": {
        "valohai.model-versions": [{
            "model_uri": "model://recommendation/",
            "model_version_tags": ["v2.1.1", "retrain", "patch"],
            "model_release_note": "Retrained on updated data - same architecture"
        }],
        "architecture": "transformer",
        "major_version": 2,
        "minor_version": 1,
        "patch_version": 1
    }
}

---

Environment-Specific Versions

Maintain separate version tracks for different environments:

# Staging version
metadata = {
    "model.pkl": {
        "valohai.model-versions": [{
            "model_uri": "model://fraud-detection/",
            "model_version_tags": ["staging", "2024-q1"],
            "model_release_note": "Deployed to staging for validation"
        }],
        "environment": "staging",
        "deployment_date": "2024-01-15"
    }
}

# Production version (after staging validation)
metadata = {
    "model.pkl": {
        "valohai.model-versions": [{
            "model_uri": "model://fraud-detection/",
            "model_version_tags": ["production", "2024-q1", "validated"],
            "model_release_note": "Promoted from staging after 2 weeks validation"
        }],
        "environment": "production",
        "deployment_date": "2024-01-29",
        "staging_metrics": {"false_positive_rate": 0.02}
    }
}

---

Model Artifact Management

Multi-File Model Packages

Package models with all required artifacts:

import pickle
import json

# Save model components
model.save('/valohai/outputs/model.h5')

with open('/valohai/outputs/tokenizer.pkl', 'wb') as f:
    pickle.dump(tokenizer, f)

with open('/valohai/outputs/label_encoder.pkl', 'wb') as f:
    pickle.dump(label_encoder, f)

with open('/valohai/outputs/config.json', 'w') as f:
    json.dump({
        'vocab_size': 10000,
        'embedding_dim': 128,
        'max_sequence_length': 512
    }, f)

# Add all files to model version
metadata = {
    "model.h5": {
        "valohai.model-versions": ["model://text-classifier/"]
    },
    "tokenizer.pkl": {
        "valohai.model-versions": ["model://text-classifier/"]
    },
    "label_encoder.pkl": {
        "valohai.model-versions": ["model://text-classifier/"]
    },
    "config.json": {
        "valohai.model-versions": ["model://text-classifier/"]
    }
}

Deployment: All files download together:

# Load complete model package
model = load_model('/valohai/inputs/model/model.h5')
tokenizer = pickle.load(open('/valohai/inputs/model/tokenizer.pkl', 'rb'))
label_encoder = pickle.load(open('/valohai/inputs/model/label_encoder.pkl', 'rb'))
config = json.load(open('/valohai/inputs/model/config.json'))

---

Framework-Specific Artifacts

TensorFlow/Keras:

import tensorflow as tf

# Save in multiple formats
model.save('/valohai/outputs/model.h5')  # HDF5 format

# Save weights only
model.save_weights('/valohai/outputs/model_weights.h5')

# Add all to model version
metadata = {
    "model.h5": {"valohai.model-versions": ["model://image-classifier/"]},
    "model_weights.h5": {"valohai.model-versions": ["model://image-classifier/"]}
}

PyTorch:

import torch

# Save complete model
torch.save(model, '/valohai/outputs/model_full.pth')

# Save state dict (recommended)
torch.save(model.state_dict(), '/valohai/outputs/model_state.pth')

# Save checkpoint with optimizer state
torch.save({
    'epoch': epoch,
    'model_state_dict': model.state_dict(),
    'optimizer_state_dict': optimizer.state_dict(),
    'loss': loss,
}, '/valohai/outputs/checkpoint.pth')

metadata = {
    "model_state.pth": {
        "valohai.model-versions": ["model://object-detection/"],
        "framework": "pytorch",
        "torch_version": torch.__version__
    }
}

---

ONNX Export for Deployment

Export to ONNX for cross-framework deployment:

import torch
import torch.onnx

# Export PyTorch model to ONNX
model.eval()
dummy_input = torch.randn(1, 3, 224, 224)

torch.onnx.export(
    model,
    dummy_input,
    '/valohai/outputs/model.onnx',
    export_params=True,
    opset_version=11,
    input_names=['input'],
    output_names=['output'],
    dynamic_axes={'input': {0: 'batch_size'}, 'output': {0: 'batch_size'}}
)

# Add to model version
metadata = {
    "model.onnx": {
        "valohai.model-versions": [{
            "model_uri": "model://image-classifier/",
            "model_version_tags": ["onnx", "optimized", "production"],
            "model_release_note": "ONNX export for cross-framework deployment"
        }],
        "format": "onnx",
        "input_shape": [1, 3, 224, 224],
        "output_shape": [1, 1000]
    }
}

---

Production Deployment Patterns

Batch Inference

Use approved models for scheduled batch predictions:

valohai.yaml:

- step:
    name: daily-batch-inference
    image: python:3.9
    command:
      - pip install scikit-learn pandas
      - python batch_predict.py
    inputs:
      - name: model
        default: model://churn-prediction/latest  # Always uses latest approved
      - name: customers
        default: dataset://customer-data/daily-snapshot

Schedule: Run daily at 2 AM to generate predictions for customer success team.

---

Real-Time Serving (External)

Export model for external serving platform:

# Training: Create model version
metadata = {
    "model.pkl": {
        "valohai.model-versions": [{
            "model_uri": "model://recommendation/",
            "model_version_tags": ["serving-ready", "optimized"],
            "model_release_note": "Optimized for low-latency serving"
        }],
        "inference_latency_ms": 45,
        "model_size_mb": 120,
        "serving_framework": "tensorflow-serving"
    }
}

# Deployment: Download from Valohai (API), deploy to serving platform
# (Run outside Valohai or via deployment execution)
# Example platforms: SageMaker, Vertex AI, KServe, Seldon, custom API

---

Related Pages

• Models Overview — Introduction to Model Hub

• Create and Manage Models — Basic model creation

• Add Context to Your Data Files — Metadata system

• Schedule Triggers - Run a pipeline on a schedule