Workflow Management

Overview

Workflows in Hyve enable automated, reproducible deployment pipelines with built-in requirements validation, dependency management, and parallel execution. This guide covers creating, running, and managing workflows.

Workflow Basics

What are Workflows?

Workflows are YAML-defined automation pipelines that:

Execute commands on clusters or locally
Validate tool and secret requirements
Manage job dependencies
Support parallel and sequential execution
Provide environment variable substitution

Creating Your First Workflow

From Template

# Create from built-in template
hyve workflow create my-deployment --template \
  --description "Deploy my application"

From File

# Create from existing YAML
hyve workflow create --file deployment.yaml

Manual Creation

Create ~/.hyve/repositories/<repo>/workflows/my-workflow.yaml:

apiVersion: v1
kind: Workflow
metadata:
  name: my-deployment
  description: Deploy application to cluster
spec:
  jobs:
    - name: deploy
      steps:
        - name: apply-manifests
          command: kubectl apply -f k8s/

Workflow Structure

Complete Workflow Example

apiVersion: v1
kind: Workflow
metadata:
  name: deployment-pipeline
  description: Complete deployment pipeline with testing and validation
  labels:
    environment: production
    team: platform
spec:
  env:
    APP_NAME: my-application
    NAMESPACE: default

  requirements:
    tools:
      - name: kubectl
        version: "1.28"
        description: Kubernetes CLI
      - name: docker
        version: "24.0"
        description: Container runtime
    secrets:
      - name: DOCKER_TOKEN
        provider: docker
        required: true
        description: Docker Hub authentication token

  jobs:
    - name: pre-checks
      description: Pre-deployment validation
      steps:
        - name: check-cluster-health
          command: kubectl get nodes

        - name: check-namespace
          command: kubectl get namespace ${NAMESPACE}

        - name: verify-images
          script: |
            echo "Verifying container images..."
            docker images | grep ${APP_NAME} || echo "No images found"

    - name: deploy-app
      description: Deploy the application
      dependsOn: ["pre-checks"]
      cluster: production
      env:
        DEPLOY_VERSION: latest
      steps:
        - name: apply-configmap
          action: kubectl-apply
          with:
            file: k8s/configmap.yaml

        - name: apply-secrets
          action: kubectl-apply
          with:
            file: k8s/secrets.yaml

        - name: deploy-application
          action: kubectl-apply
          with:
            file: k8s/deployment.yaml

        - name: wait-for-deployment
          command: kubectl rollout status deployment/${APP_NAME} -n ${NAMESPACE} --timeout=300s

    - name: post-deploy-tests
      description: Post-deployment testing and validation
      dependsOn: ["deploy-app"]
      steps:
        - name: health-check
          command: kubectl exec deployment/${APP_NAME} -n ${NAMESPACE} -- curl -f http://localhost:8080/health

        - name: smoke-tests
          script: |
            echo "Running smoke tests..."
            kubectl get pods -n ${NAMESPACE} -l app=${APP_NAME}
            kubectl logs deployment/${APP_NAME} -n ${NAMESPACE} --tail=10

        - name: verify-service
          command: kubectl get service ${APP_NAME} -n ${NAMESPACE}

    - name: notification
      description: Send deployment notification
      dependsOn: ["post-deploy-tests"]
      steps:
        - name: success-notification
          script: |
            echo "🎉 Deployment of ${APP_NAME} completed successfully!"
            kubectl get deployment ${APP_NAME} -n ${NAMESPACE} -o wide

Workflow Fields

Metadata Section

metadata:
  name: my-workflow              # Required: Unique workflow name
  description: What it does       # Optional: Human-readable description
  created: 2024-01-15T10:00:00Z  # Auto-generated
  updated: 2024-01-15T10:00:00Z  # Auto-updated
  labels:                         # Optional: Key-value labels
    environment: production
    team: platform

Spec Section

spec:
  env:                            # Workflow-level environment variables
    APP_NAME: my-app
    VERSION: v1.0.0

  requirements:                   # Validation requirements
    tools: []                     # Required CLI tools
    secrets: []                   # Required secrets/tokens

  jobs: []                        # Job definitions

Jobs

jobs:
  - name: deploy                  # Required: Job name
    description: Deploy app       # Optional: Description
    cluster: production           # Optional: Target cluster
    dependsOn: ["build"]          # Optional: Job dependencies
    if: "${ENV} == production"    # Optional: Conditional execution
    env:                          # Optional: Job-level env vars
      DEPLOY_ENV: prod
    steps: []                     # Required: Steps to execute

Steps

Steps can use one of three execution methods:

Command

- name: check-health
  command: kubectl get nodes

Simple shell command execution

Script

- name: complex-task
  script: |
    echo "Starting deployment..."
    for i in 1 2 3; do
      echo "Attempt $i"
      kubectl apply -f deployment.yaml && break
    done

Multi-line shell script

Action

- name: deploy-manifests
  action: kubectl-apply
  with:
    file: k8s/deployment.yaml

Built-in actions with parameters

Requirements Validation

Tool Requirements

Validate that required tools are installed:

spec:
  requirements:
    tools:
      - name: kubectl
        version: "1.28"
        description: Kubernetes CLI tool

      - name: helm
        version: "3.0"
        description: Kubernetes package manager

      - name: git
        version: "2.0"
        description: Version control

Validation behavior:

Checks if tool is in PATH
Validates version matches requirement
Fails workflow if tools are missing

Secret Requirements

Validate that required secrets/tokens are configured:

spec:
  requirements:
    secrets:
      - name: DOCKER_TOKEN
        provider: docker
        required: true
        description: Docker Hub authentication token

      - name: AWS_ACCESS_KEY
        provider: aws
        required: true
        description: AWS credentials

      - name: GITHUB_TOKEN
        provider: github
        required: false
        description: Optional GitHub token for private repos

Validation behavior:

Checks if secret exists in environment
Fails if required secret is missing
Warns if optional secret is missing

Running Workflows

Basic Execution

# Run workflow on current cluster
hyve workflow run deploy-app

# Run on specific cluster
hyve workflow run deploy-app --cluster production

# Run without cluster context (local commands only)
hyve workflow run local-build

Execution Output

Standard Output

hyve workflow run deploy-app --cluster production

Output:

🚀 Starting workflow 'deploy-app'
🎯 Target cluster: production

✅ Workflow 'deploy-app' completed successfully
⏱️  Duration: 2m34s

With Logs

hyve workflow run deploy-app --cluster production --logs

Shows detailed execution logs with timestamps

With Output

hyve workflow run deploy-app --cluster production --output

Shows step outputs and command results

Environment Variables

Variable Levels

Variables can be defined at multiple levels:

Workflow Level

Available to all jobs:

spec:
  env:
    APP_NAME: my-application
    VERSION: v1.0.0

Job Level

Override workflow variables:

jobs:
  - name: deploy-prod
    env:
      ENVIRONMENT: production
      REPLICAS: "3"

System Variables

Built-in variables:

${CLUSTER_NAME} - Current cluster name
${WORKFLOW_NAME} - Workflow name
${JOB_NAME} - Current job name

Variable Substitution

spec:
  env:
    APP_NAME: podinfo
    NAMESPACE: default
    REPLICAS: "3"

  jobs:
    - name: deploy
      steps:
        - name: deploy-app
          command: |
            kubectl create deployment ${APP_NAME} \
              --image=ghcr.io/stefanprodan/podinfo:latest \
              --replicas=${REPLICAS} \
              -n ${NAMESPACE}

        - name: expose-service
          command: kubectl expose deployment ${APP_NAME} --port=9898 -n ${NAMESPACE}

Job Dependencies

Sequential Execution

jobs:
  - name: build
    steps:
      - name: compile
        command: make build

  - name: test
    dependsOn: ["build"]
    steps:
      - name: run-tests
        command: make test

  - name: deploy
    dependsOn: ["test"]
    steps:
      - name: apply
        command: kubectl apply -f manifests/

Execution order: build → test → deploy

Parallel with Dependencies

jobs:
  - name: build-frontend
    steps:
      - name: build
        command: npm run build

  - name: build-backend
    steps:
      - name: build
        command: go build

  - name: test-frontend
    dependsOn: ["build-frontend"]
    steps:
      - name: test
        command: npm test

  - name: test-backend
    dependsOn: ["build-backend"]
    steps:
      - name: test
        command: go test ./...

  - name: deploy
    dependsOn: ["test-frontend", "test-backend"]
    steps:
      - name: deploy-all
        command: kubectl apply -f k8s/

Execution:

build-frontend and build-backend run in parallel
test-frontend runs after build-frontend
test-backend runs after build-backend
deploy runs after both tests complete

Conditional Execution

Using If Conditions

jobs:
  - name: deploy-to-prod
    if: "${ENVIRONMENT} == production"
    steps:
      - name: deploy
        command: kubectl apply -f production/

  - name: deploy-to-staging
    if: "${ENVIRONMENT} == staging"
    steps:
      - name: deploy
        command: kubectl apply -f staging/

Built-in Actions

kubectl-apply

Apply Kubernetes manifests:

steps:
  - name: apply-deployment
    action: kubectl-apply
    with:
      file: k8s/deployment.yaml

kubectl-delete

Delete Kubernetes resources:

steps:
  - name: cleanup
    action: kubectl-delete
    with:
      file: k8s/old-deployment.yaml

Workflow Management

Listing Workflows

hyve workflow list

Output:

📋 Workflows in repository (5):

NAME                 DESCRIPTION                              JOBS  CREATED
deployment-pipeline  Complete deployment pipeline             4     2024-01-15
simple-deploy        Basic deployment workflow                1     2024-01-14
test-suite          Run integration tests                    2     2024-01-13
backup-database     Backup production database               3     2024-01-12
rollback            Rollback to previous version             2     2024-01-11

💡 Commands:
   hyve workflow show <name>     # Show workflow details
   hyve workflow run <name>      # Run workflow
   hyve workflow delete <name>   # Delete workflow

Showing Workflow Details

hyve workflow show deployment-pipeline

Output:

📋 Workflow: deployment-pipeline
📝 Description: Complete deployment pipeline with testing and validation
📅 Created: 2024-01-15 10:30:45
📅 Updated: 2024-01-15 14:22:10
🏷️  Labels:
   environment: production
   team: platform

🌍 Environment Variables:
   APP_NAME: my-application
   NAMESPACE: default

🚀 Jobs (4):

  1. pre-checks
     📝 Pre-deployment validation
     📋 Steps (3):
       1. check-cluster-health
          🔧 Command: kubectl get nodes
       2. check-namespace
          🔧 Command: kubectl get namespace ${NAMESPACE}
       3. verify-images
          📜 Script: echo "Verifying container images..."

  2. deploy-app
     📝 Deploy the application
     🔗 Depends on: pre-checks
     🎯 Cluster: production
     📋 Steps (4):
       1. apply-configmap
          ⚡ Action: kubectl-apply
       2. apply-secrets
          ⚡ Action: kubectl-apply
       3. deploy-application
          ⚡ Action: kubectl-apply
       4. wait-for-deployment
          🔧 Command: kubectl rollout status...

💡 Run with: hyve workflow run deployment-pipeline

Deleting Workflows

# Delete with confirmation
hyve workflow delete old-workflow

# Force delete without confirmation
hyve workflow delete old-workflow --force

Validating Workflows

Run Validation

hyve workflow validate deployment-pipeline

Validation checks:

Required fields (apiVersion, kind, metadata.name)
Job structure and naming
Step definitions
Dependency resolution
Circular dependency detection
Action parameter validation

Validation Output

Valid Workflow

🔍 Validating workflow 'deployment-pipeline'...

✅ Workflow is valid
📋 Jobs: 4
📋 Total steps: 12
✨ No warnings

With Errors

🔍 Validating workflow 'broken-workflow'...

❌ Validation Failed

Errors:
  • Job 'deploy' has no steps
  • Job 'test' depends on non-existent job 'build'
  • Step 'deploy-app' has no command, script, or action
  • Circular dependency detected in job dependencies

With Warnings

🔍 Validating workflow 'my-workflow'...

⚠️  Warnings:
  • Job 'deploy', step 2 is missing a name
  • Unknown action 'custom-action'
  • Unexpected apiVersion 'v2', expected 'v1'

✅ Workflow is valid
📋 Jobs: 2
📋 Total steps: 5

If several state repositories need the same workflow — a standard monitoring setup, a backup routine, a teardown sequence — you don’t have to copy the file into each one. Reference it remotely instead.

1. Add a remote reference

In any template or cluster’s spec.workflows, use a {source, path} mapping instead of a local name:

spec:
  workflows:
    onCreate:
      - source: github.com/myorg/shared-workflows//setup-monitoring.yaml@v1.2.0

Version is optional — omit @v1.2.0 to track the latest commit on the default branch. Hyve resolves it once and locks the resolved SHA, so reconciles stay reproducible even though the source string itself is versionless.

2. Lock it

hyve workflow install

This scans every template and cluster definition for remote references, resolves and content-hashes each one, and writes the results into hyve.lock. Commit hyve.lock — it’s how every teammate and every CI run reproduces the exact same workflow content.

3. Run it directly (optional)

You don’t need a lifecycle hook to use a remote workflow — hyve workflow run accepts a bare name (resolved against hyve.lock if not found locally) or a full source string:

# By name, once installed
hyve workflow run setup-monitoring

# By full source string — fetches on the fly if not installed, without
# writing to hyve.lock
hyve workflow run github.com/myorg/shared-workflows//setup-monitoring.yaml@v1.2.0

4. Keep it fresh

# Re-resolve to the latest commit/tag and refresh the lock entry
hyve workflow update github.com/myorg/shared-workflows//setup-monitoring.yaml

# Confirm every locked workflow's cached content still matches its sha256
hyve workflow verify

See Remote Workflow References in the CLI reference for the full source-string format and path-resolution rules.

Workflow Examples

Simple Deployment

apiVersion: v1
kind: Workflow
metadata:
  name: simple-deploy
  description: Deploy application to cluster
spec:
  env:
    APP_NAME: my-app

  jobs:
    - name: deploy
      steps:
        - name: apply-manifests
          command: kubectl apply -f k8s/

        - name: wait-for-ready
          command: kubectl wait --for=condition=available deployment/${APP_NAME} --timeout=300s

Database Backup

apiVersion: v1
kind: Workflow
metadata:
  name: backup-database
  description: Backup PostgreSQL database
spec:
  env:
    DB_NAME: production_db
    BACKUP_DIR: /backups

  requirements:
    tools:
      - name: kubectl
        version: "1.28"
      - name: pg_dump
        version: "14.0"

  jobs:
    - name: create-backup
      steps:
        - name: dump-database
          command: |
            kubectl exec deployment/postgres -- \
              pg_dump ${DB_NAME} > ${BACKUP_DIR}/backup-$(date +%Y%m%d-%H%M%S).sql

        - name: compress
          command: gzip ${BACKUP_DIR}/backup-*.sql

Multi-Environment Deployment

apiVersion: v1
kind: Workflow
metadata:
  name: multi-env-deploy
  description: Deploy to multiple environments
spec:
  jobs:
    - name: deploy-dev
      cluster: dev-cluster
      steps:
        - name: deploy
          command: kubectl apply -f k8s/dev/

    - name: deploy-staging
      cluster: staging-cluster
      dependsOn: ["deploy-dev"]
      steps:
        - name: deploy
          command: kubectl apply -f k8s/staging/

    - name: deploy-production
      cluster: prod-cluster
      dependsOn: ["deploy-staging"]
      steps:
        - name: deploy
          command: kubectl apply -f k8s/production/

CI/CD Pipeline

apiVersion: v1
kind: Workflow
metadata:
  name: ci-cd-pipeline
  description: Complete CI/CD pipeline
spec:
  env:
    IMAGE_NAME: my-app
    IMAGE_TAG: latest

  requirements:
    tools:
      - name: docker
        version: "24.0"
      - name: kubectl
        version: "1.28"
    secrets:
      - name: DOCKER_TOKEN
        provider: docker
        required: true

  jobs:
    - name: build
      steps:
        - name: build-image
          command: docker build -t ${IMAGE_NAME}:${IMAGE_TAG} .

        - name: push-image
          script: |
            echo ${DOCKER_TOKEN} | docker login -u ${DOCKER_USER} --password-stdin
            docker push ${IMAGE_NAME}:${IMAGE_TAG}

    - name: test
      dependsOn: ["build"]
      steps:
        - name: unit-tests
          command: docker run ${IMAGE_NAME}:${IMAGE_TAG} npm test

        - name: integration-tests
          command: docker run ${IMAGE_NAME}:${IMAGE_TAG} npm run test:integration

    - name: deploy
      dependsOn: ["test"]
      cluster: production
      steps:
        - name: update-deployment
          command: kubectl set image deployment/my-app app=${IMAGE_NAME}:${IMAGE_TAG}

        - name: wait-for-rollout
          command: kubectl rollout status deployment/my-app --timeout=600s

Best Practices

1. Use Requirements Validation

spec:
  requirements:
    tools:
      - name: kubectl
        version: "1.28"
    secrets:
      - name: DEPLOY_TOKEN
        provider: github
        required: true

Catch missing dependencies before execution

2. Organize Jobs Logically

jobs:
  - name: pre-checks      # Validate before deployment
  - name: deploy          # Main deployment
  - name: post-tests      # Validate after deployment
  - name: notify          # Send notifications

Clear separation of concerns

3. Use Environment Variables

spec:
  env:
    APP_NAME: my-app
    VERSION: v1.0.0
    NAMESPACE: production

Avoid hardcoding values

4. Add Descriptions

metadata:
  name: deploy-app
  description: Deploy application with health checks and rollback capability

jobs:
  - name: deploy
    description: Deploy application to production cluster
    steps:
      - name: apply-manifests
        # Description helps others understand purpose

Document workflow purpose and steps

5. Handle Failures Gracefully

jobs:
  - name: deploy
    steps:
      - name: deploy-app
        command: kubectl apply -f k8s/

      - name: verify-deployment
        command: kubectl rollout status deployment/my-app --timeout=300s

      - name: rollback-on-failure
        # This step runs if previous step fails
        command: kubectl rollout undo deployment/my-app

Plan for failure scenarios

Troubleshooting

Workflow Not Found

Problem: Workflow file doesn’t exist

Solutions:

# List available workflows
hyve workflow list

# Check workflow directory
ls ~/.hyve/repositories/production/workflows/

# Create workflow if missing
hyve workflow create my-workflow --template

Requirements Validation Fails

Problem: Missing required tools or secrets

Solutions:

# Check which tools are required
hyve workflow show my-workflow

# Install missing tools
# For kubectl:
brew install kubectl  # macOS
# For helm:
brew install helm     # macOS

# Set missing secrets
export DOCKER_TOKEN=your_token_here
export AWS_ACCESS_KEY=your_key_here

Circular Dependency

Problem: Jobs have circular dependencies

Solution:

# Validate workflow structure
hyve workflow validate my-workflow

# Fix dependency chain in YAML:
# Before (circular):
# job-a depends on job-b
# job-b depends on job-a

# After (fixed):
# job-a has no dependencies
# job-b depends on job-a

Step Execution Fails

Problem: Individual step returns error

Solutions:

# Run with verbose logging
hyve workflow run my-workflow --logs --output

# Test step command manually
hyve cluster auth my-cluster
kubectl get nodes  # Test the actual command

# Check cluster connectivity
kubectl cluster-info

Variable Substitution Not Working

Problem: Variables not being replaced

Solution:

# Ensure variables are defined
spec:
  env:
    APP_NAME: my-app

# Use correct syntax: ${VARIABLE}
steps:
  - name: deploy
    command: kubectl apply -f ${APP_NAME}/manifests/

# NOT: $VARIABLE or {VARIABLE}

Cluster Management

Manage target clusters for workflows

Template Management

Templates with automated workflows

Kubeconfig Management

Manage cluster access

CLI: Workflow Commands

Complete workflow command reference

Workflow Management

Overview

Workflow Basics

What are Workflows?

Creating Your First Workflow

Workflow Structure

Complete Workflow Example

Workflow Fields

Metadata Section

Spec Section

Jobs

Steps

Requirements Validation

Tool Requirements

Secret Requirements

Running Workflows

Basic Execution

Execution Output

Environment Variables

Variable Levels

Variable Substitution

Job Dependencies

Sequential Execution

Parallel with Dependencies

Conditional Execution

Using If Conditions

Built-in Actions

kubectl-apply

kubectl-delete

Workflow Management

Listing Workflows

Showing Workflow Details

Deleting Workflows

Validating Workflows

Run Validation

Validation Output

Sharing Workflows Across Repositories

1. Add a remote reference

2. Lock it

3. Run it directly (optional)

4. Keep it fresh

Workflow Examples

Simple Deployment

Database Backup

Multi-Environment Deployment

CI/CD Pipeline

Best Practices

Troubleshooting

Related