grafana/apps/dashboard/pkg/migration

Dashboard migrations

This document describes the Grafana dashboard migration system, including metrics, logging, and testing infrastructure for dashboard schema migrations and API version conversions.

Overview

Metrics

The dashboard migration system now provides comprehensive observability through:

• Prometheus metrics for tracking conversion success/failure rates and performance
• Structured logging for debugging and monitoring conversion operations
• Automatic instrumentation via wrapper functions that eliminate code duplication
• Error classification to distinguish between different types of migration failures

1. Dashboard conversion success metric

Metric Name: grafana_dashboard_migration_conversion_success_total

Type: Counter

Description: Total number of successful dashboard conversions

Labels:

• source_version_api - Source API version (e.g., "dashboard.grafana.app/v0alpha1")
• target_version_api - Target API version (e.g., "dashboard.grafana.app/v1beta1")
• source_schema_version - Source schema version (e.g., "16") - only for v0/v1 dashboards
• target_schema_version - Target schema version (e.g., "41") - only for v0/v1 dashboards

Example:

grafana_dashboard_migration_conversion_success_total{
  source_version_api="dashboard.grafana.app/v0alpha1",
  target_version_api="dashboard.grafana.app/v1beta1",
  source_schema_version="16",
  target_schema_version="41"
} 1250

2. Dashboard conversion failure metric

Metric Name: grafana_dashboard_migration_conversion_failure_total

Type: Counter

Description: Total number of failed dashboard conversions

Labels:

• source_version_api - Source API version
• target_version_api - Target API version
• source_schema_version - Source schema version (only for v0/v1 dashboards)
• target_schema_version - Target schema version (only for v0/v1 dashboards)
• error_type - Classification of the error (see Error Types section)

Example:

grafana_dashboard_migration_conversion_failure_total{
  source_version_api="dashboard.grafana.app/v0alpha1",
  target_version_api="dashboard.grafana.app/v1beta1",
  source_schema_version="14",
  target_schema_version="41",
  error_type="schema_version_migration_error"
} 42

Error types

The error_type label classifies failures into three categories:

1. conversion_error

• General conversion failures not related to schema migration
• API-level conversion issues
• Programming errors in conversion functions

2. schema_version_migration_error

• Failures during individual schema version migrations (v14→v15, v15→v16, etc.)
• Schema-specific transformation errors
• Data format incompatibilities

3. schema_minimum_version_error

• Dashboards with schema versions below the minimum supported version (< v13)
• These are logged as warnings rather than errors
• Indicates dashboards that cannot be migrated automatically

Logging

Log structure

All migration logs use structured logging with consistent field names:

Base Fields (always present):

• sourceVersionAPI - Source API version
• targetVersionAPI - Target API version
• dashboardUID - Unique identifier of the dashboard being converted

Schema Version Fields (v0/v1 dashboards only):

• sourceSchemaVersion - Source schema version number
• targetSchemaVersion - Target schema version number
• erroredSchemaVersionFunc - Name of the schema migration function that failed (on error)

Error Fields (failures only):

• errorType - Same classification as metrics error_type label
• erroredConversionFunc - Name of the conversion function that failed
• error - The actual error message

Log levels

Success (DEBUG level)

{
  "level": "debug",
  "msg": "Dashboard conversion succeeded",
  "sourceVersionAPI": "dashboard.grafana.app/v0alpha1",
  "targetVersionAPI": "dashboard.grafana.app/v1beta1",
  "dashboardUID": "abc123",
  "sourceSchemaVersion": 16,
  "targetSchemaVersion": 41
}

Conversion/Migration Error (ERROR level)

{
  "level": "error", 
  "msg": "Dashboard conversion failed",
  "sourceVersionAPI": "dashboard.grafana.app/v0alpha1",
  "targetVersionAPI": "dashboard.grafana.app/v1beta1",
  "erroredConversionFunc": "Convert_V0_to_V1",
  "dashboardUID": "abc123",
  "sourceSchemaVersion": 16,
  "targetSchemaVersion": 41,
  "erroredSchemaVersionFunc": "V24",
  "errorType": "schema_version_migration_error",
  "error": "migration failed: table panel plugin not found"
}

Minimum Version Error (WARN level)

{
  "level": "warn",
  "msg": "Dashboard conversion failed", 
  "sourceVersionAPI": "dashboard.grafana.app/v0alpha1",
  "targetVersionAPI": "dashboard.grafana.app/v1beta1",
  "erroredConversionFunc": "Convert_V0_to_V1",
  "dashboardUID": "def456",
  "sourceSchemaVersion": 10,
  "targetSchemaVersion": 41,
  "erroredSchemaVersionFunc": "",
  "errorType": "schema_minimum_version_error",
  "error": "dashboard schema version 10 cannot be migrated"
}

Implementation details

Automatic instrumentation

All dashboard conversions are automatically instrumented via the withConversionMetrics wrapper function:

// All conversion functions are wrapped automatically
s.AddConversionFunc((*dashv0.Dashboard)(nil), (*dashv1.Dashboard)(nil),
    withConversionMetrics(dashv0.APIVERSION, dashv1.APIVERSION, func(a, b interface{}, scope conversion.Scope) error {
        return Convert_V0_to_V1(a.(*dashv0.Dashboard), b.(*dashv1.Dashboard), scope)
    }))

Error handling

Custom error types provide structured error information:

// Schema migration errors
type MigrationError struct {
    msg            string
    targetVersion  int
    currentVersion int
    functionName   string
}

// API conversion errors  
type ConversionError struct {
    msg               string
    functionName      string
    currentAPIVersion string
    targetAPIVersion  string
}

Registration

Metrics registration

Metrics must be registered with Prometheus during service initialization:

import "github.com/grafana/grafana/apps/dashboard/pkg/migration"

// Register metrics with Prometheus
migration.RegisterMetrics(prometheusRegistry)

Available metrics

The following metrics are available after registration:

// Success counter
migration.MDashboardConversionSuccessTotal

// Failure counter  
migration.MDashboardConversionFailureTotal

Conversion matrix

The system supports conversions between all dashboard API versions:

From ↓ / To →	v0alpha1	v1beta1	v2alpha1	v2beta1
v0alpha1	✓	✓	✓	✓
v1beta1	✓	✓	✓	✓
v2alpha1	✓	✓	✓	✓
v2beta1	✓	✓	✓	✓

Each conversion path is automatically instrumented with metrics and logging.

API versions

The supported dashboard API versions are:

• dashboard.grafana.app/v0alpha1 - Legacy JSON dashboard format
• dashboard.grafana.app/v1beta1 - Migrated JSON dashboard format
• dashboard.grafana.app/v2alpha1 - New structured dashboard format
• dashboard.grafana.app/v2beta1 - Enhanced structured dashboard format

Schema versions

Schema versions (v13-v41) apply only to v0alpha1 and v1beta1 dashboards:

• Minimum supported version: v13
• Latest version: v41
• Migration path: Sequential (v13→v14→v15...→v41)

Migration testing

The implementation includes comprehensive test coverage:

• Backend tests: Go migration tests with metrics validation
• Frontend tests: TypeScript conversion tests
• Integration tests: End-to-end conversion validation
• Metrics tests: Prometheus counter validation

Backend migration tests

The backend migration tests validate schema version migrations and API conversions:

• Schema migration tests: Test individual schema version upgrades (v14→v15, v15→v16, etc.)
• Conversion tests: Test API version conversions with automatic metrics instrumentation
• Test data: Uses curated test files from testdata/input/ covering schema versions 14-41
• Metrics validation: Tests verify that conversion metrics are properly recorded

Test execution:

# All backend migration tests
go test ./apps/dashboard/pkg/migration/... -v

# Schema migration tests only
go test ./apps/dashboard/pkg/migration/ -v

# API conversion tests with metrics
go test ./apps/dashboard/pkg/migration/conversion/... -v

Frontend migration comparison tests

The frontend migration comparison tests validate that backend and frontend migration logic produce consistent results:

• Test methodology: Compares backend vs frontend migration outputs through DashboardModel integration
• Dataset coverage: Tests run against 42 curated test files spanning schema versions 14-41
• Test location: public/app/features/dashboard/state/DashboardMigratorToBackend.test.ts
• Test data: Located in apps/dashboard/pkg/migration/testdata/input/ and testdata/output/

Test execution:

# Frontend migration comparison tests
yarn test DashboardMigratorToBackend.test.ts

Test approach:

• Frontend path: jsonInput → DashboardModel → DashboardMigrator → getSaveModelClone()
• Backend path: jsonInput → Backend Migration → backendOutput → DashboardModel → getSaveModelClone()
• Comparison: Direct comparison of final migrated states from both paths

Related documentation

• PR #110178 - Dashboard migration: Add missing metrics registration