Cluster Migration

Use rabbitmq-backup to migrate messages and definitions from one RabbitMQ cluster to another. This is useful when upgrading to a new version, moving between cloud providers, or consolidating clusters.

Migration Overview

Old Cluster                                    New Cluster
+-----------------+                           +-----------------+
|  RabbitMQ 3.12  |                           |  RabbitMQ 4.0   |
|  (source)       |                           |  (target)       |
+-----------------+                           +-----------------+
        |                                             ^
        v                                             |
+------------------+       +---------+       +------------------+
| rabbitmq-backup  |  -->  |   S3    |  -->  | rabbitmq-backup  |
|    backup        |       | Storage |       |    restore       |
+------------------+       +---------+       +------------------+

Step 1: Back Up the Source Cluster

Create a full backup of the source cluster including definitions:

migration-backup.yaml
mode: backup
backup_id: "migration-001"

source:
  amqp_url: "amqp://admin:${OLD_PASSWORD}@old-rabbitmq:5672/%2f"
  management_url: "http://old-rabbitmq:15672"
  management_username: admin
  management_password: "${OLD_PASSWORD}"
  queues:
    include:
      - "*"

storage:
  backend: s3
  bucket: rabbitmq-migration
  region: us-east-1
  prefix: migration/

backup:
  compression: zstd
  compression_level: 3
  prefetch_count: 200
  max_concurrent_queues: 4
  include_definitions: true
  stop_at_current_depth: true

rabbitmq-backup backup --config migration-backup.yaml

Step 2: Verify the Backup

# List the backup
rabbitmq-backup list --path s3://rabbitmq-migration

# Describe to confirm contents
rabbitmq-backup describe \
  --path s3://rabbitmq-migration \
  --backup-id migration-001 \
  --format json

# Deep validation
rabbitmq-backup validate \
  --path s3://rabbitmq-migration \
  --backup-id migration-001 \
  --deep

Step 3: Restore to the New Cluster

Simple Restore (Same Topology)

If the new cluster should mirror the old one exactly:

migration-restore.yaml
mode: restore
backup_id: "migration-001"

target:
  amqp_url: "amqp://admin:${NEW_PASSWORD}@new-rabbitmq:5672/%2f"
  management_url: "http://new-rabbitmq:15672"
  management_username: admin
  management_password: "${NEW_PASSWORD}"

storage:
  backend: s3
  bucket: rabbitmq-migration
  region: us-east-1
  prefix: migration/

restore:
  restore_definitions: true
  publish_mode: exchange
  publisher_confirms: true
  max_concurrent_queues: 4
  produce_batch_size: 100

rabbitmq-backup restore --config migration-restore.yaml

Restore with Remapping

If the new cluster uses different queue names, vhosts, or exchanges, use the mapping options:

migration-restore-remapped.yaml
mode: restore
backup_id: "migration-001"

target:
  amqp_url: "amqp://admin:${NEW_PASSWORD}@new-rabbitmq:5672/%2f"
  management_url: "http://new-rabbitmq:15672"
  management_username: admin
  management_password: "${NEW_PASSWORD}"

storage:
  backend: s3
  bucket: rabbitmq-migration
  region: us-east-1
  prefix: migration/

restore:
  restore_definitions: true
  publish_mode: exchange
  publisher_confirms: true

  # Remap vhosts
  vhost_mapping:
    "/": "production"
    "staging": "staging-v2"

  # Remap queues
  queue_mapping:
    orders-queue: orders-queue-v2
    payments-queue: payments-queue-v2

  # Remap exchanges
  exchange_mapping:
    orders-exchange: orders-exchange-v2
    payments-exchange: payments-exchange-v2

rabbitmq-backup restore --config migration-restore-remapped.yaml

Step 4: Verify the Migration

Compare Definitions

Export definitions from both clusters and compare:

# Export from old cluster
rabbitmq-backup definitions-export --config old-config.yaml --output old-defs.json

# Export from new cluster
rabbitmq-backup definitions-export --config new-config.yaml --output new-defs.json

# Compare (ignoring user password hashes)
diff <(jq 'del(.users[].password_hash)' old-defs.json) \
     <(jq 'del(.users[].password_hash)' new-defs.json)

Compare Message Counts

# Old cluster
curl -s -u admin:old_pass http://old-rabbitmq:15672/api/queues | \
  jq '[.[] | {name: .name, messages: .messages}]'

# New cluster
curl -s -u admin:new_pass http://new-rabbitmq:15672/api/queues | \
  jq '[.[] | {name: .name, messages: .messages}]'

Migration Strategies

Strategy 1: Stop-and-Migrate (Simple)

Best for maintenance windows where downtime is acceptable.

Stop all producers and consumers
Wait for in-flight messages to settle
Back up the source cluster
Restore to the target cluster
Point applications to the new cluster
Start producers and consumers

Strategy 2: Live Migration (Minimal Downtime)

Ongoing backup: Run continuous scheduled backups of the source cluster
Restore definitions: Import definitions to the target cluster ahead of time
Cutover window: Stop producers briefly
Final backup: Run one last backup to capture remaining messages
Restore messages: Restore the final backup to the target cluster
Switch traffic: Point applications to the new cluster

# Pre-migration: import definitions
rabbitmq-backup definitions-import --config new-config.yaml --input definitions.json

# Cutover: final backup
rabbitmq-backup backup --config migration-backup.yaml

# Cutover: restore messages only (definitions already imported)
rabbitmq-backup restore --config migration-restore-messages-only.yaml

migration-restore-messages-only.yaml
restore:
  restore_definitions: false  # Already imported
  publish_mode: exchange
  publisher_confirms: true

Strategy 3: Selective Migration

Migrate only specific queues or vhosts:

source:
  queues:
    include:
      - "orders-*"
      - "payments-*"
    vhosts:
      - "production"

Cross-Version Migration

rabbitmq-backup works across RabbitMQ versions because:

It uses the stable AMQP 0-9-1 protocol for messages
It uses the Management HTTP API for definitions (compatible across versions)
Message payloads are stored as opaque bytes with all AMQP properties preserved

Tested migration paths:

Source	Target	Status
3.8.x	3.12.x	Supported
3.12.x	4.0.x	Supported
3.8.x	4.0.x	Supported

caution

When migrating from 3.x to 4.x, review RabbitMQ's deprecation notices. Some policy settings or queue arguments may have changed. Import definitions with definitions_dry_run: true first to check for issues.

Rollback

If the migration fails, roll back by restoring the backup to the original cluster:

rollback-restore.yaml
mode: restore
backup_id: "migration-001"

target:
  amqp_url: "amqp://admin:${OLD_PASSWORD}@old-rabbitmq:5672/%2f"
  management_url: "http://old-rabbitmq:15672"
  management_username: admin
  management_password: "${OLD_PASSWORD}"

storage:
  backend: s3
  bucket: rabbitmq-migration
  region: us-east-1
  prefix: migration/

restore:
  restore_definitions: false  # Topology still exists on old cluster
  publish_mode: exchange
  publisher_confirms: true

Migration Overview​

Step 1: Back Up the Source Cluster​

Step 2: Verify the Backup​

Step 3: Restore to the New Cluster​

Simple Restore (Same Topology)​

Restore with Remapping​

Step 4: Verify the Migration​

Compare Definitions​

Compare Message Counts​

Migration Strategies​

Strategy 1: Stop-and-Migrate (Simple)​

Strategy 2: Live Migration (Minimal Downtime)​

Strategy 3: Selective Migration​

Cross-Version Migration​

Rollback​