Skip to main content

Error Reference

rabbitmq-backup uses a structured error type with the following categories. Each error message includes a prefix indicating the category (e.g. AMQP error:, Storage error:). This page documents common errors, their causes, and solutions.

Error Categories

CategoryError PrefixDescription
AMQPAMQP error:AMQP 0-9-1 protocol errors during backup or restore.
StreamStream protocol error:RabbitMQ Stream Protocol errors.
Management APIManagement API error:HTTP API errors when exporting/importing definitions or discovering queues.
StorageStorage error:Object storage read/write/list failures.
ConfigurationConfiguration error:Invalid or missing configuration.
SerializationSerialization error:JSON/YAML parse errors or segment format errors.
CheckpointCheckpoint error:SQLite offset store errors.
CompressionCompression error:zstd/LZ4 compression or decompression failures.
IOIO error:Low-level filesystem I/O errors.
ConnectionConnection error:TCP connection establishment failures.
AuthenticationAuthentication error:AMQP or Management API authentication failures.
ManifestManifest error:Backup manifest parsing or validation errors.

AMQP Errors

Connection Refused

Connection error: Failed to connect to amqp://guest:***@localhost:5672/%2f

Cause: The RabbitMQ broker is not running, not reachable, or not listening on the specified port.

Solutions:

  • Verify the broker is running: rabbitmqctl status or docker ps
  • Check the hostname and port in source.amqp_url or target.amqp_url
  • Check firewall rules and network connectivity
  • If using Docker, ensure the port is mapped: -p 5672:5672

Authentication Failed

Authentication error: AMQP authentication failed for user "guest"

Cause: Invalid username or password in the AMQP URL, or the user does not have permission to access the specified vhost.

Solutions:

  • Verify credentials: rabbitmqctl authenticate_user <user> <pass>
  • Check vhost permissions: rabbitmqctl list_permissions -p <vhost>
  • The default guest user can only connect from localhost unless configured otherwise
  • Check the vhost encoding in the URL (use %2f for the default / vhost)

Channel Error: Queue Not Found

AMQP error: NOT_FOUND - no queue 'orders' in vhost '/'

Cause: The specified queue does not exist on the broker. This can occur during restore if definitions were not restored first.

Solutions:

  • Ensure restore.restore_definitions: true when restoring to a fresh cluster
  • Set restore.create_missing_queues: true with restore.publish_mode: direct-to-queue and target Management API credentials to auto-create missing queues
  • Verify the queue exists: rabbitmqctl list_queues -p <vhost>

Channel Error: Access Refused

AMQP error: ACCESS_REFUSED - queue 'orders' in vhost '/' owned by another connection

Cause: The queue has an exclusive consumer or is owned by another connection.

Solutions:

  • Check for existing consumers: rabbitmqctl list_consumers -p <vhost>
  • Close the exclusive consumer before backup
  • Wait and retry -- the exclusive lock may be temporary

Connection Reset / Heartbeat Timeout

Connection error: connection reset by peer

Cause: The broker closed the connection, typically due to a heartbeat timeout or resource alarm.

Solutions:

  • Check broker logs for resource alarms: rabbitmqctl status
  • Increase the heartbeat timeout on the broker side
  • Check if the broker is under memory or disk pressure
  • Use -vv to enable trace logging for frame-level debugging

Stream Protocol Errors

Stream Connection Failed

Stream protocol error: Failed to connect: StreamError(Authentication)

Cause: Cannot connect to the RabbitMQ Stream Protocol endpoint.

Solutions:

  • Verify the rabbitmq_stream plugin is enabled: rabbitmq-plugins enable rabbitmq_stream
  • Check source.stream_port (default: 5552)
  • Verify the port is exposed if using Docker: -p 5552:5552
  • Check that the credentials match the stream user

Consumer Creation Failed

Stream protocol error: Failed to create consumer for events-stream: StreamDoesNotExist

Cause: The specified stream queue does not exist.

Solutions:

  • Verify the stream exists: rabbitmqctl list_queues name type | grep stream
  • Check that source.queues.types includes stream
  • Ensure the queue was declared with x-queue-type: stream

Management API Errors

Connection Refused

Management API error: error sending request for url (http://localhost:15672/api/overview)

Cause: The Management API is not reachable.

Solutions:

  • Verify the rabbitmq_management plugin is enabled: rabbitmq-plugins enable rabbitmq_management
  • Check source.management_url or target.management_url
  • The Management API default port is 15672 (or 15671 for HTTPS)
  • Check firewall rules

HTTP 401 Unauthorized

Management API error: HTTP 401 for GET /api/overview

Cause: Invalid Management API credentials.

Solutions:

  • Verify management_username and management_password
  • Check user tags: the user needs the monitoring or administrator tag
  • Verify with curl: curl -u user:pass http://host:15672/api/overview

HTTP 403 Forbidden

Management API error: HTTP 403 for POST /api/definitions

Cause: The user does not have sufficient permissions to import definitions.

Solutions:

  • The user needs the administrator tag to import definitions
  • Check user tags: rabbitmqctl list_users
  • Add the tag: rabbitmqctl set_user_tags <user> administrator

HTTP 500 Internal Server Error

Management API error: HTTP 500 for POST /api/definitions

Cause: The definitions JSON contains invalid or conflicting entries that the broker cannot apply.

Solutions:

  • Validate the definitions file with a dry run: restore.definitions_dry_run: true
  • Check for conflicts with existing objects on the target cluster
  • Check broker logs: rabbitmqctl log_tail

Storage Errors

S3: Access Denied

Storage error: Failed to put backup-001/manifest.json: 403 Forbidden

Cause: The IAM credentials do not have permission to write to the S3 bucket.

Solutions:

  • Verify IAM policy grants s3:PutObject, s3:GetObject, s3:ListBucket
  • Check that access_key and secret_key are correct
  • If using IAM roles (EC2/ECS/EKS), verify the instance profile or service account
  • For MinIO, check the bucket policy

S3: Bucket Not Found

Storage error: Failed to put backup-001/manifest.json: NoSuchBucket

Cause: The specified S3 bucket does not exist.

Solutions:

  • Verify the bucket name in the storage configuration
  • Create the bucket: aws s3 mb s3://bucket-name
  • Check the region is correct

S3: Endpoint Connection Error

Storage error: Failed to create S3 backend: endpoint connection error

Cause: Cannot reach the S3 endpoint.

Solutions:

  • For MinIO: verify endpoint URL and that MinIO is running
  • For MinIO: set path_style: true and allow_http: true (if using HTTP)
  • For AWS: check region and network connectivity

Azure: Container Not Found

Storage error: Failed to put backup-001/manifest.json: ContainerNotFound

Cause: The Azure Blob container does not exist.

Solutions:

  • Create the container in the Azure portal or with az storage container create
  • Verify container_name in the storage configuration

GCS: Permission Denied

Storage error: Failed to create GCS backend: 403 Forbidden

Cause: The service account does not have storage permissions.

Solutions:

  • Grant the Storage Object Admin role to the service account
  • Verify GOOGLE_APPLICATION_CREDENTIALS points to a valid key file
  • Check that the bucket exists and is in the correct project

Filesystem: Permission Denied

Storage error: Failed to write backup-001/manifest.json: Permission denied

Cause: The process does not have write permission to the filesystem path.

Solutions:

  • Check directory permissions: ls -la /path/to/storage
  • Run with appropriate user permissions
  • Create the directory and set ownership: mkdir -p /path && chown user:group /path

Key Not Found

Storage error: Key not found: backup-001/manifest.json

Cause: The requested backup does not exist in storage, or the backup_id is wrong.

Solutions:

  • Use rabbitmq-backup list --path <storage> to see available backups
  • Verify the backup_id in the configuration
  • Check the prefix in the storage configuration

Configuration Errors

Missing Required Field

Configuration error: Source configuration required for backup

Cause: A required configuration section is missing for the specified mode.

Solutions:

  • For mode: backup: ensure the source section is present
  • For mode: restore: ensure the target section is present
  • Check that the YAML is valid (proper indentation, no tabs)

Invalid YAML

Serialization error: invalid type: found string, expected struct Config at line 5 column 3

Cause: The YAML configuration file has a syntax or type error.

Solutions:

  • Validate YAML syntax with a linter: yamllint config.yaml
  • Check indentation (YAML uses spaces, not tabs)
  • Ensure field names match the expected schema (see Configuration Reference)
  • Check that enum values are lowercase: backup not Backup, zstd not ZSTD

Invalid Storage URL

Configuration error: Invalid storage URL: relative URL without a base

Cause: The --path argument to list, describe, or validate is not a valid URL.

Solutions:

  • Use full URL format: s3://bucket, file:///path, gcs://bucket
  • For local paths, use file:///absolute/path
  • Check for typos in the URL scheme

Unknown Storage Scheme

Configuration error: Unknown storage scheme: ftp

Cause: An unsupported URL scheme was used.

Solutions:

  • Supported schemes: s3, s3a, azure, az, gcs, gs, file, memory

Segment / Serialization Errors

Invalid Segment Magic

Serialization error: Invalid segment magic bytes

Cause: The file does not start with the RBAK magic bytes. The file may be corrupted, truncated, or not an RBAK segment.

Solutions:

  • Re-run the backup to regenerate the segment
  • Check for storage corruption or incomplete uploads
  • Use validate --deep to check all segments

CRC32 Mismatch

Serialization error: CRC32 mismatch: expected 1234567890, got 9876543210

Cause: The segment data has been corrupted in transit or at rest. The CRC32 of the header + payload does not match the stored checksum.

Solutions:

  • Re-run the backup to regenerate the segment
  • Check storage integrity (S3 data integrity, filesystem health)
  • If using network storage, check for network-level corruption

Unsupported Segment Version

Serialization error: Unsupported segment version: 2

Cause: The segment was created by a newer version of rabbitmq-backup that uses a different format version.

Solutions:

  • Upgrade rabbitmq-backup to the version that created the backup
  • Check backup_tool_version in manifest.json

Record Count Mismatch

Serialization error: Expected 1000 records but found 999

Cause: The number of records parsed from the decompressed payload does not match the count in the segment header. Indicates data corruption.

Solutions:

  • Re-run the backup
  • Check compression library compatibility

Checkpoint Errors

SQLite Open Failed

Checkpoint error: unable to open database file: ./offsets.db

Cause: Cannot create or open the SQLite checkpoint database.

Solutions:

  • Check write permissions for the db_path directory
  • Ensure the parent directory exists
  • Check disk space

Troubleshooting Tips

Enable Debug Logging

Use -v for debug level or -vv for trace level:

rabbitmq-backup backup -vv --config backup.yaml

Trace logging shows individual AMQP frames, storage operations, and segment details.

Override Log Level with RUST_LOG

For module-specific logging:

RUST_LOG=rabbitmq_backup_core::amqp=trace,rabbitmq_backup_core::storage=debug \
rabbitmq-backup backup --config backup.yaml

Validate Before Restore

Always validate a backup before restoring, especially after copying between storage backends:

rabbitmq-backup validate --path s3://bucket --backup-id backup-001 --deep

Dry-Run Restore

Test the restore plan without publishing messages:

restore:
dry_run: true
rabbitmq-backup restore --config restore-dryrun.yaml