Skip to main content

Debug Mode

When something goes wrong, rabbitmq-backup provides multiple levels of logging to help diagnose the issue.

Verbosity Flags​

The -v flag is global and can be applied to any subcommand.

Default (Info Level)​

Normal output showing high-level progress:

rabbitmq-backup backup --config backup.yaml
INFO  Starting backup my-backup-001
INFO  Connected to RabbitMQ at localhost:5672
INFO  Discovered 5 queues matching selection criteria
INFO  Backup my-backup-001 completed: 3,500 messages in 4 segments

Debug Level (-v)​

Adds per-queue details, connection events, and configuration details:

rabbitmq-backup backup -v --config backup.yaml
DEBUG Config loaded: mode=backup, backup_id=my-backup-001
DEBUG Source: amqp://guest:***@localhost:5672/%2f
DEBUG Storage: S3 bucket=rabbitmq-backups, region=us-east-1
INFO  Starting backup my-backup-001
DEBUG Opening AMQP connection to localhost:5672
INFO  Connected to RabbitMQ at localhost:5672
DEBUG Fetching queue list from Management API
DEBUG Found queue: orders-queue (classic, 1234 messages, vhost=/)
DEBUG Found queue: payments-queue (classic, 567 messages, vhost=/)
DEBUG Starting consumer on orders-queue with prefetch=100
DEBUG Segment buffer: 0/134217728 bytes
INFO  Segment segment-0001.zst written (1,234 records, 2.3 MB)
DEBUG Checkpoint synced to S3: state/offsets.db
INFO  Backup my-backup-001 completed: 1,801 messages in 2 segments

Trace Level (-vv)​

Maximum detail including individual message handling, frame-level AMQP events, and storage API calls:

rabbitmq-backup backup -vv --config backup.yaml
TRACE AMQP frame: Connection.Start received
TRACE AMQP frame: Connection.Tune received (channel_max=2047, frame_max=131072)
TRACE AMQP frame: Connection.Open-Ok received
DEBUG Opening AMQP connection to localhost:5672
TRACE Basic.Consume sent for queue=orders-queue, consumer_tag=rmq-backup-001
TRACE Basic.Deliver received: delivery_tag=1, exchange=orders-exchange, routing_key=orders
TRACE Recording message: delivery_tag=1, size=1024, timestamp=1705312800000
TRACE Basic.Deliver received: delivery_tag=2, exchange=orders-exchange, routing_key=orders
...
TRACE PUT s3://rabbitmq-backups/prod/my-backup-001/queues/%2f/orders-queue/segment-0001.zst (2.3 MB)
TRACE S3 response: 200 OK, ETag="abc123"
caution

Trace-level logging produces very high output volume. Use it only for targeted debugging, not in production.

RUST_LOG Environment Variable​

For fine-grained control over logging, use the RUST_LOG environment variable. It takes priority over the -v flag.

Set a Global Level​

RUST_LOG=debug rabbitmq-backup backup --config backup.yaml

Filter by Module​

Target specific modules to reduce noise:

# Only show debug logs from the backup module
RUST_LOG=rabbitmq_backup_core::backup=debug rabbitmq-backup backup --config backup.yaml

# Debug storage, trace AMQP, info for everything else
RUST_LOG=info,rabbitmq_backup_core::storage=debug,rabbitmq_backup_core::amqp=trace rabbitmq-backup backup --config backup.yaml

Useful Module Filters​

FilterWhat It Shows
rabbitmq_backup_core=debugAll core library debug output
rabbitmq_backup_core::backup=debugBackup engine details
rabbitmq_backup_core::restore=debugRestore engine details
rabbitmq_backup_core::amqp=traceAMQP frame-level tracing
rabbitmq_backup_core::stream=debugStream Protocol activity
rabbitmq_backup_core::storage=debugStorage read/write operations
rabbitmq_backup_core::segment=debugSegment writer details
rabbitmq_backup_core::definitions=debugDefinitions export/import
rabbitmq_backup_core::metrics=debugMetrics registration
rabbitmq_backup_core::offset_store=debugCheckpoint activity

Combine Modules​

RUST_LOG="info,\
rabbitmq_backup_core::backup=debug,\
rabbitmq_backup_core::storage=debug,\
rabbitmq_backup_core::amqp=debug" \
rabbitmq-backup backup --config backup.yaml

Logging Priority​

The logging level is determined by this priority order:

  1. RUST_LOG environment variable (highest priority)
  2. -v / -vv CLI flags
  3. Default: info

Reading Trace Output​

Connection Events​

DEBUG Opening AMQP connection to localhost:5672
INFO  Connected to RabbitMQ at localhost:5672

If the connection fails, you will see:

ERROR Connection error: Connection refused (os error 111)

Queue Discovery​

DEBUG Fetching queue list from Management API
DEBUG Found queue: orders-queue (classic, 1234 messages, vhost=/)
DEBUG Skipping queue: dlx-queue (matches exclude pattern '*-dead-letter')

If no queues match, you will see:

WARN  No queues matched the selection criteria

Segment Writing​

DEBUG Segment buffer: 65536/134217728 bytes (0.05%)
INFO  Segment segment-0001.zst written (1,234 records, 2.3 MB compressed from 8.1 MB)

Storage Operations​

TRACE PUT s3://bucket/prefix/backup-001/queues/.../segment-0001.zst
TRACE S3 response: 200 OK

If storage fails:

ERROR Storage error: Access Denied (Service: Amazon S3; Status Code: 403)

Checkpoint Syncs​

DEBUG Checkpoint: saved offset queue=orders-queue delivery_tag=1234
DEBUG Checkpoint synced to S3: state/offsets.db

Logging to a File​

Redirect stderr (where logs are written) to a file:

rabbitmq-backup backup -v --config backup.yaml 2> backup.log

Or capture both stdout and stderr:

rabbitmq-backup backup -v --config backup.yaml > backup.log 2>&1

Systemd Journal​

When running as a systemd service, view logs with:

sudo journalctl -u rabbitmq-backup.service -f
sudo journalctl -u rabbitmq-backup.service --since "1 hour ago"

Kubernetes Logs​

kubectl logs -n rabbitmq-backup job/rabbitmq-backup-manual -f
kubectl logs -n rabbitmq-backup job/rabbitmq-backup-manual --since=1h

Debugging Specific Issues​

Connection Issues​

RUST_LOG="rabbitmq_backup_core::amqp=trace" rabbitmq-backup backup -v --config backup.yaml

Storage Issues​

RUST_LOG="rabbitmq_backup_core::storage=trace" rabbitmq-backup backup -v --config backup.yaml

Restore Issues​

RUST_LOG="rabbitmq_backup_core::restore=debug,rabbitmq_backup_core::segment=debug" \
  rabbitmq-backup restore -v --config restore.yaml

Config Parsing Issues​

Use -v to see the parsed configuration:

rabbitmq-backup backup -v --config backup.yaml 2>&1 | head -20