When we launched our new stage env, there's a bit of manual ssh-in process required to get the nodes to a functional state. We should minimize that as much as possible. Some ideas include 2 way syncs to S3 techops bucket, occasional AMI snapshots which update the consul sg.

https://pypi.python.org/pypi/consulate might be helpful here.

I think the easy thing would be: 1) on a regular basis (hourly? nightly?) run "consulate kv backup backup-${DATE}.json" 2) copy to S3 - "aws s3 cp backup-${DATE}.json s3://some-backup-bucket/consul/" This would require allowing the consul servers to write to a backup bucket.