setup backups for new generic and bugzilla servers in scl3

cc'ing puppet masters.... At first I had: err: Failed to apply catalog: Cannot alias File[/data/] to ["/data"] at /etc/puppet/modules/mysql2/manifests/backups.pp:27; resource ["File", "/data"] already declared at /etc/puppet/modules/base/manifests/files.pp:12 But then I commented some stuff out in the manifests/nodes/databases.pp file for backups3, and got a bit farther, but I can't get it to actually install the Percona server packages: info: Applying configuration version '54961' err: /Stage[main]/Mysql2::Backups/File[/var/run/mysqld/]: Could not evaluate: Could not find user mysql err: /Stage[main]/Mysql2::Backups/Mysql2::Backups::Cluster[buildbot]/File[/data/buildbot/]: Could not evaluate: Could not find user mysql err: /Stage[main]/Mysql2::Backups/Mysql2::Backups::Cluster[buildbot]/File[/data/backups/buildbot/]: Could not evaluate: Could not find user mysql notice: /Stage[main]/Mysql2::Backups/Mysql2::Backups::Cluster[buildbot]/File[/data/backups/buildbot/rawcopies/]: Dependency File[/data/backups/buildbot/] has failures: true warning: /Stage[main]/Mysql2::Backups/Mysql2::Backups::Cluster[buildbot]/File[/data/backups/buildbot/rawcopies/]: Skipping because of failed dependencies notice: /Stage[main]/Mysql2::Backups/Mysql2::Backups::Cluster[buildbot]/File[/data/backups/buildbot/sqldumps/]: Dependency File[/data/backups/buildbot/] has failures: true warning: /Stage[main]/Mysql2::Backups/Mysql2::Backups::Cluster[buildbot]/File[/data/backups/buildbot/sqldumps/]: Skipping because of failed dependencies notice: Finished catalog run in 57.32 seconds [root@backup3.db.scl3 ~]# rpm -qa | grep -i mysql [root@backup3.db.scl3 ~]# rpm -qa | grep -i percona [root@backup3.db.scl3 ~]# And in fact, the yum repo isn't set up :( [root@backup3.db.scl3 ~]# yum install Percona-Server-server-51-5.1.66-rel14.1.496.rhel6.x86_64 Loaded plugins: downloadonly, rhnplugin, security Setting up Install Process No package Percona-Server-server-51-5.1.66-rel14.1.496.rhel6.x86_64 available. Error: Nothing to do I tried including settings.pp in the backups.pp of modules/mysql2 but that hasn't helped.

mysql2::backups doesn't actually install MySQL, which is at least *why* this is happening.

This is for backup3.db.scl3.mozilla.com. Let's make this a MySQL 5.5 server. Once the backups server is set up with packages installed via puppet., the first 2 backups on it should be generic and bugzilla in scl3 (which are both MySQL 5.5). Here's the checklist for that: https://checkmarkable.com/team/cowlytoft/checklist/915/dashboard

Note that the backup server is successfully backing up buildbot and stage. backups are not 100% puppetized but some improvements have been made (there's a different bug for that; I think there are only a few things left to do, like puppetizing the my.cnf files). The checklist may be redundant in some places (e.g. no need to make a startup script) which you can flag in checkmarkable so it can be changed. It should be trivial now to set up these backups.

Just a note here...Bugzilla backups need to be online next week so we can have enough tests with them before we go live on March 8th. Would be nice if we could do a test restore too from this backup and ensure things are okay.

backups will be live today, and I'll make sure to do a restore test. I leave Saturday for vacation.

backups for bugzilla are live and a restore was tested. They work excellently! generic failed several refreshes due to a replication duplicate key error I didn't get to investigate yet.

I tried and got the same problems.....I skipped the replication duplicate key errors (there were many) just to get replication working and caught up. We'll need to check checksums on it, but that's another bug (getting checksum checking working on backups).