Here is one way to add some data redundancy to your Jenkins instance.  The solution below won’t auto recover, but the data itself should be safe if your active Jenkins master fails.

The following are some key points not really mentioned in the beautiful Google Docs created chart above!

1. A dedicated file system is created on the NAS that is attached to each Jenkins master server
   • The NAS has snapshot backups enabled for X number of days on this file system
2. The same user and group is created on both master servers
   • The group ID must be the same on both master servers
     • groupadd -g <unique_id> <group_name>
3. The new user’s home dir should be on the NAS under the dedicated file system and the user should own it
   • /etc/passwd – home dir
   • chown – change ownership
4. Tomcat is installed from 1 master server to the NAS
5. Jenkins is installed from 1 master server to the NAS
6. Tomcat is only kept running from 1 master server
7. A load balancer like HAProxy can be used for active\passive load balancing

Some other recovery recommendations.

• Store all Jenkins job scripts in Git and setup your jobs to pull the latest scripts from Git a execution time
• Implement the Sync Jenkins plugin.  This plugin will push all job\config changes to Git for easy recovery.
  • https://wiki.jenkins.io/display/JENKINS/SCM+Sync+configuration+plugin