Time to Get Your Broken VSAN Back On Track, With Kroll
We all love a Cinderella story of how we created a great storage infrastructure which wowed the customer, and let us show off some really interesting new technology that backs it. But unfortunately, in the world of technology, not every story is a Cinderalla story. Imagine when the story takes a dark turn for a perfecly valid reason: hardware failure.
VSAN is Cool, not Perfect
Nothing is a perfect solution. There is always a failure scenario that can take down even the most robust of solutions. Hardware and software are not built to perfection, but they have safety measures built in for us to be able to architect a well protected deployment.
We’ve talked in the past about the relatively well known VSAN Nightmare as it was dubbed. The reality of the situation was highlighted as a hardware queue depth issue which actually resulted in the modification of the VSAN HCL to remove unsupported controllers.
Recovering from Severe Data Loss
It is the worst case scenario. Despite all of our protective layers, we have a catastrophic hardware or software error that results in complete data loss. Then we go to the backup tapes and realize our worst nightmare has come true; nobody has tested the backups.
VMware VSAN has many backup tools which are compatible with it, however it is only as good as the process that you wrap around it. Luckily there is the one last hope for total data loss in your environment which is the tools for near to forensic recovery.
Kroll has stepped up with their OnTrack product and in a real-life situation, they were able to recover data lost from a VMware VSAN environment that was spread across 15 magnetic disks and 3 SSD drives.
In extreme cases like this, there can be significant challenges recovering data where there is striping of content across multiple disks to create aggregated space for server environments. With a successful history in traditional single disk data recovery, Kroll has done well to make the jump into providing a production proven recover from a failed VSAN environment.
While rare, the reality of data loss in any system has to be considered as a potential result. It does lead us to do as much as possible to prevent such situations by designing the best distributed storage solutions across our storage environments. For those extreme situations though, you can lean on the last resort which is solutions such as the Kroll OnTrack product in this case.
Kroll had some press on the results of their work with VMware VSAN recovery here that is well worth a read. Let’s just hope that we don’t need to test it out 🙂