Facebook Glitch Loses Photos

For Facebook users who might have noticed that some previously-uploaded photos were mysteriously missing from their Facebook pages, Facebook wants you to know that it is aware of the problem, it is fixing the problem right now, and perhaps most importantly: "Don't worry: Your photos are safe."

Apparently, while Facebook engineers were performing a "routine software upgrade" this last Friday night, several hard drives that store user photos experienced "simultaneous hardware failures." The Facebook engineers are still a bit baffled by this, as such multiple, simultaneous drive failures are not very common. (Note to Facebook engineers: Make sure you check the drive controllers for hardware failures too.) The upshot of the failure of the drives is that roughly 10 to 15-percent of user photos got unceremoniously wiped, resulting in photos not appearing or question marks appearing where the photo should be.

 
 Example of a missing image from a user's photos.
The photos might be missing from the failed drives, but they are not gone forever. Facebook claims that they "still have all your photos because we store them in a way that maintains multiple copies of the data in case of hardware failures like this." Whether this redundancy is in form of RAID arrays, backups, or some combination of the two, it will still take Facebook some time to restore the affected photos back to the site. Not only do the Facebook engineers have to restore massive amounts of data, they also need to repair and or replace the failed hardware and make sure everything is once again operational.

"We're restoring photos as we repair the hard drives, so some should be working again today and we should be back to normal by early next week. New photo uploads will continue to work properly during the repairs, because we write them to different storage volumes. Thanks for bearing with us while we return things to normal."

This is by no means the first time that users of a social networking site have been impacted by lost data. This last January, JournalSpace.com experienced such a catastrophic failure that it was unable to retrieve the lost data and JournalSpace's owner chose to close up shop and sell the JournalSpace.com and JournalSpace.net domains. (JournalSpace still exists, albeit, in a new iteration, and with a new owner.) Similarly, last month Nokia experienced a failure on a server that stored user information for its Contacts On Ovi service, and wound up losing three weeks of user data.

Facebook's recent glitch is not as bad as what befell JournalSpace or Nokia, but because of Facebook's much larger reach, far more users were impacted. Pundits are claiming that these events should serve as warnings against too much of a reliance on cloud services; but when these cloud-based services are the best means for social networking, it doesn't leave users with many alternatives.
Tags:  Facebook, Failure