Power Loss and Database Recovery…

 

About a week ago we had a big power outage at work. What with UPS and generators, most systems coped fine, but one system didn’t fair so well…

One of the mount points, and some of the data files within it, got corrupted and the Solaris container holding the database decided to repeatedly bounce between nodes. After a few days work on the part of the sysadmins, including a file system restore to repair the Oracle software installation, the server was given back to me.

A DBVerify showed there were loads of block corruptions in the data files, so I did a full restore and recovery of the database. The app was released for testing to the users this morning. A little before midday we had another power outage, even more severe than the previous one. Some of the UPS had not fully recharged since the last outage…

We will not get power restored until this evening. Everyone has an early start planned tomorrow. We have no idea what we are walking in to though!

The moral of this story is always have a good DR strategy!

Fun, fun, fun…

Cheers

Tim…

PS. The other DBA is on holiday this week. LOL.

Update: The DB was mangled again so we had to restore it again. 🙁

Author: Tim...

DBA, Developer, Author, Trainer.

2 thoughts on “Power Loss and Database Recovery…”

Comments are closed.