Database Corruption and Recovery

[Tony Ferraro]

Has anyone using Automic Workload / UC4 ever experienced a complete database corruption?

My production DB is Oracle and runs on several hosts. The data is also replicated to a DR/BCP database in real time.

My theoretical scenario is what if the production DB or DB data gets badly corrupted by any means(software bug, manual SQL change, malicious intent, just bad luck).
The corruption would be replicated to the DR DB instance immediately.

We do daily backups, but restoring a prod DB with data 24 hours old would be a major crisis. I could do hourly backups.

So my questions are- 
   How likely is this scenario? 
   Has anyone experienced this?

Though I use Oracle, but my question applies to any DB.

Thanks!
Tony.

[Carsten Schmitz]

​We have tested this scenario as part of a desaster recovery test, and also once had to perform a complete db recovery from backups when the transport case (which was enabled in an automated fashion by my predecessor) malfunctioned and the loader job wiped all objects from a staging installation db.

How likely is db corruption? I don't think anyone can really provide dependable figures, and it depends on many factors, but my advise is to consider the potential impact, then decide with all stakeholders how much data loss (and downtime!) can be accepted at most. Our DBA can roll back databases with a granularity of 1h (I'm no expert but the terms "rman" and "redo logs" are mentioned), so foremost you might want to look into possibilities of lowering your restore scenario from 24h down, if the installation is vital enough.

Depending on the industry, DR tests might even be practically mandatory (SOX act etc.), e.g. in finance.

Best,
Carsten

[Pete Wirfs]

We depend upon hourly backups, and run our DR recovery tests every year.  Our DBAs assure us they could also apply database transaction logs to a specific point in time, if necessary.

We've had several catastrophic data center power outages, and no database corruption occurred.  We just started the servers back up and we were back in business.  So it seems highly unlikely to me.