Data Center Leaders: Business Continuity & Disaster Recovery With John “Traenk” Traenkenschuh, Part 1

Business Continuity & Disaster Recovery Expert John “Traenk” Traenkenschuh

In a recent interview with Lawrence Webber, we discussed the ‘hows’ and ‘whys’ of business continuity and disaster recovery planning.  This week, our Data Center Leaders Interview Series drives home the importance of this topic during our two part interview with John “Traenk” Traenkenschuh.

Budding author, book editor, and Information Technology worker at three Fortune 100 companies, John “Traenk” Traenkenschuh’s insight into business continuity and disaster recovery planning comes courtesy of years of real world experience.

For his time spent introducing students to Microsoft’s Visual Basic, Traenk has been awarded the Microsoft Most Valuable Professional (MVP) designation since 2004.   He has also authored  VCP VMware 310 Cert Flash Cards as a late stage exam preparation tool.

In part one of our interview below, Traenk demonstrates the value of business continuity and disaster recovery planning through a look at both the evolution and future of  these solutions:

Evolving Solutions:
You have extensive experience in the fields of disaster recovery and business continuity.  How would you describe the evolution of recovery and continuity solutions since you first entered IT?

John “Traenk” Traenkenschuh:
Circular, the path from older Business Resumption Planning (BRP) options to today’s BRP options seems to have gone in a circle.

In times past, well defined applications were housed on well maintained and highly available hosts.  This design simplified identifying both the critical applications and the business data (and important external datastores) being acted on.  Flash forward a decade or more, and the emergent client-server model has us splattering app bits and pieces all over an increasingly ‘splashed to the four winds’ technical infrastructure.

The UNIX server acts as a client to the z/Series, fetching a copy of Accounts Receivable from some obscure PDS and then acting on it.  The new generation of Accounts Receivable data results are now posted to a Windows 2000 server, where someone’s copy of excel, running on a PC, performs data transformations that a staff assistant posts as authoritative graphs of the organization’s Accounts Receivable status.  And this mix of platforms and informal accesses is driving business decision making!

Indeed, everyone applauds the data without realizing the BRP issues:

•    How can we secure that data (and transformations that occur) across so many network and SAN paths?
•    What constitutes ‘safe storage’ in this ad hoc design?  Are any of the data generations ever reckoned back to the z/Series?
•    Which devices are now promoted to our high-priority computer/application list, those systems that MUST be restored by hour four of our BRP planning?  (And are we really comfortable with important data being manipulated and stored on the Staff Assistant’s laptop, possibly misplaced by absent-minded baggage handlers???)

Now flash forward, again, to 2009.  The right application of virtualization technologies can alleviate many of the harms we thought unsolvable just a few years ago.  We begin centralizing the technical infrastructures into a handful of virtualization hosts.  The mandate to virtualize means the company begins alerting and responding to the ad hoc IT flows that flooded our 90’s networks.

Throw in Desktop virtualization, and even those sore-point endpoints, the thousands of laptops and desktops winking on and off the internal network (so-called ‘Intraverse’), these are now backed up reliably.  (No one is saying goodbye to fat clients with the new scheme either.)

This is the core premise of virtualization technologies, that we might begin returning to required centralized technical and governance structures, structures that allow the organization to meet regulatory requirements, to cut costs, and to begin adopting a more green footprint as hundreds of dedicated computers are folded into a handful.

Evolving Solutions:
What disaster recovery and business continuity solutions do you see emerging in the next 5 years?

John “Traenk” Traenkenschuh:
If the term ‘solutions’ equals IT technologies, I think we start poorly?  BRP has always been a practices and procedures discussion; one implemented through technology to be sure, but one that has never been about technology, per se.

I believe, strongly, that the regulatory pressures and economic costs of today’s IT infrastructures require increased virtualization.  This will begin normalizing the infrastructure, the applications, the data (and access methods—maybe, more on that below), etc.  This will impact BRP in several fundamental ways:
•    Technical infrastructure BRP plans must no longer mirror a fractured infrastructure/Intraverse, one that includes all known and planned flavors of linux, a few Macs in the warehouse, Billy-bob’s mobile phone app, and who-knows-what ancient systems lingering in any one building’s computer center.  We lessen the options and force updates.
•    ‘Stealth’ processing and data results will be identified, making system- and application-prioritization more reliable.  Much like the show, “Cash in the Attic”, virtualization forces us to check into all the dataflows and systems , if we are to achieve our goal.
•    Flows that are difficult to manage may go outside the organization.  Increasingly, internal IT shops are no longer required to host every website nor to code up each and every workgroup-level Word macro.  Some of the processing, lurking in baling wire informal technologies that often run on volunteer hardware, these may need to go elsewhere for support.

I see governance to governmental regulations (and business partner practices) increasing the pressure to change.  If IT organizations cannot get a handle on internal pressures to [mis]manage application design and basic information access, away from longtime informal practices; at some point some-to-all IT services may be moved to Cloud Computing organizations, who will reduce an complex IT equation to a true Software as a Service (SaaS) offering.

At this time, IT organizations are in flux regarding whether to build an internal virtualization infrastructure or whether to vFarm IT Out to a third party.  There are compelling reasons for and against building or sourcing your vFarm.

There may be a middle ground:  ‘enhanced resources’, those you and I call ‘Consultants’.  These will be tasked with mapping legacy organizational practice with externally dictated Best Practices, with the idea that there can be a smooth transition plan to a world-class IT infrastructure.  However, that is an expensive course; and as a former consultant, I know that there are some very insular organizations that will not transition until they must.

In my mind, the Enron, and now banking crises, have made regulatory oversight of most organizations inevitable.  Payment Card Industry (PCI) compliance requires security testing with world-class tools.  Although imperfect, this system anticipates all organizations submitting to third-party security audits.

Please remember that I had seven great years in the Insurance industry, at a premier company.  Insurance works, providing premium sufficiency, because industry members manage to common processes that examine the risk exposures faced by all.  In fact, if I can add one small point, I’m shocked that the world of IT security metrics STILL does not have data-driven risk experience models as sophisticated as those used in the insurance industry.

Ask most insurance pro’s the relative claim value of the loss of mechanic’s finger, and they can arrive at a figure, no matter how obscure the facts.  Ask an IT person the relative value of a hairball analysis application, running at a pet food company with 23% market share, and you will wait, despite so much business data near at hand.

The Center for Internet Security (www.cisecurity.org) has an intriguing system of metrics announced recently.  Mitre.org’s CVSS is some help.  But overall, much remains mysterious, although commodity virtualization infrastructures and service offerings may bring us to a more common understanding of the worth of systems and their data, should a disaster occur.

If readers would like a short list of trackable technologies, those aiding BRP, let me offer this one:

•    Security Information and Event Management (SIEM) – Thanks to responsible vendors publishing long lists of security best practices, (Microsoft and others) and to the work of responsible security think-tanks covering security for multi-vendor environments (Center for Internet Security and SANS (@ www.SANS.org)); many organizations have enabled all the logging we can.  In debug mode at that!  This has made incident response as difficult as finding the proverbial needle in a haystack [of event information].  How can you prioritize BRP responses for the important applications, when you cannot separate the security wheat from the chaff?

•    Virtualization Security (known by many names and techniques) – Now that vFarms are hosting our applications, either on-site or off-site, we need to track what goes on at the virtualization ‘back-plane’.  The reoccurring fear is that a hacker can use a virtualized machine’s (VM) security weaknesses to attack the hypervisor, and then use it as gateway to other VM’s.  Another fear is the ‘rogue’ vFarm administrator who does all manner of bad things, accidently or maliciously.  In this world, the vendors to watch are those virtualization vendors with a long history of security prowess and competent tools AND those security vendors who offer solutions for the virtualization layer and for those VM’s needing their host security tracked and alerted (CA and eTrust and others).  The lack of security API’s in many virtualization packages is a leveling factor; few tools can operate at the backplane layer.  But to be sure, configuration audit and management is still possible.  As security API’s are provided by the vendors, being aligned to a solid security vendor will provide valuable.

•    Risk evaluation tools – As mentioned before, there is a fundamental fuzziness to security evaluation that makes risk mitigation difficult, if not dangerously off-the-mark.  Once regulations and cyber-security governmental appointments begin leveling the playing field, we’ll see new, improved risk models and companion tools that make risk evaluation less subject to personal and professional biases.  Maybe.

•    Green IT Movement – Complementary to BRP is the Green IT Movement.  Whether the computers gain electrical efficiency or we find ourselves growing a more extensive IT intraverse on fewer systems, these factors impact BRP directly.  Uninterruptable Power Supplies may be cut back, either because of fewer/more efficient computers OR because we do not want to proliferate an IT environment full of lead-acid batteries belching hydrogen fumes, possibly spilling sulphuric acid during a disaster.   Computer room temperature control units may be scaled back because of fewer computers, improving BRP focus.  I recently read a toilet paper wrapper that proudly proclaimed that the energy used during production was generated through windmills!  Expect all organizations to be encouraged to offer similar claims to environmental sensitivity—and for reasonable adjustments to be made to our BRP plans.

Part 2 of our interview with discussing Business Continuity & Disaster Recovery with John “Traenk” Traenkenschuh, discussing the factors guiding continuity and disaster recovery planning, and tips for getting a plan started, will publish later this week.