On July 19th, Evolving Solutions was pleased to welcome Forrester Analyst, James Staten, to our annual customer event.  Staten delivered a presentation on Cloud Computing, including the challenges and the benefits.

According to Staten, the biggest challenge with Cloud Computing is understand what it really is. Forrester defines cloud computing as “a standardized IT capability (services, software, or infrastructure) delivered via Internet technologies in a pay-per-use, self-service way.”

Over the next 7 weeks, we will be posting a series of blog posts introducing you to the various types of Cloud Computing:

1. Infrastructure-as-a-Service (Iaas)
2. Platform-as-a-Service (PaaS)
3. Software-as-a-Service (SaaS)
4. Private Clouds
5. Public Clouds
6. Combined Private & Public Clouds
7. Is the Cloud Right for You?

Check back next Wednesday (August 4th) for a post on Infrastructure-as-a-Service.

Evolving Solutions is pleased to share a recent solution success story about a rapidly growing construction and agricultural equipment provider that needed a high performing, flexible datacenter that would expand with them.

Evolving Solutions overhauled the existing datacenter and conducted an enterprise-wide IT infrastructure redesign using JD Edwards ERP software.  We helped implement and configure the ERP solution, as well as migrating data from legacy systems.

Evolving Solutions designed a storage infrastructure using the IBM XIV storage system, X86 and IBM system p Bladecenter.  This solution sets the company up for uninterrupted growth up to 79 terabytes and can be easily updated if more storage is needed later.

The solution also included virtualization and disaster recovery components. We implemented an IBM System p Blade center appliance, which allows the customer to run five diverse workloads inside a single architecture.  Their virtualized environment allows them to move running workloads between servers to maximize availability and avoid downtime, as well as dynamically adjusting server capability to meet changing workload demands.

With the company’s location prone to flooding, backing up files and moving the tapes off-site is extremely important.  They cannot risk their data being lost.  We therefore implemented a Tivoli Storage Manager disaster recovery solution to act as their insurance policy.

Summary of Benefits

• Consolidation – Streamlining and integration of disparate systems
• Scalability – Room to grow without having to implement a “forklift upgrade”
• Enhanced Performance – Virtualization has increased server utilization
• Network Connectivity – Fast and remote access to company data
• Cost Savings – Savings in systems maintenance costs  following AS400 decommissioning
• Managed Services – Scheduled healthchecks to monitor system performance

Read the full solution success story.

At this point, most companies have dabbled with virtualization in some shape or form and it is becoming more accepted as a viable platform. As virtualization matures, it will become less about hardware and more about how to efficiently manage a virtualized data center environment that has impact company-wide.

Companies need to start viewing virtualization holistically. They don’t fully understand the implications of deploying virtualization and that there are software and security considerations in play. From a business standpoint, when everything is running on one virtualized server, how do you account for the cost internally? There is a need to revisit internal charge codes and develop protocols for cost allocation and accounting. The same goes for troubleshooting. Which department should assume responsibility for addressing any technical issues that arise? Many companies currently separate IT and datacenter line items, but implementing virtualization will require them to be folded into one cost.

The evolution of virtualization, therefore, is as much about managing maturity as it is about technological innovation, which paves the way for managed service providers.

Data replication involves data being replicated and sent across your Wireless Area Network (WAN) to a remote disaster recovery location. Replication is scheduled for a certain time every day and automatically backed up to your remote server.

Data deduplication is the process of backing up data by eliminating redundancies. With data deduplication, only one unique instance of data is retained, meaning that every subsequent instance of that piece of data is referenced back to the one saved copy.

Data deduplication is beneficial for both replication and tape backup. With replication, it reduces replication time and bandwidth, improving recovery point objective (RPO) and recovery time objective (RTO) at the disaster recovery site with increased replication frequency. With tape backup, the increase of data retention on disk may result in lower frequency of tape copies and less tapes being stored off-site.

A more sophisticated version of replication is synchronous replication. This is a technique for replicating data between databases (or file systems) where the system being replicated waits for the data to be recorded on the duplicate system before proceeding. The synchronous replication approach requires access to all slave databases and 100% network availability for the replication to be successful. Therefore, network managers have to plan for synchronous replication and ensure that network availability is sufficient.

With synchronous replication, you have the guarantee that the duplicate system has a copy of the data, but the disadvantages that the primary system must wait for the secondary system before proceeding and replication will not be completed without high network availability.

Synchronous replication is currently the most sophisticated and costly form of data backup.

Coming next Wednesday in our Disaster Recovery series: Virtualization.

Our Discussion With John “Traenk” Traenkenschuh Continues

From natural disasters such as tornadoes, to (recently declared) pandemics like swine flu, our world practically demands companies protect data with business continuity and disaster recovery plans.  And yet, some do not.

In part 1 of our Data Center Leaders Interview with IT veteran John “Traenk” Traenkenschuh, we discuss how business continuity and disaster recovery plans have evolved and how they may look years from now.

In Part 2 below, John leverages real world insight, including that  gained from his experience as a ‘Wave One’ responder in the wake of  Hurricane Andrew, to discuss the  factors that should compel all companies to develop strong continuity and disaster recovery plans:

Evolving Solutions:
Dan Blacharski of IT World ran a poll asking readers to share the disaster recovery plans they have in place.  As of this writing, 31% of respondents answered ‘backup software with off-site storage.’  However, almost as high a percentage of respondents had no plan.  What do you find most surprising about these results?
(Editor’s Note:  Poll numbers have since updated to 40% ‘backup software with off-site storage, 28% with no plan.)

John “Traenk” Traenkenschuh:
I’m surprised that organizations are discussing, publically, those problems that they are screening privately.

How many organizations must admit that not only are their backup media stolen, the same media are unencrypted as well?  Too many organizations have no BRP plan and associate BRP with system back up only, whether a prioritized list of core applications exists or not.

These organizations are most vulnerable to a massive disaster that breaks operations for hours, if not days.  Too many organizations have no identified BRP coordinator.  Most have never had a practice outage.  Most have no NOC (Network Operations Center) documentation to guide a BRP event.  Just as bad, too many have documentation done without the help of the technical staff.  These plans are full of ‘smoke-and-mirror’ assumptions such as ‘IP just plain happens here’.

Whether swine flu, bankruptcy, regulations, or natural disasters like tornadas (what happens to a tornado that swings though my part of the country); there have never been a more compelling set of factors requiring true BRP expertise.
Sadly, never have our most prominent organizations been least prepared to cope with today’s events.

Don’t believe me?  Read the news.  Find out how much critically important data is kept on laptops.  Experience the horror as a backup media theft has a ripple effect that has hundreds of thousands of customers given public notice that an organization is negligent at best.  A good Business Resumption Plan anticipates these exposures and helps the organization take steps to lessen impacts.

Most concerning are those stories that reveal significant parts of the infrastructure are open to attack.  Surprised by the results of the poll?  Not at all.  Hopeful those will change?  Certainly.

Evolving Solutions:
What factors play most heavily in developing a disaster recovery or business continuity plan, for example, government regulations, client contracts, or something more?

John “Traenk” Traenkenschuh:
Yes.

There is no single one factor that motivates even the busiest organizations.  It is a series of factors, acting randomly, that is creating ‘Perfect Storm’ conditions for compliance.  While you list excellent and compelling factors, we miss some of the more compelling.

Swine flu is a perfect example of an exposure that crops up out of nowhere.  Much like a 1970’s ship disaster movie, the premise seems almost ludicrous.  There is a new flu that can kill (and has).  It incubates in pigs, but leaps onto humanity like plague-filled fleas from centuries ago.  World Healthcare organizations predict a pandemic, in a vain attempt to predict the spread (in a small-world global community full of same-day flights).

When too little happens, organizations take this as a sign of needless panic and assume it’s ok to assume the best.  And then it comes, as assuredly as the ship’s belly-flop, everyone is backpedaling to try to fix the blame–instead of working to fix the problem.

I was a ‘Wave One’ responder after the Hurricane Andrew disaster.  Discussing Disaster Recovery conditions with those who have never helped in a sizeable Disaster Recovery event is so very frustrating.  Much like those BRP plans done without detailed knowledge of the underlying technologies, conversing with inexperienced BRP people is a lesson in futility.

It is rife with unfounded assumptions, those as ridiculous as “IP happens here”.  What happens when your organization calls in BRP workers from far away and your area has most road signs blown away?  When the switched landlines are gone, what mobile and cellular options have you lined up?  The water is yellow and brown; what do you do for your onsite workers, knowing local bottled water was sold out two days before?  You’re counting on local expertise to bridge the documentation gap; they have their own tragedies and family illnesses to work through.  What do you do now?

It is the raw unpredictability of today’s business events that compels us to work through the Disaster Recovery specifics, earlier, versus waiting for later.  Those organizations that aggressively prepare for BRP, with noted and experienced BRP experts, will survive.  Those that do not will find their instance of a Heartland data loss (or flu quarantine or…) may find survival impossible.

Evolving Solutions:
What tips would you offer for a business as it develops disaster recovery or business continuity plan for the first time?

John “Traenk” Traenkenschuh:
Surprisingly the most basic, the simplest advice, is seldom heard.  I would like to review a few basic ideas I share freely.  BRP need not be an overly expensive exercise in paperwork and meaningless reports.

a.     Talk with your business insurance professional – I am always surprised to find a business, no matter how small, that has no business insurance plan.

The Insurance Industry can only profit as your business avoids losses and/or lessens impact.  The right representative will take a personal interest in your business and keeping it free from grave exposures unconsciously assumed.  An onsite inspection, useful when calculating premium, can help find those crowded exit hallways that are littered with combustibles.  The inspections may be done for free in some cases; and if so, this can help you begin BRP efforts with low costs.

At some point, your organization will need to decide which exposures (and resultant costs) you will assume yourselves.  The biggest BRP issue too many face is:

•    Vagueness regarding what risks you still face,
•    Uncertainty concerning which of these you have self-insured, and
•    Indecision and inertia over which loss and impact mitigating steps your organization has signed up for.

b.    Create a BRP team within your organization – Let me guess…  You’ve already identified your ‘Goto Guy’ and have pinned all your hopes (and equal wrath) on him or her.  If this is true for your organization, this is ill advised for both political and for BRP reasons.  BRP is seldom popular when times are good and funding is available.  It becomes ‘hysterical over-reaction’ when times are tight and layoffs are ongoing.  Who in your organization is encouraging and funding BRP in these difficult times?

(Ironically, the downturns and layoffs are the most important times to have BRP plans updated!)

Your team needs to have members from a few influential parts of the organization, to ensure minimal funding at all times.  The same small team also ensures that the loss of any one member keeps BRP event handling continuous.  Seeing BRP as an IT-only exercise is another ‘biggest’ mistake an organization might make.  Until BRP is funded and supported by the organization itself, it simply will never be completed.

c.    Engage the services of a ‘Hired Gun’ – The stakes are high, very high.  The required skills involve IT, security, disaster recovery, virtualization, system and application design, IP networking, etc.  You will need an objective third party to bounce ideas off of.  At some point, the same party will need to coordinate disaster simulation exercises.  There is no reason to create forms and process flows for readiness—there is no time for that.  Implement, and not just plan, your BRP initiative!

Be ready for a series of tough questions that challenge your assumptions regarding how your organization operates.  Process improvements, including a painstaking inspection of externally sourced important applications and infrastructure, will be part of the package you hope to buy.  If there is no appetite for change, there can be none.  Meanwhile, how many of your best, most knowledgeable workers just retired or were laid off?  Change is inevitable despite our rejection or acceptance of it.

Hint:  BRP professionals who cannot work remotely or create suitable remote access systems, who insist on expensive and frequent onsite visits, these same people may be incapable of helping your organization’s flexibility during an actual BRP incident.

d.    Test (and Retest) your BRP technologies – Get the right support from the leadership and simply shut down a critically important system and/or application.  How quickly is control and coordination responsibility (and decision making authority) transferred to the correct group?  Were the promised backups performed reliably and were they available?  Hopefully, you will find that all went as planned.  If not, you will have the means to identify and remedy the problems.

I know that I have over-simplified what needs to go into a BRP plan and readiness effort.  Those organizations that are truly interested in improving their BRP readiness will act on steps this simple.  Others will simply laugh it off as hysteria, until it is too late.

Evolving Solutions:
Wild Card: Anything else you’d like to add?

John “Traenk” Traenkenschuh:
We started this interview with a brief glimpse at the business and IT improvements that are possible with virtualization technology, our second chance to exercise good governance and stewardship over the applications and information placed under our care.  The latter part of the interview discusses Business Resumption Planning and the many ways this field of study has grown, despite lack of interest from those organizations risking public and regulatory ridicule for simple mistakes.

Virtualization can provide focus for BRP efforts, all the while ‘Perfect Storm’ factors call on today’s BRP professionals to be ready for new and more challenging disaster scenarios.  In all of this, the issue is not a technical issue.  Instead, it is an issue with organizational efficiency and ability to face the challenges.  All the advice in this interview is provided freely, with no claims given to its sufficiency or perfection.  These views do not reflect the views of my employers, the organization publishing this interview, or any of my professional organizations with which I am associated.

I promise to do my best to respond to any and all queries left in response to this interview.  BRP, Virtualization, and Green IT are more than slogans to me.  They are a key to surviving in a radically changed business and social environment that seems more the stuff of nightmares than the evening news.

Business Continuity & Disaster Recovery Expert John “Traenk” Traenkenschuh

In a recent interview with Lawrence Webber, we discussed the ‘hows’ and ‘whys’ of business continuity and disaster recovery planning.  This week, our Data Center Leaders Interview Series drives home the importance of this topic during our two part interview with John “Traenk” Traenkenschuh.

Budding author, book editor, and Information Technology worker at three Fortune 100 companies, John “Traenk” Traenkenschuh’s insight into business continuity and disaster recovery planning comes courtesy of years of real world experience.

For his time spent introducing students to Microsoft’s Visual Basic, Traenk has been awarded the Microsoft Most Valuable Professional (MVP) designation since 2004.   He has also authored  VCP VMware 310 Cert Flash Cards as a late stage exam preparation tool.

In part one of our interview below, Traenk demonstrates the value of business continuity and disaster recovery planning through a look at both the evolution and future of  these solutions:

Evolving Solutions:
You have extensive experience in the fields of disaster recovery and business continuity.  How would you describe the evolution of recovery and continuity solutions since you first entered IT?

John “Traenk” Traenkenschuh:
Circular, the path from older Business Resumption Planning (BRP) options to today’s BRP options seems to have gone in a circle.

In times past, well defined applications were housed on well maintained and highly available hosts.  This design simplified identifying both the critical applications and the business data (and important external datastores) being acted on.  Flash forward a decade or more, and the emergent client-server model has us splattering app bits and pieces all over an increasingly ‘splashed to the four winds’ technical infrastructure.

The UNIX server acts as a client to the z/Series, fetching a copy of Accounts Receivable from some obscure PDS and then acting on it.  The new generation of Accounts Receivable data results are now posted to a Windows 2000 server, where someone’s copy of excel, running on a PC, performs data transformations that a staff assistant posts as authoritative graphs of the organization’s Accounts Receivable status.  And this mix of platforms and informal accesses is driving business decision making!

Indeed, everyone applauds the data without realizing the BRP issues:

•    How can we secure that data (and transformations that occur) across so many network and SAN paths?
•    What constitutes ‘safe storage’ in this ad hoc design?  Are any of the data generations ever reckoned back to the z/Series?
•    Which devices are now promoted to our high-priority computer/application list, those systems that MUST be restored by hour four of our BRP planning?  (And are we really comfortable with important data being manipulated and stored on the Staff Assistant’s laptop, possibly misplaced by absent-minded baggage handlers???)

Now flash forward, again, to 2009.  The right application of virtualization technologies can alleviate many of the harms we thought unsolvable just a few years ago.  We begin centralizing the technical infrastructures into a handful of virtualization hosts.  The mandate to virtualize means the company begins alerting and responding to the ad hoc IT flows that flooded our 90’s networks.

Throw in Desktop virtualization, and even those sore-point endpoints, the thousands of laptops and desktops winking on and off the internal network (so-called ‘Intraverse’), these are now backed up reliably.  (No one is saying goodbye to fat clients with the new scheme either.)

This is the core premise of virtualization technologies, that we might begin returning to required centralized technical and governance structures, structures that allow the organization to meet regulatory requirements, to cut costs, and to begin adopting a more green footprint as hundreds of dedicated computers are folded into a handful.

Evolving Solutions:
What disaster recovery and business continuity solutions do you see emerging in the next 5 years?

John “Traenk” Traenkenschuh:
If the term ‘solutions’ equals IT technologies, I think we start poorly?  BRP has always been a practices and procedures discussion; one implemented through technology to be sure, but one that has never been about technology, per se.

I believe, strongly, that the regulatory pressures and economic costs of today’s IT infrastructures require increased virtualization.  This will begin normalizing the infrastructure, the applications, the data (and access methods—maybe, more on that below), etc.  This will impact BRP in several fundamental ways:
•    Technical infrastructure BRP plans must no longer mirror a fractured infrastructure/Intraverse, one that includes all known and planned flavors of linux, a few Macs in the warehouse, Billy-bob’s mobile phone app, and who-knows-what ancient systems lingering in any one building’s computer center.  We lessen the options and force updates.
•    ‘Stealth’ processing and data results will be identified, making system- and application-prioritization more reliable.  Much like the show, “Cash in the Attic”, virtualization forces us to check into all the dataflows and systems , if we are to achieve our goal.
•    Flows that are difficult to manage may go outside the organization.  Increasingly, internal IT shops are no longer required to host every website nor to code up each and every workgroup-level Word macro.  Some of the processing, lurking in baling wire informal technologies that often run on volunteer hardware, these may need to go elsewhere for support.

I see governance to governmental regulations (and business partner practices) increasing the pressure to change.  If IT organizations cannot get a handle on internal pressures to [mis]manage application design and basic information access, away from longtime informal practices; at some point some-to-all IT services may be moved to Cloud Computing organizations, who will reduce an complex IT equation to a true Software as a Service (SaaS) offering.

At this time, IT organizations are in flux regarding whether to build an internal virtualization infrastructure or whether to vFarm IT Out to a third party.  There are compelling reasons for and against building or sourcing your vFarm.

There may be a middle ground:  ‘enhanced resources’, those you and I call ‘Consultants’.  These will be tasked with mapping legacy organizational practice with externally dictated Best Practices, with the idea that there can be a smooth transition plan to a world-class IT infrastructure.  However, that is an expensive course; and as a former consultant, I know that there are some very insular organizations that will not transition until they must.

In my mind, the Enron, and now banking crises, have made regulatory oversight of most organizations inevitable.  Payment Card Industry (PCI) compliance requires security testing with world-class tools.  Although imperfect, this system anticipates all organizations submitting to third-party security audits.

Please remember that I had seven great years in the Insurance industry, at a premier company.  Insurance works, providing premium sufficiency, because industry members manage to common processes that examine the risk exposures faced by all.  In fact, if I can add one small point, I’m shocked that the world of IT security metrics STILL does not have data-driven risk experience models as sophisticated as those used in the insurance industry.

Ask most insurance pro’s the relative claim value of the loss of mechanic’s finger, and they can arrive at a figure, no matter how obscure the facts.  Ask an IT person the relative value of a hairball analysis application, running at a pet food company with 23% market share, and you will wait, despite so much business data near at hand.

The Center for Internet Security (www.cisecurity.org) has an intriguing system of metrics announced recently.  Mitre.org’s CVSS is some help.  But overall, much remains mysterious, although commodity virtualization infrastructures and service offerings may bring us to a more common understanding of the worth of systems and their data, should a disaster occur.

If readers would like a short list of trackable technologies, those aiding BRP, let me offer this one:

•    Security Information and Event Management (SIEM) – Thanks to responsible vendors publishing long lists of security best practices, (Microsoft and others) and to the work of responsible security think-tanks covering security for multi-vendor environments (Center for Internet Security and SANS (@ www.SANS.org)); many organizations have enabled all the logging we can.  In debug mode at that!  This has made incident response as difficult as finding the proverbial needle in a haystack [of event information].  How can you prioritize BRP responses for the important applications, when you cannot separate the security wheat from the chaff?

•    Virtualization Security (known by many names and techniques) – Now that vFarms are hosting our applications, either on-site or off-site, we need to track what goes on at the virtualization ‘back-plane’.  The reoccurring fear is that a hacker can use a virtualized machine’s (VM) security weaknesses to attack the hypervisor, and then use it as gateway to other VM’s.  Another fear is the ‘rogue’ vFarm administrator who does all manner of bad things, accidently or maliciously.  In this world, the vendors to watch are those virtualization vendors with a long history of security prowess and competent tools AND those security vendors who offer solutions for the virtualization layer and for those VM’s needing their host security tracked and alerted (CA and eTrust and others).  The lack of security API’s in many virtualization packages is a leveling factor; few tools can operate at the backplane layer.  But to be sure, configuration audit and management is still possible.  As security API’s are provided by the vendors, being aligned to a solid security vendor will provide valuable.

•    Risk evaluation tools – As mentioned before, there is a fundamental fuzziness to security evaluation that makes risk mitigation difficult, if not dangerously off-the-mark.  Once regulations and cyber-security governmental appointments begin leveling the playing field, we’ll see new, improved risk models and companion tools that make risk evaluation less subject to personal and professional biases.  Maybe.

•    Green IT Movement – Complementary to BRP is the Green IT Movement.  Whether the computers gain electrical efficiency or we find ourselves growing a more extensive IT intraverse on fewer systems, these factors impact BRP directly.  Uninterruptable Power Supplies may be cut back, either because of fewer/more efficient computers OR because we do not want to proliferate an IT environment full of lead-acid batteries belching hydrogen fumes, possibly spilling sulphuric acid during a disaster.   Computer room temperature control units may be scaled back because of fewer computers, improving BRP focus.  I recently read a toilet paper wrapper that proudly proclaimed that the energy used during production was generated through windmills!  Expect all organizations to be encouraged to offer similar claims to environmental sensitivity—and for reasonable adjustments to be made to our BRP plans.

Part 2 of our interview with discussing Business Continuity & Disaster Recovery with John “Traenk” Traenkenschuh, discussing the factors guiding continuity and disaster recovery planning, and tips for getting a plan started, will publish later this week.

Neil McAllister at InfoWorld wrote a great piece recently titled “Server Virtualization Under The Hood.”

Beyond offering insight purely on the benefits of server virtualization, McAllister takes the time to define and offer historical perspective on this burgeoning solution.

McCallister begins with an excellent definition that simplifies the idea of server virtualization, stating:

“Virtualization is a solution that basically fools an operating system (and any applications that run on top of it) into thinking the virtual machine is actual hardware. Running multiple virtual machines can fully exploit a physical server’s compute potential – and provide a rapid response to shifting datacenter demands.”

The benefit of virtualization is essentially this:  as a physical server will cost business capital to run, and only have so much computing power, it makes sense from both a financial and technical standpoint to minimize its inherent limitations and enable  virtual servers to shoulder the work.

As McAllister alerts us, the concept behind virtualization is not a new one.

Individual computers have been running multiple instances of operating systems simultaneously as far back as the 1970s.

What is new, however, is the feverish pitch surrounding virtualization as it has made its way into the general business lexicon as a cost-saving and efficiency enhancing solution.

For those seeking more detail on the different types of server virtualization available, McAllister defines the advantages of those most typically in play today:

• Full virtualization – allows nearly any operating system to be virtualized without modification
• Para virtualization – similar to full virtualization, but offering greatly improved response time for virtual servers
• OS-Level virtualization – an architecture that uses a single, standard  operating system across all virtual servers providing even greater speed

If you are considering implementing a server virtualization solution, more important than the different types of solution are the benefits your company will receive from the solution.

To get to the heart of the ultimate benefits of server virtualization, assess your IT solution needs by asking yourself questions like:

• Are you seeking primarily to reduce data center costs?
• What type of response time is necessary to ensure IT efficiency?
• What are your data capacity requirements?
• What is your timeline for implementation of a solution?

In the end, any IT solution, including server virtualization, should be looked at and discussed through the lens of its end-benefit to your business.

This is perhaps the most important idea to keep in mind when looking under the hood of any new solution.