I’ve been taking a close look at some virtualization technologies both on the client and server side. Here are a few pointers to understanding what’s happening out there – presented as 10 myths.

1. Virtualization is a ground breaking technology

Well, not really. First of all virtualization, as a technique, has been around since the 1960s. There are three major uses of virtualization:

1. OS virtualization, as with VMware, which separates the OS from the hardware. IBM was doing this in the 1970s but VMware had the neat idea of making this available on the commodity Intel platform. As a consequence OS virtualization is now de rigeur, so everyone and his dog offers OS virtualization. Note that desktop virtualization is just OS virtualization but the associated technical arrangements can be complex.
2. Resource virtualization means making one resource emulate and substitute for another. So virtual memory is disk pretending to be memory. RAID disks make a collection of disks appear to be one big disk resource. Network virtualization turns physical wires (or fiber) into multiple channels with variable bandwidth.
3. Application virtualization means abstracting an individual app from its native environment so it can run anywhere. There are many ways to do this and many complexities, but most application are now written in a language that’s portable (C, C++, Java and just about every kind of scripting language). That makes life a little easier, because very little application code is going to be hardware or platform specific.

What happened that made a big difference is that technology just keeps getting faster and it makes virtualization a sensible thing to do in many environments. By about 2004 most commodity Intel servers were running at maybe 6% efficiency or even worse. Virtualization could raise that level of efficiency. When power consumption became a data center problem there was suddenly a huge incentive to use virtualization for server consolidation.

2. OS Virtualization optimizes the server infrastructure.

OS virtualization doesn’t optimize anything. It makes use of unused resources, and it is a powerful capability for doing that, but it doesn’t necessarily make the best use of such resources. The point here is that virtualization itself needs to be co-ordinated within an optimization framework – which will need to be at least partly automated and will make use of performance measurements. Such a framework will naturally use virtual machines where sensible and it will carry out the optimization because it has the information to do that.

3. OS Virtualization optimizes the use of an individual server (or server blade).

Again virtualization doesn’t optimize anything. The point is that running anything extra on an underutilized  server makes better use of the server. The simple truth is that many servers have been deployed running only one application. Under Windows particularly, but also under Linux, it is possible for one application to interfere with another, if you run them under a single OS. By running applications under separate OSes, even if you are running on the same machine, you achieve a kind of “shared nothing” software environment. So, if an application fails, then it will not be because another clashed with it.

But if VMs were a really good idea, we’d have been running them since Bill Gates was a boy. There is a very real overhead to running multiple virtual machines on a single server – it means you have to run multiple operating systems and each of these presents an overhead. You can end up with the situation where most of the work being done by the server is in running the hypervisor and running the operating systems. If you weren’t doing anything with the resource anyway, then it doesn’t matter if this arrangement isn’t maximally efficient, because you’re running more applications on the server than you were before.

Let’s be realistic. Server virtualization can certainly take you to somewhere in the region of 60% utilization without causing resource clashes and even if half of this usage is the hypervisor and the OS, you are still getting 30% real work from the server where previously it may have been below 6%.

4. If you are going to partition a server you have to run virtual machines in each partition.

Virtualization is about partitioning resources between competing needs in order to make better use of resources. You don’t have to do this by setting up virtual machines. There’s a company called Trigence which takes a different approach. Instead of deploying virtual machines it deploys “virtual applications.” Here’s how it works:

It has a hypervisor that runs all the applications, but it packages the applications up so that all the components of the OS that the applications need are kept with the application. In this way it can run Windows apps under Linux, simply by holding and executing all the Windows components (DLLs etc.) in a single partition. The advantage is that it is likely to be more efficient than running whole OSes, but it is still a shared-nothing environment.

5. Server virtualization should involve the dynamic provisioning of VMs.

Dynamic provisioning is a great idea. You monitor the network as a whole and all the applications that are running and you predict traffic (based on current trends and previous patterns) then you dynamically allocate all the resources needed to meet the service levels of all the business processes involved – neatly adding new resources if any device anywhere fails.

This is the same idea that the IT industry had decades ago when it was trying to squeeze as much as possible from very expensive mainframe computers. As computers fell in price most IT users ceased to care about optimization. If you wanted more power you just bought another cheap computer and clipped it in. Optimizing a single mainframe was not a cake-walk and optimizing a network of hundreds or thousands of servers is not a cake walk either. Doing it dynamically (in a way that’s effective) is beyond the capabilities of current technology.

So dynamic provisioning of VMs doesn’t make sense right now, because we are only just learning how to provision VMs effectively in a manual way. Also there’s a gotcha here – anyone who’s in charge of a large data center would probably not allow automated dynamic provisioning, just in case it just happened to screw everything up. The closest we can get right now is to have software that can suggest specific provisioning actions, which someone can choose to do if the suggestion seems sensible. There are many issues that make dynamic provisioning complex, from fail-over requirements on the technical side to licensing complications on the business side. But most of all, right now it’s simply not an imperative, because there’s no guarantee that it will deliver more than can be achieved using manual intervention.

6. Virtualization reduces the problem of managing servers.

This is not really true. All that server OS virtualization does is reduce the number of physical servers. Doing this may create some unexpected hardware effects – like for example changing traffic loading on the network and hence requiring network reconfiguration or maybe even upgrade. In physical terms the “net net” is likely to be very positive – especially in terms of power consumption. However, in software terms you are still managing the number of servers that you were before and the situation is likely to be more complex because you no longer know exactly what is running where. So that means you now have to track the location of all software in real-time, so you can fix an application properly if it breaks.

And that, of course, is why VMware has a great business in providing the management software for managing virtual machines.

7. Virtual environments are more secure because no intruder will know where software is running.

If a talented intruder gets into your network he/she can find out where things are running very easily by listening to the packet traffic on the network. That’s what dynamic discovery software does and hackers can use such software too. Virtual environments are less likely to be secure rather than more likely because, right now, there is no simple way to map the dependencies between security software and the applications it protects. If the virtualized environment gets complicated then unintended security holes could open up.

The fundamental problem here is that the security software is not build into applications. Instead it tries to surround them to protect them. But if you move applications about regularly, it’s entirely possible that you just might move one of them “outside the castle walls”. Until security dependencies are explicit and mapped, inadvertent vulnerabilities are bound to occur. This, by the way is another problem with dynamic virtualization – it needs to keep track of, and enforce, security.

8. You can virtualize all the PCs and desktop computers in the organization.

It may be possible, but probably not. In general it’s possible to virtualize “standard desktops” – say about 80% of the typical desktop population. It’s the high-end machines that are difficult to virtualize. Even some of these can be virtualized (HP has particularly good technology for this), but it’s unlikely that you can do them all.

And just in case you think this is just like server consolidation, it isn’t. Desktop virtualization is complex for quite a few reasons, but the important point to understand is that you are not virtualizing the PC specifically. You are virtualizing the access capability, so that the user’s capabilities live in the network and can be used from different access points (maybe a thin client, or laptop, or desktop PC).

9. A virtualized PC is cheaper than a real PC.

You’ll find that it isn’t (at the moment) because there’s a very definite start-up cost. You need a thin client and a portion of a server and broking software and the Microsoft license doesn’t vanish and the virtualization software doesn’t cost nothing either. However, the cost of ownership of the virtualized PC is going to be much lower in almost all cases.

10. Eventually every environment, access point or server, will be virtualized and we’ll run everything from a browser.

It’s a beguiling idea and if we were to freeze the user interface, it definitely would be true, because eventually we’ll split applications in a AJAX-like way, with the interface running wholly on the client and everything else on the server. This actually could transpire in the corporate environment in some areas of usage given time.

However the trends tell me that computing has been driven by the user interface ever since the first user was invented. Interfaces cause obsolescence and thus are the heart and soul of consumer computing. They’ll be selling us better interfaces in 50 years time, I’m sure. And as employees we’ll encourage our organizations to adopt such interfaces too – for the sake of “productivity.”

The IT vendors need to sell us hardware in order to sell us the interface, so they’ll develop teh Interface on the front-end device for as long as they can.

This is a posting in the Virtualization Focus Series. Click here to see an index of such postings.

I’m republishing here an interview I originally did for BPM Delivers quite a while ago – but updating it with further comments which are in italics. Strangely, not much has changed.

Question 1: In Delphi Group recently surveyed 110 firms about their 2006 investment priorities. The top three responses were related to Service Oriented Architectures (SOA) and Business Process Management (BPM). Is this a hype or a real trend?

Answer: The trend is real. A number of major organizations that I’ve talked to – mostly in the financial sector – have multi-year projects running in these areas. As time passes the number of such companies just grows. In my view BPM ceased being hype in 2004 and SOA ceased to be hype in 2005. Those were the years when I noticed genuine benefits being delivered to IT users by these respective technologies (neither of which, by the way, are mature).

Comment: Now, of course, there’s no question. All the questions are about the maturity of the technology mostly from the sudden discovery that there are lots of technology gaps that need to be plugged. Now we’ve moved into a software world that is driven by components, engines and platforms. It’s the world of SOA whether it’s good SOA, bad SOA, or so what SOA.

Question 2: What is your personal definition of the two approaches?

Answer: Technically, BPM is the marriage of work flow with web services. It is a business process designing and building capability that empowers business analysts. The important point is that you can design and implement end-to-end processes utilizing existing business applications and adding new business logic where necessary.

As for SOA, I like to think of BPM as the tip of the iceberg with SOA being the iceberg. SOA embraces BPM because it is the architecture that best enables BPM. But SOA also orchestrates the IT infrastructure and delivers sustainable IT service levels. If you want a definition, try this: SOA is an architecture for building business applications as a set of loosely-coupled black-box components orchestrated to deliver a well-defined level of service.

Comment: There’s nothing to add. The SOA definition here is the one we used in SOA for Dummies and it has legs. It’s now used everywhere.

Question 3: How do they relate to each other? Is there success for one without the other?

Answer: I answered the first part of this in the last question. So, could SOA be successful without BPM? In theory it could, but SOA does so much to enable BPM that it is difficult even to imagine any organization implementing SOA without BPM. Why would anyone do that? It would be like building a boat and not sailing it anywhere.

Can BPM be successful without SOA? Well, to be honest, it could. Some of the workflow and document management systems of the past could be called BPM-without-SOA. The problem is that without SOA you end up over-investing in technology and ultimately you run into scalability problems – and actually many other problems too. No-one in their right mind would implement BPM without planning to implement SOA at some point. Mix a lot of BPM with too little SOA and you will surely have a mess on your hands.

Comment: We’ve moved on a little here I think. Now everyone sees BPM and SOA as brother and sister.

Question 4: What do you consider the greatest myth about BPM and SOA?

Answer: There are a number of myths surrounding BPM and SOA. Choose your favourite. All are equal, but some are more equal than others.

• BPM and SOA are just marketing terms. Not so, but they are terms that do seem to confuse marketing people.
• BPM mandates rules-based software development. Only if you’re a salesman of rules-based BPM tools.
• We can do BPM with the software tools we’ve got. And you can fell a tree with a hammer rather than an axe, but it takes a lot longer and it’s not clever.
• SOA is just Object Orientation with a new hat on. Not so. It’s got a new wardrobe entirely and it’s had surgery.
• SOA is like sex with aliens, you hear reports but there’s no evidence that it happens. Actually, close encounters of the third kind do take place.

Comment: Nobody would now dispute that SOA really happens and happens in your back yard. The jury is still out on sex with aliens.

Question 5: What are the most difficult steps within a BPM project – and what makes a SOA project tedious?

Answer: The most difficult steps within a BPM project are the early ones. The problem is cultural. As a fact of business history and IT history, all organizations are siloed. Hell, I know it’s a cliché and a platitude, but its also true. The siloed nature of organizations is ingrained. You have to get people to think end-to-end rather than silo. This means everyone, the business folk and the IT folk and any other folk who happen to be around. The IT folk are siloed too, you know. You need to “get their minds right” because with BPM you need cross-discipline teams who don’t indulge in turf wars.

As for SOA projects, I don’t believe one should even think in terms of implementing SOA as a project. SOA is a road and it’s a road that everyone will ultimately have to take, because it’s the road that the IT industry has already taken.

Is there anything tedious on this road? Yes there is; turf wars and inadequate technology.

Comment: It’s still true. It’s still the case that the cultural problems are the biggest block to SOA.

Question 6: What best practices do you recommend to organisations looking to initiate a BPM / SOA project?
I could write a book about this, in fact we did write a book; SOA for Dummies. So let’s just pick two things that I believe to be critically important:

Answer: Get sponsorship right from the top. There are many reasons why this is necessary, because SOA and BPM usually cause significant changes to an organization.
Also pick an easy first target. Make sure to go for low hanging fruit on the first project. You know what I mean, low risk, high benefit. You really don’t want the first project to stall in any way.

Comment: Now I would add, that you should look to implement comprehensive Identity Management as soon as possible and also go after coherent Asset management. The big note on the wall shoudl read: “It’s the plumbing, stupid.”

This was written “with tongue partly in cheek,” but despite that, where it counts it is factual and accurate.

Question 1: Are paper bags better than plastic bags, ecologically?

Answer: Absolutely not. They do more damage in almost every way. They destroy trees which eat carbon dioxide. You get about 1400 paper bags from a single tree. Plastic removes no trees. It takes more than four times as much energy to manufacture a paper bag than a plastic bag, which means a much bigger carbon footprint per bag in manufacture. The chemicals used in paper production contribute to acid rain and water pollution. Paper bags generate 70% more air pollutants and 5000% more water pollutants than plastic bags. Additionally, it takes 91% less energy to recycle equivalent weights of plastic than paper. Paper isn’t even much more degradable in landfills.

This, of course, is why China, Rwanda, Eritrea, South Africa and the city of San Francisco have banned the use of plastic bags in retail.

Question 2: What can we do to save the polar bears?

Answer: Collect plastic bags and recycle them so they can be made into large reinforced polystyrene blocks coated in melamine. We can then set these blocks floating in the Arctic Ocean. When the ice disappears, the polar bears will have something to scramble on to when they get tired of swimming. They’ll be able to sit there and sun themselves as they snack on baby seal pups and wave to the passing whales.

This green idea would work extremely well if cities, and even whole countries, didn’t keep banning the use of plastic bags.

Question 3: Is methane (CH4) worse than carbon dioxide (CO2)?

Yes it’s fifty times as bad. It’s so bad that nowadays eco-responsible people regard “methane” as a cuss word. They breathe in deeply whenever they use the M word. Currently, the Earth’s crust contains huge amounts of methane. Large amounts of methane are produced anaerobically by methanogenesis. Other sources of methane include mud volcanoes, which are connected with deep geological faults and cows, which are connected with hamburgers.

Question 4: So should I eat hamburgers?

What? Are you kidding? Hamburgers are a complete disaster – except when served in a sesame seed bun with white onions, a tomato slice, jalapenos and cheddar cheese that knows how to melt properly. Nevertheless, even if you can get hamburgers made this way (hint: try Poodies Hilltop Bar and Grill, a honky tonk on rt 71 to the west of Austin) you might like to reflect on the fact that:

While they are preparing to be hamburgers, cows chew grass all day or eat animal feed grown on fields that could have been growing wheat, soya bean or anything that can be made into ethanol. While chewing grass all day they emit methane, which is not only rude, but also a greenhouse gas. They have to be taken to the slaughterhouses via train or truck, because the days of Rawhide are long gone and nowadays cowboys spend all their time at rodeos riding bulls. Once dead and packed, more greenhouse gases are pumped into the atmosphere as the meat is distributed across the US to steak houses and burger joints

According to someone who’s spent an awful lot of time with a calculator, the carbon footprint of all the hamburgers eaten in the US is about equivalent to the carbon foot-print of all the gas-guzzling SUVs that are driven in the US (click here for the details). So stop eating hamburgers (except those referred to above from Poodie’s).