CASE

Virtualization's big push to fame was arguably kick started by VMware's Workstation product, which allowed individual users to run a bunch of operating systems (OS), versions, or instances (similar to multiple application windows) instead of having a one-at-a-time multi-boot environment. In many companies, virtualization arrived with developers first using the technology quietly to do testing and development, then introducing the virtualization tools to IT executives. Although computer virtualization today fuels many production environments (e.g., servers, desktop infrastructures, and as a provisioning tool), virtualization is also used by a still-growing number of software developers. For starters, they use virtualization tools to provide a range of target environments for development and testing (such as different operating systems, OS versions, and browsers), and also to provision/reprovision configuration instances quickly and easily.

Mark Friedman, a senior software architect, works in Microsoft's Developer Division, where upwards of 3,000 people create Visual Studio and the .NET Framework. Friedman himself works mainly on the performance tools that ship with Microsoft's Visual Studio Team System. "About two-thirds of the people in my division are in development and testing- and most of these developers and testers are using system virtualization (via Microsoft's Hyper-V technology) as one of their key productivity tools," says Friedman, who is also a board director of The Computer Measurement Group.

One key advantage is virtualization's ability to set apart an unstable environment, which is something any developer expects in early phases of application design. As the Microsoft tools are developed, says Friedman, testing early versions may destabilize a developer's entire computing environment. "That's the nature of the beast," he says. "Almost anything except the simplest desktop application can crash the system. I often tell my developers that if they aren't crashing the system regularly, they are not trying hard enough. We appreciate virtualization technologies, because they save time, and let our developers spend more of their time on the challenging stuff, not the mundane and extremely time consuming aspects of prepping test environments." Like other virtualization tools, Microsoft Hyper-V lets users "snapshot" the system at a "last-known good version."

"We create a rollback that allows us to restore the system to that previous good state within minutes," Friedman says. "The alternative is having to re-image the computer or rebuild the environment, which can take hours," says Friedman. "This is a tremendous timesaver." Friedman's group also makes extensive use of virtualization and virtual machine images in quality assurance (QA) testing. "Once you get past unit testing by the developers, we like to talk about the test matrix-what versions of the OS does it run on, for example," he says. "Then we need to test against both the 32-bit or 64-bit versions of each. Using Virtual Machine (VM) images is the fastest, cheapest, most flexible way to maintain a variety of testing environments. In our QA labs, we are doing this all the time; we do a lot of automated testing, so we stockpile these images as VMs, and point our test suite at them.

It's invaluable. We save so much time you wouldn't believe it." The ability to snapshot, and restore a working image within minutes, rather than wait hours for a rebuilt system, is particularly important. "You're often looking at bugs where you have to spend a day or two to set up the environment just to reproduce a bug. so the time you spend having to rebuild again and again-that's a time waster," says Friedman. "That is loss of productivity." For example, when a bug causes a system crash, the developer has to work backward to identify the point where the system crashed. "People often initially set breakpoints too far into the process and the system crashes again. Since a developer can easily be crashing the system several times a day, being able to restore quickly and then re-run the debugging session while the problem is still fresh in their minds is a great benefit,"

Friedman says "When you're troubleshooting a complicated bug, it's good to stay in that groove." Virtualization is a great boon to iterative testing, developers say. "We have lots of test systems for use by our developers-each of which needs its own x86 control system. Rather than get a lot of PCs, even whatever's cheapest, we're using virtualization to create VMs which then 'attach' to the external cluster boards," says Adam Moskowitz, senior software engineer at SiCortex, which builds and sells a range of energy-efficient computers. A primary benefit of virtualizing the control systems, says Moskowitz, is ease of testing. "Want an experimental system? Clone a standard VM and away you go," he says. The toolset is also valuable because of its ease of configuration; developers can build a VM with the latest software once, then clone it as many times as needed.

Mike Brescia, who works for a company making real time environmental data recording, and retrieval, says, "In addition to offering its system as a virtual appliance, our company's developers use virtualization to put up different test environments, e.g. Windows and Linux, on fewer hardware boxes." Cloning a clean computer system for testing purposes is much easier than copying a disk image and using Clonezilla, Brescia says. "Running VM does not require tearing down and wiping a complete system; we only need to be careful not to overload resources on the host machine," adds Brescia. The 15 Web developers at design and marketing firm Eli Kirk have to cater to a number of Web browsers and operating systems, says Connor Boyack, Eli Kirk's senior web developer, who uses Parallels version 4 on Mac OS X Leopard, and runs VMs of Windows XP, Open SUSE 11, and Windows 7 Beta. "Creating functional and aesthetically identical sites requires a great deal of browser testing, which makes virtualization a must," says Boyack. "Virtualization allows me to use multiple operating systems and multiple browsers all collaboratively and seamlessly."

There's a huge advantage in the ability to test a dozen different scenarios simultaneously to ensure a consistent user experience, agrees Nick Gauthier, a developer at Smart Logic Solutions. "Another advantage is being able to boot up a specific version and browser when a client has a complaint. We no longer have to say 'Well, I don't run that browser, I'm on Linux;' now we can say, 'One moment please. Ah, I see the problem and I'm fixing it now.' " "We use virtualization to test our software on the platforms and applications we support (for QA), and to replicate customer environments when there are customer issues for support," says Eric Floehr, CTO at 3X Systems, a start-up that developed and sells a remote backup appliance. "Virtualization allows us to quickly bring up and tear down environments, and allows us to do so with a minimum amount of physical hardware."

The ability to test against a large number of platforms and environments with a minimum of capital outlay is especially important in a small start-up company. Plus, "It greatly improves the quality and reliability of our product," adds Floehr. There are, developers acknowledge, parts of the process where virtualization can't help. "One area where virtualization doesn't help is in stress testing our product, which is a network-based backup appliance," notes 3X's Floehr. "While we could test 10 simultaneous clients virtually, it isn't a true stress test because the 10 clients are only going through, say, four physical CPUs and two NICs, which are constraints that would not exist if they were 10 physical clients in the real world." "For load testing or scaling, you have to run on the raw hardware, the physical machine," says Microsoft's Mark Friedman. "Performance characteristics, unless you expect to deploy on a VM, are very different. And the same applies to the state of the art for performance measurements; you can't trust the [Physical Machines] you get on a VM currently." Although, Friedman notes, over time he expects this to be fixed.

Also, says Friedman, the para-virtualization approach used by the virtualization industry can and does change some underlying aspects of the OS, notably the OS drivers. "So if you have a dependency on specific drivers or hardware environment, that will be impacted. For example, there's a virtual NIC, and there's a specific driver. The driver is 'virtualization-aware.' You have to understand that if you need testing against a native set of drivers, you won't see that within a VM. So this impacts people doing driver development, but not web developers, etc." Despite those quibbles, virtualization already gives developers real benefits. "Virtualization makes our developers more productive once they learn to work with the technology," says Microsoft's Friedman. "It allows them to spend more of their time in coding, testing and debugging. It eliminates a lot of time and delay spent in set-up and preparation, which isn't where we want them to be spending the bulk of their time."

CASE STUDY QUESTIONS

1. How does virtualization work? In your own words, describe what you understand about the virtualization process as depicted in the case.

2. In software development and testing, where does virtualization help, and where does it not help? Why?

3. What are the business benefits of implementing virtualization technologies in software development? Classify them into those that enhance the effectiveness and the efficiency of the development and testing process.