Windows XP both native and in VirtualBox

There are several good tutorials about similar topics, namely:

1. How to migrate existing Windows installations to VirtualBox
2. HOWTO: Windows XP in both VM and native

but I couldn't find any that would describe what I was trying to achieve.

What I had: Windows XP installed on a physical partition. What I wanted: Being able to run the installation both inside VirtualBox and also natively. I did not want to reinstall the WinXP installation nor did I want to have to fiddle with any Windows Genuine Disadvantage stuff. And I didn't want to break the native boot in the process. Also, as this was some über-corporate install I didn't want to have to ever use the WinXP install CD + its rescue console.



The second link above ([2]) was almost what I needed, but it expected that you're starting with a new windows installation (and it went the other way round: from VirtualBox to native). Nevertheless, most of what the guide says is applicable here too.


Btw. I'm not much of a windows person and many things (like the windows boot) are quite opaque to me. Probably some steps are not needed/might be adapted, but the instructions described here worked for me.

I strongly suggest you first read the above mentioned HOWTOs. Also have a look at the VirtualBox User Manual. Back up your installation. It might be useful in case you're not lucky. Also watch out -- using raw partitions with VirtualBox (or any other virtualization software for that matter) can be dangerous: A typo in the process and your data is history. Also, if you're reading this, read this whole first, before trying anything out on your box.

The outline is as follows:

1. Prepare windows for the shock

Boot into windows (yuck) and prepare the system for the shock. Take inspiration from [1]:

1. Run MergeIDE (There might be a way to avoid that -- see below, but in general just proceed this way.)
2. Create a new hardware profile to be used for the VirtualBox boot (My Computer, Properties, Hardware). The idea is to have two profiles -- one for native boot and another for VirtualBox boot. See [2], section IV, 2. Turn off timeout so that you don't end up with the wrong profile by accident and give the profiles a reasonable names. I'm not sure how badly would windows suffer if you used the other profile but I was too afraid to try. :-)

In [1] there are some rumours that you should watch out for some .sys files (namely agp440.sys, intelppm.sys) and remove/disable these. On the box I was doing this procedure there was intelppm.sys. I moved it away from windows directory. After the process was finished I moved it back and it has not caused any trouble. I don't know if it would have affected the process had I not moved the file away.

2. Allow the user who will be running the VirtualBox instance R/W access to the windows partition

There are several ways how to do this. A way I chose was:

1. Create a special group for this:
   
   addgroup vbox-disk
2. Add myself to the group:
   
   adduser $USER vbox-disk
3. Create an udev rule to change the group of the right device file to the new group:
• create /etc/udev/rules.d/vbox-disk.rules with the following contents (assuming sda1 is the windows partition):
  
  KERNEL=="sda1", GROUP="vbox-disk"

Hm, I yet have to figure out how to make udev actually change the ownership without a reboot. Anyway, a reboot works. :-)

3. Create a VMDK file that refers to the real windows partition.

You could theoretically use a whole-disk VMDK file, but unless the disk is really dedicated to windows I'd advise against that. Restrict the VM to only what it needs to use. You don't want windows to overwrite your linux partition in a fit of rage. :-)

The process is described in the VirtualBox User Manual. Assuming that windows live on the first partition of the first hard disk (typical scenario) you would do something like this:

VBoxManage internalcommands createrawvmdk -filename /path/to/file.vmdk -rawdisk /dev/sda -partitions 1 -relative

If you don't have read access to the whole disk device you need to run this as root. Then change the ownership so that the file is owned by the user who will be running the VirtualBox instance.

Make sure you're not accessing the windows partition from linux while the VM will be running. While some guides say mounting the partition read-only is ok, I would advice against doing that. I could imagine this resulting in your host oopsing. Use VirtualBox's shared folders to exchange data between host and guest.

4. Create a new virtual machine

Mostly follow what [2] has to say. Just watch out: As this is not a new install, you need to have the right setting for the IO APIC. As explained in [1] windows are quite sensitive to this change. Most probably you will need to enable it.

• Set IO APIC to match your PC.
• Attach the raw VMDK file as the virtual HDD, attach it to IDE controller.
• Enable exactly one Ethernet adapter.

It might be possible to use SATA adapter instead of the IDE one but then I gather you'd have to have a very similar adapter to the one that VirtualBox is emulating (it seems to work ok with the Intel Matrix Storage SATA drivers -- so if you're running with these in the real machine it might just work). On the machine I was doing this procedure there were SATA drivers installed but they did not work with the VirtualBox virtual SATA adapter.

Unfortunately the Intel Matrix Storage STA driver refuses to install in native environment if you don't have a supported adapter so it seems impossible to prepare the windows installation for the virtual SATA adapter in advance. But with MergeIDE windows can be prepared at least for the VirtualBox IDE adapter. It's possible to move the VMDK to virtual SATA adapter later.

5. Fine-tune virtual machine settings

In order to not bump into the ridiculousness of the Windows Genuine Disadvantage thing you (I gather) need to copy the DMI data from your real PC into the virtual machine. Follow [2] (section I.2 - DMI BIOS settings).

Furthermore [2], section VII - Windows activation suggests:

• Make a backup of %WINDIR%\system32\wpa.dbl - hopefully if you bump into the WGA ridiculousness, you can revert and try again.
• Set the MAC address of the one emulated Ethernet adapter to that of your real Ethernet adapter. If you've got more of them, you might be in for some fun. See the above-referenced section VII in [2].

6. Restore windows MBR

If you have Grub in the MBR, booting the VM as it is now, provided you allowed access only to the windows partition, would result in the rescue prompt of Grub as it can't access the linux partition with the data it requires.

Luckily the VMDK file allows for its own MBR. The MBR is not written to the real disk (if it's not a whole-disk file, but a file referencing only partitions) and the data is not discarded either -- it's kept in the VMDK file. So you can have different MBR in the virtual machine and in the real box.

To boot windows all you need is to overwrite the VMDK's MBR with something that will correctly start NTLDR on the windows partition. One way to do this is to use ms-sys. Luckily SystemRescueCd contains ms-sys, so all you need to do is download SystemRescueCd image, attach it to the virtual machine and boot from the ISO image. Once you get the SystemRescueCd prompt, check that the hard drive looks as expected (VirtualBox should not allow you to access any partitions that were not listed when creating the VMDK file) and then do something like:

ms-sys -m -w /dev/sda

This effectively nukes Grub in the VMDK image. Shut down the virtual machine and detach the ISO image. After a fresh start you should see windows trying to boot.

Note: If you have the right MBR on hand, you could have used the -mbr parameter when creating the VMDK file. I said: read this whole first! :-)

7. Making windows boot to the GUI

You should now see at least the HW profile-selection menu of windows. Select the profile for VirtualBox boot and hope... Depending on how lucky you are or are not, windows will either boot into GUI or will hang/BSOD.

In my case I saw the following behaviour:

• If IO APIC was disabled: The boot would hang up after loading mup.sys -- this is visible in safe mode boot.
• If IO APIC was enabled: After loading mup.sys the system would seemingly do nothing for a while and then I'd get a BSOD.

Btw. to ease troubleshooting it might be (moderately) helpful to add the option /SOS into the boot.init file. Just watch out -- don't mount the partition while the VM is running and unmount it before you start the VM again.

The BSOD made me unsure about the IO APIC setting for my box and I was considering the option should be unchecked on my setup. But it turns out that, as expected, IO APIC on was the right setting, despite the BSOD.

I'm not exactly sure what caused the BSOD, but the following seems to have helped:

• Select different IDE controller type. Setting it to ICH6 worked for me. The default, that was PIIX4, constantly produced bluescreens on startup for me.
• Increase VM RAM size. I guess I was too harsh on windows. I increased it from 512M (on the 2G RAM box) to 1G.

After I got windows to boot I tried decreasing the memory size back to 512M to see if it would BSOD but it was not reproducible, so I'm not at all sure about the influence of the memory size, but I can't rule that out either.

If you're lucky then you got windows to boot into the login screen at this point. If not... try to play with the VM setting. Determine (for sure) if windows are using the APIC or PIC HAL and adjust the VM settings accordingly. Check that MergeIDE was ran. Change the IDE adapter type or maybe even try the SATA one. Use Google.

8. Polishing

Try native boot. If you're lucky it should still work. (Naturally use the HW profile designated for native boot.)

If you got that far then most of the work is done. What else can be done:

1. Install the VirtualBox guest additions. It increases the performance and usability of the guest. There are some rumours that this would have negative impact on the native boot of windows. It worked ok for me, but if you wish you can run the additions only inside the VM -- see [2], section IV, 8.
2. Enable VirtualBox shared folders to share files between the host and the quest.
3. Move the VMDK image to virtual SATA controller. Now I'm not sure to what extent is that true, but rumour has it that virtual SATA has superior performance to virtual IDE. To move to virtual SATA proceed like this:
1. Shut down the VM.
2. Add a SATA adapter in the VM config. There is no need for any image to be connected to that. Leave the VMDK image hooked to the IDE adapter for now.
3. Boot the VM. You should see that windows detected new hardware. Dismiss all the dialogs and run the Intell Matrix Storage driver installer instead. Note: This mostly comes from a guide on r3dux.org. There's a warning that the current Intel Matrix Storage driver might not work with VirtualBox any more and a link to an older version is provided. I used that version and it worked for me. I don't know if the current one would or would not work.
4. Shut down the VM.
5. Now you can move the VMDK image to the SATA adapter.
6. That's it. Boot the VM again and hope it works.

I was lucky and managed all that while the windows installation can still boot natively. (Not that I need it, but it's not my computer, so I didn't want to break it.)

Good luck!

Feel free to post correction to comments. Thanks. :-)