Copying a running VM to external host

Friend had a small incident the other day where KVM/QEMU seg-faulted and lost communication with a running VM.  The VM was still running but it isn't writing to persistent storage.  We decided to do a live backup of the running VM.  Although this is definitely NOT recommended, we were left with no other choice as shutting down the VM would mean losing all changes made. 

The VM is running Linux with non LVM-ed partitions.

To start, run the following command as root in the VM to make a copy of the drive and send it over SSH to an external host:

dd if=/dev/sda bs=1k conv=sync,noerror | gzip -c | ssh -c blowfish myuser@extern.host "dd of=/images/damaged-vm.gz bs=1k"

Once that's done, access the host where the file is transferred to and decompress the file:

# gunzip damaged-vm.gz

You can try to either boot up the image via virsh or mount it via loopback to copy any files.  We chose the latter. 

Run the following commands as root.

Find the first unused loop device:

# losetup -f
/dev/loop0

Now we setup the loop device against the image that was transferred:

# losetup /dev/loop0 /images/damaged-vm

Use kpartx to map the partitions:

# kpartx -av /dev/loop0
add map loop0p1 : 0 29333504 linear /dev/loop0 2048
add map loop0p5 : 0 1380352 linear /dev/loop0 29337600

Proceed to mount the partitions:

# mount /dev/mapper/loop0p5 /mnt/p5

References:

• http://www.inference.phy.cam.ac.uk/saw27/notes/backup-hard-disk-partitions.html