{{Header}} {{Title|title= Full Raw Disk Backup }} {{#seo: |description=1 to 1 disk backup |image=Backupraw123123.jpg }} [[File:Backupraw123123.jpg|thumb]] {{intro| 1 to 1 disk backup }} = Introduction = TODO: what is a full raw disk backup It is not possible to backup an operating system installed on an internal disk such as for example Qubes OS while that operating system is running from that operating system. The advantage of a full raw disk backup is that it should be bootable either by: * disconnecting the internal disk and booting from the external backup disk, OR * changing the disk ids ([[undocumented]]) (of either the internal disk or external disk after the backup), OR * using a different computer with compatible hardware to boot from the external disk. Unfortunately without either above method any attempt to boot the external drive after the backup might likely lead to actually boot the internal drive because these share the same disk ids after the raw disk backup. = Prerequisite Knowledge = * How to install an operating system (Debian) on a external USB drive. = Prerequisite Exercises = == echo Output Redirection Essentials == This should be exercised in a safe environment such as in a disposable VM. '''1.''' Learn the syntax. Do not run the following command. * > - the greater-than sign - means the output file to write to. * {{CodeSelect|code= echo "input content" > path-to-file-name }} '''2.''' Exercise the redirect to file redirection. {{CodeSelect|code= echo "test output" > test-output-file }} '''2.''' View the created file. {{CodeSelect|code= cat test-output-file }} '''3.''' Sample printout. 
test output
'''4.''' Exercise more. Based on above example, keep changing the file content and output file names until this becomes clear. == PipeViewer Read-Only Essentials == This should be exercised in a safe environment such as in a disposable VM. '''1.''' Create a new file such as for example: {{Open File|filename= file-input }} '''2.''' Add some contents such as for example: {{CodeSelect|code= test input }} '''3.''' Save and close the file: '''4.''' Learn the syntax. Do not run the following command. * < - the less-than sign - means the input file to read from. * {{CodeSelect|code= pv < path-to-file-name-to-read-from }} '''5.''' Test reading from the file. {{CodeSelect|code= pv < file-input }} '''6.''' Read the output. Sample output. 
test input
11.0 B 0:00:00 [34.9KiB/s] [================================================================>] 100%
'''7.''' Exercise more. Based on above example, keep changing the input file names until this becomes clear. == PipeViewer Read-Write Essentials == This should be exercised in a safe environment such as in a disposable VM. '''1.''' Create a new file such as for example: {{Open File|filename= file-input }} '''2.''' Add some contents such as for example: {{CodeSelect|code= test input }} '''3.''' Save and close the file. '''4.''' Learn the syntax. Do not run the following command. * < - the less-than sign - means the input file to read from. * > - the greater-than sign - means the output file to write to. * {{CodeSelect|code= pv < path-to-file-name-to-read-from > path-to-file-name-to-write-to }} '''5.''' Test reading from an input file and redirecting the output to another output file. {{CodeSelect|code= pv < file-input > file-output }} '''6.''' Read the output. Sample output. 
11.0 B 0:00:00 [ 149KiB/s] [================================================================>] 100%
'''7.''' Compare the files. View the two different files. These should look the same. {{CodeSelect|code= cat file-input }} {{CodeSelect|code= cat file-output }} '''8.''' Exercise more. Based on above example, keep changing the input files and output files until this becomes clear. = Prerequisites = * An installed operating system (such as for example Qubes OS) on an (internal) disk. * An operating system installed on an external disk, most likely USB such as for example Debian. * A separate external backup disk where the backup should be stored. = Information Gathering = == Qubes Users Recommendations == For educational purposes, it is useful to run gparted and gnome-disks from the Qubes installation which should backed up as full raw disk backup. Tested in Qubes R4.0 only. In later Qubes versions with untrusted storage domain, these instructions might need changes. gparted is a disk partitioning tool which will be used as an easy way to find out how many hard drives the system has and what size they have. gnome-disks is a similar utility. Disconnect any unneeded storage devices. In dom0, install gparted and gnome-disks. {{CodeSelect|code= sudo qubes-dom0-update gparted gnome-disk-utility }} In dom0, run gparted. {{CodeSelect|code= sudo --set-home gparted }} Check the arrow down button below the X (which would close the window) if there are multiple disks. In dom0, run gnome-disks. {{CodeSelect|code= gnome-disks }} Make notes. For example. Note: Modify these notes according to what can be seen in gparted and gnome-disks. 
/dev/sda (476.91 GiB)
Obviously easiest if there is only 1 disk. Assuming there is only 1 disk. = Recommendations = '''1.''' Exercise with completely different test hardware. It is recommended against to apply this procedure with production hardware if doing this procedure for the first time. Since data loss is possible if making a mistake during the raw disk backup procedure, it is recommended to exercise the procedure with completely different hardware. Such as a second computer as well as an external boot drive and backup disk that does not contain any important data. '''2.''' Creating a backup date note file. After booting from the internal disk (which should be backed up). Create a text file with a small explanation for yourself "Today is day x with date y and time z prior backup number 1." This will later be handy when doing a restoration test. '''3.''' USB boot operating system should have a graphical diff viewer such as meld installed as well as the following tools. After booting the operating system from USB. {{Install Package| package=meld lxqt-sudo gnome-disk-utility gsmartcontrol gparted pv lshw hwinfo ddrescue }} = Backup High Level Overview = '''1.''' Boot from external USB drive. '''2.''' Find out the device paths of the internal drive and the USB boot drive. '''3.''' Find out the device path of the USB backup drive. '''4.''' Use pv to read from the internal drive and to write to the USB backup drive. '''5.''' Restoration test. (Optional but highly recommended.) = Backup Instructions = In Linux, unfortunately device names and device paths are non-deterministic, unpredictable, might change with kernel versions and operating system upgrades. A raw backup with the pv can lead to data loss if used incorrectly as pv is a very powerful tool. For example, sd'''a''' is a device name and /dev/sd'''a''' is a device path. Other device path examples are dev/sd'''b''', dev/sd'''c'''. The actual pv command is not very difficult but the device paths need to be carefully determined before starting the backup, otherwise data loss is at risk. '''1.''' Boot from external USB disk. '''2.''' Do not attach any other disks at this time. If any other disks are already attached, remove them for now for simplicity. '''3.''' Write output of fdisk to file "old". {{CodeSelect|code= sudo fdisk -l > old }} '''4.''' Information gathering with a few hard drive utilities. Using lshw. {{CodeSelect|code= sudo lshw -class disk }} Using hwinfo. {{CodeSelect|code= sudo hwinfo --disk }} See what disks are currently attached with an alternative tool such as gnome-disks as well. Try: {{CodeSelect|code= gnome-disks }} Try gsmartcontrol. {{CodeSelect|code= gsmartcontrol-root }} See what disks are currently attached with an alternative tool such as gparted as well. Try: * Start gparted from start menu if it can be found there. * Try with pkexec if that works: {{CodeSelect|code= /usr/sbin/gparted }} * Or try with lxsudo if that works: {{CodeSelect|code= lxsudo /usr/sbin/gparted }} '''4.''' Attach another disk, the external USB disk which should be used for the backup. '''5.''' Write output of disk to file "new". {{CodeSelect|code= sudo fdisk -l > new }} '''6.''' Compare file "old" with file "new" using "diff". {{CodeSelect|code= diff old new }} '''7.''' Compare file "old" with file "new" using "diff". {{CodeSelect|code= meld old new }} '''8.''' View contents of file /etc/fstab. {{CodeSelect|code= cat /etc/fstab }} Example printout. 
# /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for a
# device; this may be used with UUID= as a more robust way to name devices
# that works even if disks are added and removed. See fstab(5).
#
#                
/dev/mapper/debian--vg-root /               ext4    errors=remount-ro 0       1

# /boot was on /dev/sdb1 during installation
UUID=86983f37-38db-401f-889a-bc93d83a3be4 /boot           ext2    defaults        0       2

#/dev/mapper/debian--vg-swap_1 none            swap    sw              0       0
Watch out for the UUID= field. In above example it is 86983f37-38db-401f-889a-bc93d83a3be4. '''8.''' View output of the blkid. For informational purposes only. {{CodeSelect|code= sudo blkid }} '''9.''' Confirm the device path of the boot device. Compare output of file /etc/fstab with blkid command. For example. Note: replace UUID with the actual UUID from /etc/fstab. {{CodeSelect|code= sudo blkid {{!}} grep 86983f37-38db-401f-889a-bc93d83a3be4 }} Sample printout. 
/dev/sdb1: UUID="86983f37-38db-401f-889a-bc93d83a3be4" TYPE="ext4" PARTUUID="decf0fe9-01"
In above example, the device path of the boot disk is /dev/sd'''b'''. Not /dev/sd'''b'''1. The 1 means partition number 1. When making raw disk backups of the full disk, partition numbers must be omitted. Otherwise it would just be a partition backup. In this case, the backup would be unbootable and couldn't be easily restored. '''10.''' Make some notes such as. 
Qubes internal disk 476.91 GiB
Debian external boot disk 931.42 GiB
Backup external disk 931.41 GiB
'''11.''' Note the device paths. Write down the device paths of the internal disk, the external USB boot disk and the external USB backup disk with help according the above instructions. For the author of this wiki page, /dev/sd'''a''' was the (Qubes) internal disk, /dev/sd'''b''' the (Debian) USB boot disk and /dev/sd'''c''' the USB backup drive. This might be different for readers! '''12.''' Note that device paths can change after reboot. Note that device paths are unfortunately unstable. It happened to the author of this wiki page that /dev/sd'''a''' and /dev/sd'''b''' paths were swapped. It is therefore required to re-identify device paths every time the computer is rebooted. '''13.''' Become root. Using sudo is unfortunately not possible. At least not without complicating the following pv command. Becoming root is required because shell redirection are a bash feature, not a sudo feature. These are set up before becoming sudo. Alternatively, it would be possible to write a small script, pasting the command there and executing that script using sudo. {{CodeSelect|code= sudo su }} '''14.''' Backup. Explanation: * < - the less-than sign - means the input device to read from * > - the greater-than sign - means the output device to write to. Syntax:
pv < /dev/xxx > /dev/yyy
Note: * Replace /dev/xxx with the actual device path of the drive which should be backed up. * Replace /dev/yyy with the actual device path of the drive where the backup should be stored. Examples: Notice: DATA LOSS POSSIBLE if used incorrectly! Do not use this without prior verification of the device paths! *
pv < /dev/sda > /dev/sdc
*
pv < /dev/nvme0n1 > /dev/sdc
* https://unix.stackexchange.com/questions/433443/ssd-backup-image * It is unfortunately not possible to compare. Running diff /dev/sda /dev/sdc for backup verification is not possible because of different disk sizes. The backup might take a long time. '''15.''' Check exit code. {{CodeSelect|code= echo $? }} Expected output if success. 
0
'''16.''' Switch back to normal user. If the user was previously running sudo su, it should now be undone. Switch back to normal user by running the following command. {{CodeSelect|code= exit }} '''17.''' Done. Backup is complete. '''18.''' Restoration test. Without restoration test, it's unclear if the backup could be restored in case needed. ---- = Backup Verification = == verification without progress meter == {{CodeSelect|code= size=$(sudo blockdev --getsize64 /dev/nvme0n1) }} {{CodeSelect|code= sudo cmp --bytes=$size /dev/nvme0n1 /dev/sdb }} == verification with progress meter == Small script with progress meter. Untested. original_disk and backup_disk need to be adjusted. {{CodeSelect|code= #!/bin/bash set -x set -e set -o pipefail original_disk=/dev/nvme0n1 backup_disk=/dev/sdb test -e "$original_disk" test -e "$backup_disk" ## Get the size of the original disk in bytes. original_size=$(blockdev --getsize64 "$original_disk") ## Using cp with --bytes="$original_size" to ensure that the comparison stops ## at the size of the original disk, even if the backup disk is larger. ## This way, the comparison is not confused if the backup_disk is larger. pv "$original_disk" {{!}} cmp --bytes="$original_size" /dev/stdin "$backup_disk" ## Alternative. Untested. #ddrescue --verbose --no-scrape --size="$original_size" "$original_disk" "$backup_disk" ~/logfile }} = Restoration Test High Level Overview = Before a restoration test can be performed, a backup is required. Once a backup has been created, attempt the following restoration test instructions. '''1.''' Boot from internal USB drive. '''2.''' Create a (or update) a backup date note file. Create a text file with a small explanation for yourself "This is prior restoration test. The current date is replace-actual-date. The current time is actual-time. This note should no longer exist after the restoration test." '''3.''' Shut down. ---- '''4.''' Boot from external USB drive. '''5.''' Find out the device paths of the internal drive and the USB boot drive. '''6.''' Find out the device path of the USB backup drive. '''7.''' Use pv to read from the USB backup drive and to write to the internal drive. (Vice versa backup procedure.) '''8.''' Shut down. ---- '''9.''' Boot from internal disk to test if the restoration was successful. '''10.''' Check the backup date note file which was created in step 2. The note "This is prior restoration test." is now expected to be gone. If the note still exists, then restoration has not been successful. Make sure step 2, the update of the backup date note file has not been previously skipped. '''11.''' Done. Restoration test has been completed. = Restoration Example = WARNING: UNTESTED NOTE: Modify for your needs. See above. {{CodeSelect|code= size=$(sudo blockdev --getsize64 /dev/nvme0n1) }} {{CodeSelect|code= sudo su }} 
pv -pterb -s "$size" /dev/sdb > /dev/nvme0n1
= Qubes Specific = * [https://github.com/QubesOS/qubes-issues/issues/8363#issuecomment-1637154529 Raw disk backup of Qubes EFI is unbootable] = Alternatives = Potential alternatives, [[untested]] by the author of this wiki page. https://www.tecmint.com/linux-disk-cloning-tools/ = Footnotes = {{Footer}} [[Category:Documentation]]