Qubes: the ultimate hardware staking wallet

in #wallet7 years ago
  • This article is a work in progress, it will be completed soon with the missing parts!

Introduction:

Cryptocurrencies are heaps of fun! You first decide to buy 500$ of bitcoin, get it on some exchange you never heard about before, see that there is a lot of other coins in there, try to see what's the difference... then a few months/years later you realize that this thing is working: enters panic mode: how can I safely store your funds that are now far from 500$.
Internet is full of resources, each of them also linking to wallet stories that make you think twice about your options.
Lets try to sumarise:

  • Exchanges
    + handle the technical details for you
    + Many currencies at one place
    - You don't have your private keys
    - You (often) don't receive airdrops and forks linked to your hodings
    - No POS (not always)
    - No payouts/dividends
    - If like many people you use the same password on all your exchanges, and no double autentication, the risk exposure rises to a whole new level!
    - If the exchange disappears or gets hacked, so do your funds

  • Software Wallets
    + You own your keys
    + You get the airdrops and forks
    + POS
    + Payouts/dividends
    - Can use many resources
    - Can be technical to setup
    - Your computer get hacked and you potentially lose all wallets on that computer
    - format and loose your keys=loose you funds

  • Hardware Wallets
    + You own your keys
    + You get the airdrops and forks
    + Payouts/dividends
    - POS
    - Doesn't use resources
    - You might need to buy different hardware wallets for different currencies
    - No copies... loose the key or hardware failure, and so do the funds

  • Paper Wallets
    + You own your keys
    + Airdrops
    + Won't get hacked by it
    - No POS
    - one picture, or one file copied and your funds get stolen
    - one fire and your funds get lost

All these solutions where ok at first, and I tried them all except the hardware wallet, switching from the one to the other, and hesitating on which solution to choose for my funds, but as prices grew steadily the money at stake became more than what I can afford to lose.
That "* and your funds get lost" line was weighting more in my mind every day, becoming a main concern and preventing me to sleep - I had to find a solution: this is what I will describe here. In short: it is a POS compatible hybrid solution between a software wallet and a paper wallet.

The solution:

I considered running a bunch of raspberry pi, which would mitigate the risks of a hacked system would lead to the loss of multiple wallets, but on the one side, I don't really trust a micro ssd, don't want to install extra switches to scale up, an external raid array for each pi would be expensive, a nas would be a single point of hack again, a pi is easily stealable, costs would scale up with the number of wallets, ... so I decided to keep on searching.

Until I found Qubes. (https://www.qubes-os.org/)

Qubes is a XEN based operating system that allows you to run applications as separate virtual machines.
Qubes 4.0 is actually being tested in RC, it uses HVM/PVM mode instead of PV mode that was used in Qubes 3.2, this has a HUGE impact on security and is the reason why I didn't use Qubes before but am so excited now!

The concept is simple, on one big machine, I can run many wallets, that can not hardly interact with each other: if one gets hacked somehow (don't forget a lot of wallets are still beta), then at least it can not affect the others.
Special note: With the "latest" Intel news, it seems that some parts of memory might be readable through Meltdown/Spectre flaws: the Qubes team has devided to postpone 4.0RC4 release in order to address this issue: kudos.

Hardware:

While I tested Qubes on a normal laptop before going for this setup, this is what I ended up with.

- ASUS Z10PE D16 WS ("cheap" second hand from amazon)
- CPU 2XEON E5-2630L V4 ("cheap" second hand from Ebay) - (+watercooling)
- 128GB DDR4-2133 ECC (That's the one that hurts... - tried 32Gb at first... too short for my requirements)
- 1
M2 256Gb (used to install main qubes system)
- 4*3Tb 7400rpm (used in raid5 to install vms)
- Sapphire HD 6450 2GB DDR3 (onboard Intel or oldish AMD cards recommended for qubes; tried with a GTX1060: miserable failure)
- A few Corsair Flash Survivor Stealth 32Gb (for the later backups)
- Corsair HX1200I (compatible with the motherboard)
- A United Power Supply (Power is unstable on my mountain)

Intallation and base config:

Anyway, enough talking, lets go to the install - I assume you have some technical knowledge, in case something seems obscure, google is your friend!
This is not as much a complete guide as a list of the steps that worked out for my particular needs.
First get the latest Qubes iso from the website, check the checksum and make it in a bootable drive.
#dd if=Qubes-R4.0-rc3-x86_64.iso of=/dev/yourusbstick

Boot and go through standard install, installing on the M2 memory (/dev/sda).
Important note: At time of writing, the chosen keyboard is not active during install, so DOUBLE CHECK your disk encryption and user password!

Once installed, boot the system and login.

In my particular case, my hardware supports msix but not msi, the symptom I experience is a constant restarting of the network stack.
I need to start the dom0 terminal emulator (start terminal emulator), and do:
#qvm-prefs sys-net kernelopts "nopat iommu=soft swiotlb=8192 pci=nomsi
then reboot

Once rebooted, I can configure my network - I like tp use 802.3ad (bonding - trunking - lacp), for this go to start/sys-net/network-manager, create a bond, add the ethernet interfaces in the bond, and be carefull to enter the same mac in the address to clone of each interface. Apply and done.
With the network working, we can update qubes, in the dom0 terminal emulator:
#qubes-dom0-update --enablerepo=qubes-dom0-current-testing
and also because I personally like it more than the default options, get the debian stretch template:
#qubes-dom0-update --enablerepo=qubes-dom0-current-testing qubes-template-debian-9
and reboot
At this point, I delete the existing vault and create a new vault, as an independent VM based on debian 9 (the default vault runs fedora25)
I like to keep my M2 "low usage"; so the rest of the VM's will go on a raid5, in the dom0 terminal:
assuming raid 5 disks are sdb sdc sdd and sde,
# gdisk /dev/sdb;gdisk /dev/sdc;gdisk /dev/sdd;gdisk /dev/sde
in gdisk, assuming you are on the right disk and want to dedicate the disk as 1 whole partition to the raid, you should probably do something like this:
o enter
d enter
n enter enter enter
disk type 8300 (linux raid)
w enter
yes

After having done the 4 drives, and without reason, I rebooted, and went for a cup of tea.
Now lets create the raid:
# mdadm --create /dev/md0 --level=5 --raid-devices=4 /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sde1
check the progress with
# watch -n1 cat /proc/mdstat # wait for raid to complete!!!!
enjoy the cup of tea, a couple of movies, and some more time

Once the raid has done initiating, we will create the raid partition:
Important note: The next command asks for a capital YES
#cryptsetup -v --hash sha512 --cipher aes-xts-plain64 --key-size 512 --use-random --iter-time 5000 --verify-passphrase luksFormat /dev/md0
and once the partition is crypted, we can add it to crypttab:
#echo "raid_pool $(blkid /dev/md0|cut -f2 -d\") none" >> /etc/crypttab
we can now open the device and create a Physical Volume (pv):
#cryptsetup open /dev/md0 raid_pool
#pvcreate /dev/mapper/raid_pool
and check the pv with
#pvs

Now that we have a the PV, we can define a Volume Group (VG) into it:
#vgcreate raidpool /dev/mapper/raid_pool
And once we have the VG, we can define a Logical Volume (LV) into it:
#lvcreate -L 8.15T --thinpool raidpoolthin raidpool

Now that we have created a logical volume as a thinpool provision (in short: it will only use what the VM's really use, not the total allocated space),
we can allocate this pool to qubes for the creation of VM
#qvm-pool -o thin_pool=raidpoolthin,volume_group=raidpool -a raidpoolthin lvm_thin
define it as the default storage:
#qvm-prefs --set default_pool raidpoolthin
set debian9 as the default VM template:
#qvm-prefs --set default_template debian-9

#BUG: for some reason it does not mount at boot... meaning that at boot I do
#cryptsetup open /dev/md0 raid_pool
... and then wait a bit so the system takes it into account
I'll update this guide when I find the solution ;p

VM creation:

This is quite straightforward, click on "Create Qubes VM" in the menu.
I know it is stupid overkill in many ways, but I like to make independent template based vm's. The reason is that most wallets are in heavy development and sometimes require specific library versions to be built, which are not always compatible between each other.
In order to avoid potential dependence hell, and because I hace enough space on the raid, don't do APP vm's.
Also, while creating the vm, click on open settings after creation, which allows you to change allocated disk space (some wallets require 150+Gb blockchains, some are happy with a few Gb).
And before applying the space, go in the applications tab to add a the tabs that you want in the vm menu (I usually use a terminal)
Once the VM is created, you can use it as any other

Backup:

stub: this part goes about the vault and how I manage my backups, and will be written in the coming days

As with all wallets setup in the virtual machines, now about my key backup system.
I created a pair of scripts, one to manage my passwords, and the other to mount a crypted file as a secure filesystem.
The directory is then copied to corsair survivor usb drives that I can keep in safe places. The pendrive contains all the keys in encrypted format, as well as the scripts to open them, and the tool to manage passwords.
https://github.com/jupiter126/blortchzj

Wallet instalation:

In the following days, I will go in the specifics of installing and backupping the following wallets, and maybe more.

  • MEW
  • Ripple
  • IOTA
  • Jaxx
  • NEON
  • pivx
  • qrl
  • Stellar
  • ...?

Tips'n Tricks

Quite short, I intend to put some more with the time...

  • The Qubes manager gives you a good overview of running vm's and ram usage
  • lvs and lvdisplay give good information about drive usage
  • qubes-prefs is helpful to set template preferences
  • qvm-copy allows to copy files from one vm to another
  • control shift c control shift v allows to copy clipboard from one vm to another

Final Notes:

The Qubes online documentation is mindblowing, most my questions where answered before I even asked them, and while some of the answers did not apply to the new 4.0 releases (still in RC), quite a lot of the topics have a 4.0 subsection, explaining the difference and evolution.
The Qubes community is absolutely stunning, aside from maintaining that documentation, I found very knowledgeable and helpful people in the freenode #qubes irc channel!!!