Migrate/Re-use UmbrelOS SSD from Raspberry Pi to X86

I was running UmbrelOS on Raspberry Pi, with an external 2TB SSD disk. The machine is obviously under powered and started to show weakness. So I decided to migrate to a Lenovo mini-PC, M710q.

I need to re-use the same the SSD disk because M710q doesn’t have enough storage by itself (500GB). Ideally I can keep all the blockchain data and setup, so that I don’t have re-sync everything and re-setup everything.

While this may sound like a common question, I did not find many answers on the internet. The closest one is this one. However, UmbrelOS is currently at v1.x. A lot of don’t apply anymore.

Below is how I did it.

  • Prepare SSD disk
    • Upgrade Raspberry Pi to the latest UmbrelOS v1.1.
    • Shut it down via Umbrel Settings and unplug SSD disk.
  • Prepare PC
  • Migrate SSD disk over
    • Plug SSD disk into PC. Permanently mount it at /mnt/umbrel-ssd
      • /etc/fstab: UUID=e9ae3217-6a06-4721-b725-78e6524b4272 /mnt/umbrel-ssd ext4 defaults 0 0
    • sudo systemctl stop umbrel
    • cd /home/umbrel; mv umbrel umbrel.bak; ln -s /mnt/umbrel-ssd/umbrel umbrel
    • sudo systemctl start umbrel

It will take a long to re-start umbrel since it will fetch various containers etc. But it will use the same blockchain and other app-specific settings you had before on SSD disk. After a couple of coffee time, you will be all set!

Setup Bitcoin/ElectrumX/Lightning Network on Raspberry Pi 4

In the previous post I described the process of setting up a headless Raspberry Pi 4 (8GB) with remote VNC desktop access. It includes 1GB SDD drive, which is needed to run bitcoin full node with full archive of blockchain transactions.

In this post I will describe the process I used to set up a powerful multi-purpose Bitcoin node for the cryptocurrency community, that includes the following elements.

  • Bitcoin core – the default and standard bitcoin full node software that started the whole thing
  • ElectrumX server – de-centralized cryptocurrency wallet server. Wallet software needs it to make payments/transfers etc.
  • Lighnting Network Daemon (LND) – Level 2 network built on top of bitcoin to facilitate fast and cheap small amount payments.

Install and Run Bitcoin Core

wget https://bitcoin.org/bin/bitcoin-core-0.21.0/bitcoin-0.21.0-aarch64-linux-gnu.tar.gz
tar xzf bitcoin-0.21.0-aarch64-linux-gnu.tar.gz
sudo cp -a bitcoin-0.21.0/* /usr/local/
  • Run bitcoin-qt -txindex and start populating the blockchain data. It will take 2+ days and will consume >350GB in disk space!
    • DON’T SELECT “prune” MODE! Otherwise Electrum server won’t work.
  • Configure bitcoin for hardening and interactions with ElectrumX/LND
    • Create ~/.bitcoin/bitcoin.conf file. See sample setup below.
    • In this setup, we don’t listen to incoming peers over IPv4/IPv6 for enhanced privacy. New transactions will be broadcasted to Tor network only.
    • I also disabled wallet feature for better privacy again.
    • zmq (ZeroMQ) is needed for LND connection.
# index all transactions; LND/ElectrumX needs it
txindex=1

# bitcoin-qt accept RPC or not; needed for EPS/ElectrumX
server=1

# EPS needs wallet feature; electrumx does not
disablewallet=1

# don't broadcast transactions from our own wallet; will submit through Tor
# only useful when disablewallet==0
walletbroadcast=0

# # Maximum number of inbound+outbound connections. default 125
maxconnections=32

# enable electrumx server and LND
rpcuser=YOUR_RPC_USER
rpcpassword=YOUR_RPC_PASSWORD

# Listening for peers, enabled by default except when 'connect' is being used
# we don't export bitcoin ports or accept any incoming connections,
# because we would expose our own transactions to peers associated with our IP
# We only connect outgoing through tor
listen=0

# outgoing traffic use onion/tor only (default any)
proxy=127.0.0.1:9050
onlynet=onion

# debug logging - values: 0, 1, net, ....
debug=0

# needed by LND; I don't understand
zmqpubrawblock=tcp://127.0.0.1:28332
zmqpubrawtx=tcp://127.0.0.1:28333

Install ElectrumX Server

  • Install dependencies
sudo apt install -y python3-pip libleveldb-dev
sudo pip3 install aiohttp aiorpc
sudo pip3 install pylru
sudo pip3 install plyvel
  • Clone the the repo and install into /usr/local.
    • Note there are 2 primary electrumx repos on github.com. Make sure choose “spesmilo” one. The other one, albeit older, stopped supporting bitcoin due to recent segwit address introduction.
git clone https://github.com/spesmilo/electrumx.git
cd ~/electrumx
sudo python3 setup.py install
  • Add user to ssl-cert group to access the private SSL key
sudo usermod -a -G ssl-cert $USER
  • If bitcoin core has finished initial data sync, you can start ElectrumX server with the following script. Make sure you forward port 50001 and 50002 to the Raspberry Pi 4.
#!/bin/bash

# REQUIRED env variables
export DB_DIRECTORY=$HOME/.electrumx/db
export COIN=Bitcoin
DAEMON_URL="http://YOUR_RPC_USER:YOUR_RPC_PASSWORD@localhost:8332"

# Optional env variables
export PUBLIC_IP=$(curl -s http://whatismyip.akamai.com/)
export SERVICES=rpc://localhost:8000,tcp://:50001,ssl://:50002
export REPORT_SERVICES=ssl://$PUBLIC_IP:50002
export SSL_CERTFILE=/etc/ssl/certs/ssl-cert-snakeoil.pem
export SSL_KEYFILE=/etc/ssl/private/ssl-cert-snakeoil.key

# replace with your wallet address!
export DONATION_ADDRESS="bc1q52we5s8qyddhvrfscjn4qyn5nvq8hf92k9rt2w"

# log all sent transactions
electrumx_server

Install LND

Lightning Network in some sense is more complex than bitcoin. You are strongly suggested to review some basic concepts before setting up the software. There are multiple LN server implementations. LND is a popular one.

  • Install the pre-built binary image for ARM64
wget https://github.com/lightningnetwork/lnd/releases/download/v0.12.0-beta.rc6/lnd-linux-arm64-v0.12.0-beta.rc6.tar.gz
tar xzf lnd-linux-arm64-v0.12.0-beta.rc6.tar.gz
sudo cp lnd-linux-arm64-v0.12.0-beta.rc6/* /usr/local/bin
  • Set up config file
mkdir ~/.lnd
wget -O ~/.lnd/lnd.conf https://raw.githubusercontent.com/lightningnetwork/lnd/master/sample-lnd.conf
  • Below is a sample setup based on my own node
listen=0.0.0.0:9735
nolisten=false
rpclisten=0.0.0.0:10009
externalip=xxx.xxx.xxx.xxx
alias=YourFavoriateNameForThisNode
[Bitcoin]
bitcoin.active=true
bitcoin.mainnet=true
bitcoin.node=bitcoind
bitcoin.basefee=1
bitcoin.feerate=1
[Bitcoind]
bitcoind.dir=~/.bitcoin
bitcoind.rpcuser=YOUR_RPC_USER
bitcoind.rpcpass=YOUR_RPC_PASSWORD
bitcoind.zmqpubrawblock=tcp://127.0.0.1:28332
bitcoind.zmqpubrawtx=tcp://127.0.0.1:28333
bitcoind.estimatemode=ECONOMICAL
  • Run LND with “lnd” command
    • First time run, you will need to create a wallet, lncli create
    • For later running, you will need to unlock wallet, lncli unlock
      • If you like to automate LND startup, you will likely need lncli unlock --stdin option to pass in the wallet password via a script.
  • (Optional) if you like to set up a public LND node so that others can reach you, enable port forward on port 9735. You need to be a public node to participate in LN routing.

(Optional) Install RTL

RTL stands for “Ride The Lightning”, which provides a web interface to visually interact with LND. Otherwise you can just the commandline interface, which can be a little boring (and hard)

  • Install package and dependencies
git clone https://github.com/Ride-The-Lightning/RTL.git
cd RTL/
sudo apt install nodejs
sudo apt install npm
npm install --only=prod
cp sample-RTL-Config.json RTL-Config.json
vi RTL-Config.json
node rtl.js 
  • In order to run “node rtl.js” successfully, you need to configure RTL-config.json properly. Below is the sample setup derived from my own node.
{
  "multiPass": "YOUR_RTL_PASSWORD",
  "port": "3000",
  "defaultNodeIndex": 1,
  "SSO": {
    "rtlSSO": 0,
    "rtlCookiePath": "",
    "logoutRedirectLink": ""
  },
  "nodes": [
    {
      "index": 1,
      "lnNode": "Node 1",
      "lnImplementation": "LND",
      "Authentication": {
        "macaroonPath": "/home/ubuntu/.lnd/data/chain/bitcoin/mainnet/",
        "configPath": "/home/ubuntu/.lnd/lnd.conf",
        "swapMacaroonPath": ""
      },
      "Settings": {
        "userPersona": "MERCHANT",
        "themeMode": "DAY",
        "themeColor": "PURPLE",
        "channelBackupPath": "/home/ubuntu/.lnd/backup",
        "enableLogging": false,
        "lnServerUrl": "https://localhost:8080",
        "swapServerUrl": "https://localhost:8081",
        "fiatConversion": false
      }
    }
  ],
}
  • Once RTL starts running, you can open a browser and point at Raspberry Pi 4 IP address at port 3000. You see stats and perform actions with LND.

How You May Benefit

  • Use your own ElectrumX server for increased trust and privacy.
    • In this guide, we disable bitcoin core listening from public IP addresses. All new transactions are broadcasted from Tor. That increases privacy and anonymity.
  • You set up donation wallet address in ElectrumX. It seems people do donate, although not that common.
  • Once you set up LND properly, you can collect routing fees.

Show Your Support

This post is only complete if I can show you how I set up the donation and LN connection. 🙂 And your reading is only complete when you perform one of the following actions. 😛

For donation, Bitcoin wallet address is “bc1qyc48kmpweyx2kpqhq8n6r0ckr2fqsfvz2qxpx4”

For Lightning Network, my node public key is “0362a4372375bd24c5b8f8bb2ea85ae2ccf783808477e68cfb06121694b34d1927”

Appendix A – Reference Links on Lightning Network

Appendix B – How to create inbound liquidity for lightning network

AWS ARM64 vs. X86_64 – Bitcoin Performance Comparison

I’m running some experiments with bitcoin core software on AWS. I’m intrigued by 2 questions:

  • Which performs better, ARM64 vs X86_64?
  • What is performance loss of 32bit ARM vs 64bit ARM?

The Experiment

  • I fired up a t3.medium instance with ubuntu desktop 20.04(x86_64) image
  • I fired a second instance of t4g.medium type with ubuntu desktop 20.04 (arm64) image
  • To run 32bit ARM program, I followed the chroot approach documented in my previous post and set up an armhf Ubuntu 20.04 (focal) chroot environment
  • Then I download all 3 versions of bitcoin core software from its download site : 32bit ARM, 64bit ARM and 64bit Intel.
  • Run bitcoin-qt from scratch (delete ~/.bitcoin directory if any) and count the time duration for the verifying first 200,000 blocks.
    • Each configuration runs 2 times. We then take the average.
    • Other system variables are monitored to ensure similar running environment. Specifically networking or disk don’t seem to be a factor.

The Results

See results listed below. It seems for bitcoin related workload 64bit ARM version performs 20% faster than 32bit ARM version and 64bit Intel instance.

category64bit ARM 32bit ARM64bit Intel
Instance typet4g.mediumt4g.mediumt3.medium
CPU Graviton2Graviton2Intel Xeon Platinum 8000 series
# CPU cores222
RAM (GB)444
run duration #18’44” (524″)10’24” (624″)11’16″(676″)
run duration #27’30” (450″)9’58″(598″)9’04″(544″)
run duration average487″611″610″
relative performance to 64bit ARM100%80%80%
Pricing (us-west-2/Orgon) ($/hr)0.03360.03360.0416