Chad Hirschhttps://chad.hirsch.host/2022-01-27T20:48:00-08:00Bootable snapshots and Full Disk Encryption on the Pinebook Pro2022-01-27T20:48:00-08:002022-01-27T20:48:00-08:00Charimstag:chad.hirsch.host,2022-01-27:/bootable-snapshots-and-full-disk-encryption-on-the-pinebook-pro.html<h2 id="intro">Intro</h2>
<p>Ever since I got the pinebook pro, I have waited for the ability to
put /boot on the same filesystem as the OS on an ecrypted partition.
While it is not enough for fully trusted computing, having a single
partition containing the entire OS would be wonderful.</p>
<p>I have …</p><h2 id="intro">Intro</h2>
<p>Ever since I got the pinebook pro, I have waited for the ability to
put /boot on the same filesystem as the OS on an ecrypted partition.
While it is not enough for fully trusted computing, having a single
partition containing the entire OS would be wonderful.</p>
<p>I have been using archlinuxarm-pbp on my pinebook pro since the
beginning, I like to live on the edge, having the latest packages. This
also means though that I have to deal with some… possibility of
breakage, especially since I tinker (a lot).</p>
<p>Putting the whole OS on single block device allows me to use btrfs
snapshots with snapper and snap-pac so that when I perform updates, I
can automatically do a full OS snapshot, excluding my home directory,
logs, and package caches. Couple this with a bootloader which can
provide menus to choose various options, I can even boot the system
(readonly) from a previous snapshot, and restore the snapshot if I
like.</p>
<p>Today, I introduce the method I used to setup bootable snapshots on
an encrypted partition on the pinebook pro. This is not for beginners,
but I am happy to help if you have problems, reach out on Matrix or in
the Fediverse, or email.</p>
<h2 id="boot-loader">Boot Loader</h2>
<p>The team at https://tow-boot.org/ have done an excellent job with
Tow-Boot, which builds upon u-boot to provide some options more familiar
to the user who has worked on X86 machines. I chose to install tow-boot
to the SPI flash. You can download the release from here, or grab it
with archlinuxarm-pbp install like we do below.</p>
<h2 id="operating-system">Operating System</h2>
<p>I run <a href="https://github.com/SvenKiljan/archlinuxarm-pbp">archlinuxarm-pbp</a>.
It does a great job of providing me something minimal to build my custom
setup with. You will probably need to use the <a href="https://wiki.archlinux.org">Arch Wiki</a> to help with
installation.</p>
<h2 id="preparation">Preparation</h2>
<p>First, download the tar.gz from the <a href="https://github.com/SvenKiljan/archlinuxarm-pbp/releases">archlinuxarm-pbp
releases</a>, to get an idea of how it is typically installed, take a
look at the <a href="https://github.com/SvenKiljan/archlinuxarm-pbp/blob/main/INSTALL.md">installation
instructions</a></p>
<p>I am just going to post how I did it though. In order to install to
the emmc, we need an sdcard with archlinuxarm-pbp on it.</p>
<h2 id="standard-install-onto-a-microsd-card">Standard install onto a
microsd card</h2>
<p>First, follow the <a href="https://github.com/SvenKiljan/archlinuxarm-pbp/blob/main/INSTALL.md#installation-on-microsd-card-or-emmc-module">microsd
installation</a>, as that will become our installation media.</p>
<p>Once done, boot the pinebook pro from the microsd card, and follow
along</p>
<h2 id="installation">Installation</h2>
<h3 id="clean-the-emmc">Clean the emmc</h3>
<p>Wipe the beginning of the emmc ( this is to ensure no previous
bootloader signatures are there )</p>
<pre><code>dd if=/dev/zero of=/dev/mmcblk2 bs=1M count=32</code></pre>
<h3 id="partitioning">Partitioning</h3>
<p>Partition the disk, but we are going to do a slightly different
partition layout:</p>
<pre><code>fdisk /dev/mmcblk2</code></pre>
<p>Press <em>o</em> to make a dos partition table (this may work with
gpt as well) Press <em>n</em> to make a new partition Press <em>p</em>
to make it primary, 1 to make it the first, 2048 for the first sector
and +64M for the last sector. Press <em>t</em>, choos partition one, and
provide hex code <em>ef</em> for EFI ^^ The above creates a 64M size EFI
partition.</p>
<p><em>now is the time to make a swap partition if you want one, since
suspend-to-disk(hibernation) doesnt currently work on pbp, i skipped
this step</em></p>
<p>Press <em>n</em> to create a new partition Choose <em>p</em> to make
it primary, 2 to make it the second Press <em>enter</em> to let it
select the first sector Press <em>enter</em> again to select the last
sector Press <em>t</em>, choose 2 to select the 2nd partition, and use
hex code <em>83</em> for Linux.</p>
<p>Press <em>w</em> to write the changes when you are done</p>
<h3 id="filesystems">Filesystems</h3>
<p>We need to put a FAT EFI filesystem on the first partition.</p>
<pre><code>mkfs.vfat -F32 /dev/mmcblk2p1</code></pre>
<p>For the 2nd, we are going to put the filesystem on top of encryption,
lets setup encryption first:</p>
<pre><code># Note we use `--type luks1` because at the time of writing, grub may not support luks2, support should be there shortly.
# After examining `cryptsetup benchmark` aes-xts-plain with size 512 seems like a good compromise for this system. Feel free to make your own choices though.
cryptsetup luksFormat /dev/mmcblk2p2 --type luks1 --cipher aes-xts-plain64 -s 512</code></pre>
<p>Type all caps <em>YES</em> and hit enter</p>
<p>Enter the passphrase twice</p>
<p>Now lets open the new encrypted partition with the name
‘cryptroot’:</p>
<pre><code>cryptsetup luksOpen /dev/mmcblk2p2 cryptroot</code></pre>
<p>Enter the password to decrypt the partition.</p>
<p>Now lets put a btrfs filesystems on it:</p>
<pre><code>mkfs.btrfs /dev/mapper/cryptroot</code></pre>
<p>Mount the filesystem:</p>
<pre><code>sudo mount /dev/mapper/cryptroot /mnt</code></pre>
<p>We need to create some btrfs subvolumes so we can do snapshots
properly:</p>
<pre><code>btrfs subvolume create /mnt/@{home,root,snapshots,var_cache_pacman,var_log}</code></pre>
<p>Unmount the root of the btrfs device and remount it properly</p>
<p><em>We will mount the snapshots subvol later</em> <em>Compression is
not required here, but from my understating, compression can actually
speed up disk access, at the very least, this shouldnt hurt things much.
You can change it later too.</em></p>
<pre><code>umount /mnt
mount -osubvol=/@root,compress=zstd:1 /dev/mapper/cryptroot /mnt
mount -osubvol=/@home,compress=zstd:1 /dev/mapper/cryptroot /mnt/home
mount -osubvol=/@var_log,compress=zstd:1 /dev/mapper/cryptroot /mnt/var/log
mount -osubvol=/@var_cache_pacman,compress=zstd:1 /dev/mapper/cryptroot /mnt</code></pre>
<h3 id="installation-1">Installation</h3>
<p>We can just copy the installation from the microsd:</p>
<pre><code>rsync -aAXq --exclude={"/dev/*","/proc/*","/sys/*","/tmp/*","/run/*","/mnt/*","/lost+found"} / /mnt</code></pre>
<p>Install Tow-Boot to the SPI flash:</p>
<p><em>Be careful here, recovery from a bad flash can be a pain, make
sure you dont power off, and that the battery doesnt die during this
process, we are writing to a chip on the motherboard</em></p>
<pre><code>flash_erase /dev/mtd0 0 0
nandwrite -p /dev/mtd0 /mnt/boot/Tow-Boot.spi.bin</code></pre>
<p>We need to fix the fstab</p>
<pre><code>genfstab -U >> /mnt/etc/fstab</code></pre>
<h3 id="customization">Customization</h3>
<p>Now we need to chroot into our new installatation and take care of a
few things!</p>
<pre><code>arch-chroot /mnt</code></pre>
<p>We need to update our initramfs to support encryption of the root
filesystem, including display drivers so it can provide a password
prompt edit the lines of the following files:</p>
<p><strong>/etc/mkinitcpio.conf</strong></p>
<pre><code># Modules to include in the initramdisk
MODULES=(gpu_sched panfrost rockchipdrm drm_kms_helper dw_mipi_dsi hantro_vpu analogix_dp rockchip_rga panel_simple panel_edp arc_uart cw2015_battery i2c-hid iscsi_boot_sysfs jsm pwm_bl uhid)
# Hooks to use
HOOKS=(base systemd autodetect keyboard sd-vconsole modconf block sd-encrypt filesystems fsck)
#Optionally speed up boot and initramfs creation by setting the following:
COMPRESSION="cat"
</code></pre>
<p>Regenerate the initramfs:</p>
<pre><code>mkinitcpio -P linux</code></pre>
<p>We also need to install grub:</p>
<pre><code>pacman -S grub efibootmgr
#Symlink the kernel image because linux-manjaro gives it a weird name that won't be detected by grub.
ln -s /boot/Image /boot/vmlinuz-linux</code></pre>
<p>Edit the grub configurations, get the UUID first:</p>
<pre><code>blkid /dev/mapper/cryptroot</code></pre>
<p><strong>/etc/default/grub</strong></p>
<pre><code>GRUB_CMDLINE_LINUX="rd.luks.name=YOUR-UUID=cryptroot"
# You can also do this instead, allowing discards(trim) on the root filesystem (this has security implications)
GRUB_CMDLINE_LINUX="GRUB_CMDLINE_LINUX="rd.luks.name=YOUR-UUID=cryptroot rd.luks.options=allow-discards"
#This is also necessary:
GRUB_PRELOAD_MODULES="part_gpt part_msdos"
#Uncomment the following line:
GRUB_ENABLE_CRYPTODISK=y</code></pre>
<p>Install grub and generate its configuration:</p>
<pre><code>grub-install --target=arm64-efi --efi-directory=/boot/efi --bootloader-id=GRUB --recheck /dev/mmcblk2p2 --removable --modules="part_gpt part_msdos"
grub-mkconfig -o /boot/grub/grub.cfg</code></pre>
<h3 id="exit-and-reboot">Exit and reboot</h3>
<pre><code>exit # To leave the chroot
cd /
umount -R /mnt
reboot</code></pre>
<p>When the system boots, it should automatically go to emmc first, and
pop up a password prompt. Note, you will have to type the password
twice, and the first one takes a LONG LONG time, please be patient.</p>
<h3 id="snapper">Snapper</h3>
<p>Once we are booted, we can now setup snapper! Login with alarm:alarm
Become root with su -</p>
<pre><code>pacman -S snapper
snapper -c root create-config /
#Snapper automatically creates /.snapshots, but we dont want that as it makes complications for restoring a snapshot.
btrfs subvol remove /.snapshots
mkdir /.snapshots
# Mount our snapshots subvolume onto /.snapshots
mount -o subvol=/@snapshots /dev/mapper/cryptroot /.snapshots
# Add the new mount to fstab
mount | grep @snapshots >> /etc/fstab
</code></pre>
<p>Now we want to setup snap-pac, to make snapshots whenever we change
packages with pacman.</p>
<pre><code>pacman -S snap-pac
</code></pre>
<p>Then we can install snap-pac-grub from the AUR, I use an AUR helper
called pikaur, but you can use whichever works for you. This will
automatically update grub configuration when we make package
changes.</p>
<pre><code>pikaur -S snap-pac-grub</code></pre>
<p>If you aren’t using a helper, you can clone the PKGBUILD and just
create and install the package yourself. You may need to import a gpg
key, or add <code>keyserver-options auto-key-retrieve</code> to your
<strong>~/.gnupg/gpg.conf</strong> file.</p>
<pre><code>git clone https://aur.archlinux.org/snap-pac-grub.git
cd snap-pac-grub
makepkg -i</code></pre>
<p>After the install, it should have automatically created your grub
config with snapshots, and you should be able to boot the snapshots as
you like. We followed
https://wiki.archlinux.org/title/Snapper#Suggested_filesystem_layout to
make it easier to roll back to a previous snapshot, and boot from them,
but note that <code>snapper rollback</code> may not work as
expected.</p>
<h2 id="conclusion">Conclusion</h2>
<p>Thanks for following along. Hopefully, you should now be able to
setup your pinebook pro with bootable btrfs snapshots on an encrypted
partition. I want to drop some special thanks here for
[SvenKiljan[(https://kiljan.org/2021/06/20/arch-linux-arm-on-a-pinebook-pro/)
for maintaining archlinuxarm-pbp and the team at tow-boot for releasing
a bootloader build that could init the display and keyboard.</p>Commentary on Self-hosting Email2020-12-14T22:17:00-08:002020-12-14T22:17:00-08:00Charimstag:chad.hirsch.host,2020-12-14:/commentary-on-self-hosting-email.html<h2 id="intro">Intro</h2>
<p>Recently, I have heard several podcasts and keyboard warriors declare
that self-hosting email is:</p>
<ul>
<li>Not worth your time.</li>
<li>Very very hard.</li>
<li>a very fragile setup.</li>
<li>very unsafe.</li>
</ul>
<p>Yes, self-hosting email is possible. Done right, I think it can be
very much worth your time. It can be hard, but …</p><h2 id="intro">Intro</h2>
<p>Recently, I have heard several podcasts and keyboard warriors declare
that self-hosting email is:</p>
<ul>
<li>Not worth your time.</li>
<li>Very very hard.</li>
<li>a very fragile setup.</li>
<li>very unsafe.</li>
</ul>
<p>Yes, self-hosting email is possible. Done right, I think it can be
very much worth your time. It can be hard, but its not the hardest thing
I’ve done for sure. As well, for all its “fragility”, email was made for
a time when the internet was not even always on, nor did we all have
highly available mail services. Delivery is retried for quite a while on
most mailservers, so even if something unforseen happens, generally, it
all gets better rather quickly, and you have to make some pretty big
mistakes to lose email.</p>
<p>The biggest problem though isn’t actually getting on blocklists, or
losing emails (though these can be challenging)… the biggest problem
with self-hosting email is actually understanding the flow of email, and
what each part of this interdpendent system you are building actually
does. Without this understanding, its going to be very hard to
troubleshoot emails stuck in your mailqueue. You might follow a tutorial
and get it setup and working for that first test email, only to find you
have been missing things for a month.</p>
<h2 id="the-pitch">The Pitch</h2>
<p>I host email now for 4 different domains, on a single mail server. I
haven’t lost an email yet, nor have I been on any blocklists. I did have
some issues with a couple emails going to spam folders in the beginning,
but I am several months in now, and I really haven’t had many issues.
Even when sending an email to google.</p>
<h2 id="the-inconvenient-truths">The Inconvenient Truths</h2>
<p>Hosting your own email takes time to understand (fully and properly)
and time to build trust. You are going to have a bad day when you can’t
get a password reset email because your mail server is down. So here’s
the deal, the risk is on you, but so is the reward. Its going to take
time to learn to trust your mail server, learn its in and outs. You are
probably going to make some mistakes, its ok, brush yourself off and
keep going.</p>
<p>Spoilers: unless all your email is encrypted, its pretty much
guaranteed someone is collecting all the email data they can get. Your
ISP, your VPS provider, and probably even the hosts and ISP’s of the
addresses you are emailing. This is a problem for anyone using
unencrypted email.</p>
<h2 id="why-do-it">Why do it?</h2>
<p>For fun? For glory? Perhaps. But maybe just to reduce your reliance
on big name companies who read your every email. We can at the very
least make it harder for them to follow our converstations, read our
reciepts, and track our every waking moment.</p>
<h2 id="i-dont-even-know-where-to-start">I don’t even know where to
start!</h2>
<p>Thats the biggest part, if you are already overwhelmed, it might be
time to take a break or work on something else. These tools are a little
old, esoteric, and niche. People are not going to run to your aide when
you make a mistake (but you should be greatful for any help you can
get). At the end of the day, if you are committed to it, go ahead.
What’s the worst that could happen? I will be publishing a guide (not
tutorial) to understand and build(and maintain) your own email server
architecture. Stay tuned!</p>Introduction2020-12-10T22:20:00-08:002020-12-10T22:20:00-08:00Charimstag:chad.hirsch.host,2020-12-10:/introduction.html<p>Welcome,</p>
<p>My name is Chad, but I go by Charims online. Why? I don’t really
know. I have had the name for years.</p>
<p>I’m a privacy evangelist, which means I encourage people to find ways
to increase their privacy, on the internet or in their day-to-day lives.
I …</p><p>Welcome,</p>
<p>My name is Chad, but I go by Charims online. Why? I don’t really
know. I have had the name for years.</p>
<p>I’m a privacy evangelist, which means I encourage people to find ways
to increase their privacy, on the internet or in their day-to-day lives.
I believe in a right to privacy, and I strongly disagree with the
statements like “but I’ve got nothing to hide!”. Do I still use some
services that compromise my privacy, yes, but I have minimized it to
where I am comfortable, and continue to choose more privacy when I
encounter choices in my day-to-day life.</p>
<p>Privacy isn’t about suddenly locking down everything, its a choice
you make every day about what you want to share with the world. I want
to have the opportunity to choose what I share, and with whom. I want
others to have that opportunity as well. There are organizations who
want to take that choice away(and are generally succeeding), some are in
government, and some are corporations.</p>
<p>As well, I love computers (sometimes). I’ve been a GNU/Linux user
since 2003 (I was there for Warty Warthog), but my first installation
was Mandrake. I’ve always been a DIY type of person. I like to solve
challenging problems. I’ve been automating Linux in the enterprise since
probably around 2010. I’m not the biggest geek, but I do OK. I spent
some time in college, and while I learned a lot of computer science
theory, it was gumption and striving with peers and on my own that
taught me a lot more in the end.</p>
<p>I’m also a hacker. No, I don’t break into computers or anything like
that. I take disparate things and put them together in new and novel
ways. I use python like a handyman uses duct tape… OK, maybe not that
bad… Regardless, I get by, and I’ve built some pretty awesome things
along the way. My strengths really lie in automation and building simple
architectures. Somewhere along the way, computers got complex. Its up to
us to make them simple again. Great, you managened to spin up 17
different programs all to serve a single slow loading javascript-heavy
web page; I’m not impressed.</p>
<p>I run Archlinux everywhere I can. When i need a graphical UI, I run
with swaywm (thanks Drew and crew!) and alacritty for my terminal.</p>
<p>I like to tinker with developer products and single board computers,
like the rockpro64, and raspberry pi. I also have one of the first
shipments of the pinebook pro, and the braveheart edition pinephone.
Why? Because I want to help make this type of hardware better, and the
software that runs on it. Open source software runs better on open
source hardware.</p>
<p>Lastly, and of most importance, I’m a family guy. I have two
beautiful little girls and a gorgeous wife, who all struggle with more
than their fair share of health problems, but are stronger for it.</p>