Upgrading Mastodon on FreeBSD

Update: On 2017-09-15 the FreeBSD Mastodon port, net-im/mastodon, was overhauled with version 1.6.0 to handle installation of Mastodon's node modules and assets. This installation guide has been updated to reflect those changes.

What follows in based on the official Mastodon release notes.

Stop the three Mastodon servers.
# service mastodon_web stop
# service mastodon_workers stop
# service mastodon_stream stop
Some special steps are required when upgrading from a version less than 1.6.0 to version 1.6.0 or greater. You should first delete the old mastodon package.
# pkg delete mastodon

Move the leftover files under Mastodon's installation directory out of the way, but keep your customized .env.production and nginx-include.conf and keep everything under public/system.

Install the new Mastodon package.
# pkg install mastodon
Upgrade your packages.
# pkg upgrade
Switch to the mastodon user.
# su - mastodon
Remove the old Gemfile.lock.
% rm Gemfile.lock
If you are upgrading for the first time from a version less than 1.4.3, an extra step needs to be run before the database migration.
% RAILS_ENV=production rails mastodon:maintenance:prepare_for_foreign_keys
If you are upgrading for the first time from a version less than 1.5.0, to enable Web Push notifications, you need to generate a few extra secrets and put them in .env.production.
% RAILS_ENV=production rake mastodon:webpush:generate_vapid_key
Check the release notes to see if the database migration script needs to be run for the new release.
% RAILS_ENV=production rails db:migrate
If you ran the database migration script, also clean out old preview cards thumbnails.
% RAILS_ENV=production rails mastodon:maintenance:remove_deprecated_preview_cards

Do a diff between nginx-include.conf.sample and nginx-include.conf to check for changes. Update nginx-include.conf if necessary.

Return to root privileges.

If you made changes to nginx-include.conf, reload the nginx configuration.
# service nginx reload
Finally, restart the Mastodon daemons.
# service mastodon_web start
# service mastodon_workers start
# service mastodon_stream start

Posted 2017-04-27 12:55 | Comments

Running Mastodon on FreeBSD

Update: On 2017-09-15 the FreeBSD Mastodon port, net-im/mastodon, was overhauled with version 1.6.0 to handle installation of Mastodon's node modules and assets. This installation guide has been updated to reflect those changes.

This guide describes how to to get Mastodon running on FreeBSD using the FreeBSD port/package. What follows is based on the official Mastodon production Guide.

Install the packages

# pkg install nginx postgresql95-server postgresql95-contrib mastodon

Enable and start required services

# sysrc redis_enable="YES"
# service redis start
# sysrc postgresql_enable="YES"
# service postgresql initdb
# service postgresql start
The Mastodon port ships with two sample nginx configuration files, a complete nginx.conf and nginx-include.conf, which mostly just includes the server block. If the web server is going to be dedicated to Mastodon, you can create a new nginx profile.
# sysrc nginx_profiles=mastodon
# sysrc nginx_mastodon_configfile="/usr/local/www/mastodon/nginx.conf"
If you prefer to continue using your current nginx.conf, you can add the line below to it. Make sure you put it inside the http block.
include /usr/local/www/mastodon/nginx-include.conf;
In either case, you need to customize nginx-include.conf. At minimum, you will have to change all instances of example.com. Once you are satisfied with its configuration, start nginx.
# sysrc nginx_enable="YES"
# service nginx start

Create mastodon database user

# psql -d template1 -U pgsql -c "CREATE USER mastodon CREATEDB;"

Switch to the mastodon user

# su - mastodon

Customize .env.production

Customize .env.production to suit your needs, but you must at least set values for LOCAL_DOMAIN and SMTP_FROM_ADDRESS. You also have to generate secrets for PAPERCLIP_SECRET, SECRET_KEY_BASE, and OTP_SECRET. Generate a different secret for each of these fields by running the command below three time. Save .env.production before moving on to the next step.
% RAILS_ENV=production rake secret
If you are installing version 1.5.0 or later, to enable Web Push notifications, you need to generate a few extra secrets and put them in .env.production.
% RAILS_ENV=production rake mastodon:webpush:generate_vapid_key

Set up the database for the first time

% RAILS_ENV=production rails db:setup

Start the Mastodon daemons

Run the following commands with root privileges.
# sysrc mastodon_web_enable="YES"
# sysrc mastodon_workers_enable="YES"
# sysrc mastodon_stream_enable="YES"
# service mastodon_web start
# service mastodon_workers start
# service mastodon_stream start

Give users administration rights

After the user alice has created an account through the web interface, you can give her administration rights by using the command below.
# sudo su - mastodon
% RAILS_ENV=production rails mastodon:make_admin USERNAME=alice

Posted 2017-04-23 21:18 | Comments

ZFS Snapshot Management and Replication with zap

In this post, I describe how I manage ZFS snapshots and remote replication from a source host, phe, to a target host, bravo. I assume you have version 0.7.3 or newer of zap installed via the FreeBSD package and the sudo command is available.

Target host (bravo)

Partition the disks to be used for the backup pool.

# for i in 1 2; do
    gpart create -s gpt ada$i
    gpart add -a 1M -l gptboot$i -t freebsd-boot -s 512k ada$i
    gpart bootcode -b boot/pmbr -p /boot/gptzfsboot -i 1 ada$i
    gpart add -a 1m -l zfs$i -t freebsd-zfs ada$i
  done

Create the backup pool and a top-level filesystem with the necessary permissions. Make the backup datasets readonly, otherwise incremental replication may fail.

# zpool create -m none zback mirror gpt/zfs1 gpt/zfs2
# zfs create -o readonly=on zback/phe
# zfs allow -u zap canmount,compression,create,mountpoint,mount,receive,refreservation,userprop,volmode zback/phe

Source host (phe)

Set permissions on the datasets to be replicated.

# zfs allow -u zap bookmark,diff,hold,send,snapshot zroot/ROOT
# zfs allow -u zap bookmark,diff,hold,send,snapshot zroot/usr/home
# zfs allow -u zap bookmark,diff,hold,send,snapshot zroot/var

Set the zap:snap user property to on for datasets to snapshot. The property will be inherited, so explicitly turn it off for datasets that we do not want to snapshot.

# zfs set zap:snap=on zroot/ROOT zroot/usr/home zroot/var
# zfs set zap:snap=off zroot/var/crash zroot/var/tmp zroot/var/mail

Create the first snapshots.

# sudo -u zap zap snap -v 1w

Set the zap:rep property for datasets that will be replicated to bravo.

# zfs set zap:rep='zap@bravo:zback/phe' zroot/ROOT zroot/usr/home zroot/var
# zfs set zap:rep=off zroot/var/crash zroot/var/tmp zroot/var/mail

Before replicating datasets, key-based ssh authentication to bravo must be set up. An important decision at this point is whether or not to protect the private key that will be generated with a passphrase. If anyone else has access to the source host, the decision is clear: add a passphrase. The downside to adding a passphrase is that you will have to enter it before you can replicate. The ssh-agent makes this more manageable.

# sudo -u zap
zap@phe % ssh-keygen

Manually copy the contents of the newly created public key on the source host (~/.ssh/id_rsa.pub by default) to ~/.ssh/authorized_keys on the target host and confirm that you can ssh from the source host to the target host using key-based authentication.

Since this is the first time these datasets are being replicated to bravo, a full stream will be sent.

zap@phe % zap rep -v

Target host (bravo)

Optionally mount the backup datasets under /zback/phe.

# mkdir -p /zback/phe
# zfs set mountpoint=/zback/phe zback/phe/ROOT/default
# zfs mount zback/phe/ROOT/default
# zfs set mountpoint=/zback/phe/var zback/phe/var
# zfs mount zback/phe/var
# zfs set mountpoint=/zback/phe/usr/home zback/phe/usr/home
# zfs mount zback/phe/usr/home

Since I also use zap on bravo, I turn off the zap:snap and zap:rep properties for the backups.

# zfs set zap:snap=off zback/phe/ROOT zback/phe/usr/home zback/phe/var

Source host (phe)

Create new snapshots and send the incremental changes to bravo.

zap@phe % zap snap -v 1d
zap@phe % zap rep -v

Automate rolling spanshots and replication with cron. Taking snapshots is normally cheap, so do it often. Destroying snapshots can thrash disks, so I only do it every 24 hours. Sensible replication frequencies can vary with different factors. Adjust according to your needs.

Edit the crontab for the zap user by adding the contents shown below.

zap@phe % crontab -e
#minute	hour	mday	month	wday	command
    
# take snapshots
*/5	*	*	*	*	zap snap 1d
14	*/4	*	*	*	zap snap 1w
14	00	*	*	1	zap snap 1m

# replicate datasets
54	*/1	*	*	*	zap rep -v

Edit the crontab for the root user by adding the contents shown below.

# crontab -e
#minute	hour	mday	month	wday	command  

# destroy snapshots
44	04	*	*	*	zap destroy

Posted 2017-01-04 12:22 | Comments

ZFS on Root and Full Disk Encryption: FreeBSD 10.3 to 11.0 or One pool to rule them all

Thanks to Allan Jude for steering me through this on IRC and Warren Block for his feedback.

The new boot loader in 11.0 is able to boot encrypted ZFS pools directly. Yes, that means you can have full disk encryption (FDE) with ZFS on root and boot environments (BEs)! However, after you upgrade from 10.3, some tinkering is necessary to get this working. The instructions that follow are for a ZFS mirror installation. The two disks (ada0 and ada1) each have the same partition layout: p1: freebsd-boot, p2: freebsd-zfs (boot pool), p3: swap, p4: freebsd-zfs (main pool). Specify your disk(s) and partition indices according to your setup.

Reencrypt the master key with only a passphrase. You can use the same passphrase as before.

# geli setkey -k /boot/encryption.key ada0p4
# geli setkey -k /boot/encryption.key ada1p4

Set the geliboot flag.

# geli configure -g ada0p4
# geli configure -g ada1p4

Remove the /boot symbolic link pointing to /bootpool/boot and copy /boot from /bootpool/ to /.

# rm /boot
# cp -r /bootpool/boot /

Install the GPT boot code into the boot partition.

# gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ada0
# gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ada1

Set the partition type of the old boot pool partition to freebsd-vinum, so it does not get detected by the boot code as a ZFS partition.

# gpart modify -t freebsd-vinum -i 2 ada0
# gpart modify -t freebsd-vinum -i 2 ada1

Remove geli_ada0p4_*, geom_eli_passphrase_prompt, and (optional) zpool_cache_* from /boot/loader.conf.

Set canmount=noauto for all BEs, including the default.

# zfs set canmount=noauto zroot/ROOT/default
# zfs set canmount=noauto zroot/ROOT/some_other_be

Reboot to confirm everything is working. If you are satisfied, you can destroy the old boot pool.

# zpool destroy bootpool

Delete the old boot pool partitions.

# gpart delete -i2 ada0
# gpart delete -i2 ada1

Delete the old swap partitions.

# swapoff -a
# gpart delete -i3 ada0
# gpart delete -i3 ada1

Use the reclaimed space for larger swap partitions.

# gpart add -t freebsd-swap -l swap0 ada0
# gpart add -t freebsd-swap -l swap1 ada1

Update /etc/fstab to use the new swap partition indices.

# Device          Mountpoint  FStype  Options  Dump  Pass#
/dev/ada0p2.eli   none        swap    sw       0     0
/dev/ada1p2.eli   none        swap    sw       0     0

Turn swap back on.

# swapon -a

Posted 2016-09-17 15:48 | Comments

Problems using Microsoft Office 365 at Dalhousie University

  1. Users of some browsers, including Safari, do not see attachments in Office 365's web interface if they are part of valid messages sent from my local mail client. A workaround is to use the drop-down menu on the right side of the screen to display offending messages in a new window. After doing this, missing attachments suddenly appear.
  2. Before the switch to Office 365, people in the Math & Stats department had @mathstat.dal.ca email addresses. Since the department's mail server was shut down, messages sent to these addresses are not reliably forwarded to @dal.ca addresses. I recently missed an important corresponding-author message sent to my @mathstat.dal.ca address. Because of similar problems, my supervisor has decided to supply a private email address on all new paper submissions.
  3. Office 365 determines that a crippled, light version of the web interface is suitable for the browser I use, Conkeror/Firefox on FreeBSD. This browser is standards compliant and receives a 100/100 on the Acid3 Test for Web Standards. Simply faking the user-agent string gives the full web interface, which works as well, or better (see 1.), as it does with more popular browsers. This is not an issue with any other webmail interface I have tried.
  4. If mail is forwarded without keeping a copy, the light web interface will show different messages than the full web interface.
  5. I attach a signature to Email messages I send, however on most browser, operating system combinations I have tried, Office 365 reports "This message has a digital signature, but it wasn't verified because the S/MIME control isn't currently supported for your browser or platform." Using Firefox on Windows, users are prompted to install owasmime.msi, but the install fails because ".NET Framework 4.5 or higher is required to run this ActiveX control."
  6. After logging in, I occasionally see the error message "There was a problem accessing the site. Try to browse to the site again. If the problem persists, contact the administrator of this site and provide the reference number to identify the problem. Reference number: ...".

Posted 2016-08-22 11:49 | Comments

Recent posts

Monthly Archives

Yearly Archives


RSS