.. title: Build Bootable FreeBSD Image for Raspberry Pi 2 with Vagrant, Crochet, and Salt
.. slug: build-bootable-freebsd-image-for-raspberry-pi-2-with-vagrant-crochet-and-salt
.. date: 2016-01-30 05:47:29 UTC
.. updated: 2016-04-19 05:03:28 UTC
.. tags: crochet, git, freebsd, salt, vagrant, virtualbox
.. category:
.. link:
.. description:
.. type: text

An issue in FreeBSD 11-CURRENT - where
`make buildworld fails when run on RPi2 <https://lists.freebsd.org/pipermail/freebsd-arm/2016-January/013049.html>`_
- prompted an investigation on automating the creation of bootable FreeBSD images
from 11-CURRENT on a Mac OS X machine.

In an ideal situation, a build process can be kicked off at any time to produce
an image that can be written to a Micro SD card for the RPi2 to boot from.

.. TEASER_END: Read more

Things you need to get this project up and running:

* git
* VirtualBox
* Vagrant
* FreeBSD 11.0-CURRENT Vagrant box
* Crochet
* Salt
* OS X host

UPDATE (2016-04-19): I have created a
`companion git repo <https://github.com/hamzasheikh/freebsd-crochet-raspberrypi-vagrant-ansible>`_
that provides a quick start alternative to the manual steps described here.

First Steps
-----------

::

    user@host:~ $ mkdir -p ~/devbox/projects/freebsd-rpi
    user@host:~ $ cd ~/devbox/projects/freebsd-rpi
    user@host:~/devbox/projects/freebsd-rpi $ git init

Salt
----

.. Create a Python (3.5) virtualenv and install Salt.
..
.. ::
..
..     user@host:~/devbox/projects/freebsd-rpi $ python3.5 -m venv pyvenv
..     user@host:~/devbox/projects/freebsd-rpi $ echo "pyvenv/" >> .gitignore
..     user@host:~/devbox/projects/freebsd-rpi $ git add .gitignore
..     user@host:~/devbox/projects/freebsd-rpi $ git commit -m "Add pyvenv/ to .gitignore"
..     user@host:~/devbox/projects/freebsd-rpi $ source pyvenv/bin/activate
..     (pyvenv) user@host:~/devbox/projects/freebsd-rpi $ pip install --upgrade pip
..
.. Salt has an issue with running under Python 3.5, namely when running
.. ``salt-master -d`` like this
..
.. ::
..
..     (pyvenv) user@host:~/devbox/projects/freebsd-rpi $ salt-master -d
..     Traceback (most recent call last):
..       File "//Users/user/devbox/projects/freebsd-rpi/pyvenv/bin/salt-master", line 9, in <module>
..         load_entry_point('salt==2015.8.3', 'console_scripts', 'salt-master')()
..       File "//Users/user/devbox/projects/freebsd-rpi/pyvenv/lib/python3.5/site-packages/salt/scripts.py", line 45, in salt_master
..         import salt.cli.daemons
..       File "//Users/user/devbox/projects/freebsd-rpi/pyvenv/lib/python3.5/site-packages/salt/cli/daemons.py", line 47, in <module>
..         from salt.utils import parsers, ip_bracket
..       File "//Users/user/devbox/projects/freebsd-rpi/pyvenv/lib/python3.5/site-packages/salt/utils/parsers.py", line 26, in <module>
..         import salt.config as config
..       File "//Users/user/devbox/projects/freebsd-rpi/pyvenv/lib/python3.5/site-packages/salt/config.py", line 40, in <module>
..         import salt.utils.sdb
..       File "//Users/user/devbox/projects/freebsd-rpi/pyvenv/lib/python3.5/site-packages/salt/utils/sdb.py", line 9, in <module>
..         import salt.loader
..       File "//Users/user/devbox/projects/freebsd-rpi/pyvenv/lib/python3.5/site-packages/salt/loader.py", line 35, in <module>
..         import salt.modules.cmdmod
..       File "//Users/user/devbox/projects/freebsd-rpi/pyvenv/lib/python3.5/site-packages/salt/modules/cmdmod.py", line 21, in <module>
..         from salt.utils import vt
..       File "//Users/user/devbox/projects/freebsd-rpi/pyvenv/lib/python3.5/site-packages/salt/utils/vt.py", line 34, in <module>
..         if subprocess.mswindows:
..     AttributeError: module 'subprocess' has no attribute 'mswindows'
..
.. `Python 3.5 has renamed subprocess.mswindows to subprocess._mswindows <https://bitbucket.org/catherinedevlin/cmd2/issues/18/python-35-renames-subprocessmswindows>`_.
.. Modify Salt code like so
..
.. ::
..
..     (pyvenv) user@host:~/devbox/projects/freebsd-rpi $ sed -i .bak -e "s/subprocess.mswindows/subprocess._mswindows/g" pyvenv/lib/python3.5/site-packages/salt/utils/vt.py
..
.. Install from my fork of Salt
..
.. ::
..
..     (pyvenv) user@host:~/devbox/projects/freebsd-rpi $ pip install git+https://github.com/usersheikh/salt.git@fixes-for-python3.5
..     (pyvenv) user@host:~/devbox/projects/freebsd-rpi $ pip install libnacl

Create a Python (2.7) virtualenv and install Salt.

::

    user@host:~/devbox/projects/freebsd-rpi $ virtualenv-2.7 pyvenv
    user@host:~/devbox/projects/freebsd-rpi $ echo "pyvenv/" >> .gitignore
    user@host:~/devbox/projects/freebsd-rpi $ git add .gitignore
    user@host:~/devbox/projects/freebsd-rpi $ git commit -m "Add pyvenv/ to .gitignore"
    user@host:~/devbox/projects/freebsd-rpi $ source pyvenv/bin/activate
    (pyvenv) user@host:~/devbox/projects/freebsd-rpi $ pip install --upgrade pip

Create the master configuration file. There's a security no-no of accepting
all minions in this config file but that's the easiest thing to do. Your
outlook may be different.

::

    (pyvenv) user@host:~/devbox/projects/freebsd-rpi $ pip install salt
    (pyvenv) user@host:~/devbox/projects/freebsd-rpi $ mkdir salt
    (pyvenv) user@host:~/devbox/projects/freebsd-rpi $ vim salt/master
    auto_accept: True
    interface: 0.0.0.0
    root_dir: /Users/user/devbox/projects/freebsd-rpi/salt
    pki_dir: /Users/user/devbox/projects/freebsd-rpi/salt/pki
    pidfile: /Users/user/devbox/projects/freebsd-rpi/salt/var/run/salt-master.pid
    user: user
    file_roots:
        base:
            - /Users/user/devbox/projects/freebsd-rpi/salt/states
    (pyvenv) user@host:~/devbox/projects/freebsd-rpi $ git add salt/
    (pyvenv) user@host:~/devbox/projects/freebsd-rpi $ git commit -m "Add master config file"
    (pyvenv) user@host:~/devbox/projects/freebsd-rpi $ echo "salt/pki/" >> .gitignore
    (pyvenv) user@host:~/devbox/projects/freebsd-rpi $ echo "salt/var/" >> .gitignore
    (pyvenv) user@host:~/devbox/projects/freebsd-rpi $ git add .gitignore
    (pyvenv) user@host:~/devbox/projects/freebsd-rpi $ git commit -m "Add salt/{pki,var} to .gitignore"

.. Create the minions config
..
.. ::
..
..     (pyvenv) user@host:~/devbox/projects/freebsd-rpi $ vim salt/minion
..     master: 10.0.2.2
..     id: 'minion1'
..     file_client: remote

.. Create keys for connection between master and minion and pre-accept them in
.. master
..
.. ::
..
..     (pyvenv) user@host:~/devbox/projects/freebsd-rpi $ mkdir keys
..     (pyvenv) user@host:~/devbox/projects/freebsd-rpi $ echo "keys/" >> .gitignore
..     (pyvenv) user@host:~/devbox/projects/freebsd-rpi $ git add .gitignore
..     (pyvenv) user@host:~/devbox/projects/freebsd-rpi $ git commit -m "Add keys/ to .gitignore"
..     (pyvenv) user@host:~/devbox/projects/freebsd-rpi $ cd keys
..     (pyvenv) user@host:~/devbox/projects/freebsd-rpi/keys $ salt-key --gen-keys=minion1 -c ../salt --log-file "./salt-key.log" --log-file-level "debug"
..     (pyvenv) user@host:~/devbox/projects/freebsd-rpi/keys $ cd ..
..     (pyvenv) user@host:~/devbox/projects/freebsd-rpi $ mkdir -p salt/pki/master/minions
..     (pyvenv) user@host:~/devbox/projects/freebsd-rpi $ cp keys/minion1.pub salt/pki/master/minions/minion1

Delete accepted key for minion1 since it will be re-added as part of Vagrant
provision

::

    (pyvenv) user@host:~/devbox/projects/freebsd-rpi $ salt-key -c salt -D minion1 -y

Now it is possible to start Salt master. Before you can provision using Vagrant
the Salt master **must** be running.

::

    (pyvenv) user@host:~/devbox/projects/freebsd-rpi $ echo "*.log" >> .gitignore
    (pyvenv) user@host:~/devbox/projects/freebsd-rpi $ git add .gitignore
    (pyvenv) user@host:~/devbox/projects/freebsd-rpi $ git commit -m "Add *.log to .gitignore"
    (pyvenv) user@host:~/devbox/projects/freebsd-rpi $ salt-master -c salt --log-file "./salt.log" --log-file-level "debug"

VirtualBox
----------

Install it.

Vagrant
-------

`Download <https://www.vagrantup.com/downloads.html>`_ and install Vagrant.

Create a brand new Vagrant file.

::

    user@host:~/devbox/projects/freebsd-rpi $ vagrant init
    user@host:~/devbox/projects/freebsd-rpi $ git add Vagrantfile
    user@host:~/devbox/projects/freebsd-rpi $ git commit -m "Add Vagrantfile"

Download FreeBSD 11.0-CURRENT "box" from HashiCorp ATLAS.

::

    user@host:~/devbox/projects/freebsd-rpi $ vagrant box add --provider virtualbox --clean freebsd/FreeBSD-11.0-CURRENT

Use the downloaded box in Vagrantfile.

::

    user@host:~/devbox/projects/freebsd-rpi $ sed -i .bak -e 's/  config.vm.box = "base"/  config.vm.box = "freebsd\/FreeBSD-11.0-CURRENT"/g' Vagrantfile
    user@host:~/devbox/projects/freebsd-rpi $ echo "*.bak" >> .gitignore
    user@host:~/devbox/projects/freebsd-rpi $ echo ".vagrant/" >> .gitignore
    user@host:~/devbox/projects/freebsd-rpi $ git add .gitignore
    user@host:~/devbox/projects/freebsd-rpi $ git commit -m "Create .gitignore"
    user@host:~/devbox/projects/freebsd-rpi $ git add Vagrantfile
    user@host:~/devbox/projects/freebsd-rpi $ git commit -m "Use FreeBSD-11.0-CURRENT box"

Configure required settings by adding these lines before ``end`` in Vagrantfile.
Modify this information as needed, especially the IP address of Salt master.

::

    config.vm.base_mac = "000AAAAAAAAA"
    config.vm.guest = :freebsd
    config.vm.hostname = "freebsd"
    config.vm.synced_folder ".", "/vagrant_data", disabled: true
    config.ssh.shell = "sh"
    config.vm.provision "shell", inline: <<-SHELL
      pkg update
      pkg install -y py27-salt
      cp /usr/local/etc/salt/minion.sample /usr/local/etc/salt/minion
      sed -i .1.bak -e 's/^#master: salt$/master: 10.201.107.246/g' /usr/local/etc/salt/minion
      sed -i .2.bak -e 's/^#file_client: remote$/file_client: remote/g' /usr/local/etc/salt/minion
      sed -i .3.bak -e 's/^#id:$/id: minion1/g' /usr/local/etc/salt/minion
      sysrc salt_minion_enable="YES"
      service salt_minion start
    SHELL

Commit keep these changes in git.

::

    user@host:~/devbox/projects/freebsd-rpi $ git add Vagrantfile
    user@host:~/devbox/projects/freebsd-rpi $ git commit -m "Configure VM"

Provision the VM. Since Salt is installed in a virtualenv we need to activate
it first and then start the daemon if it isn't already.

::

    user@host:~/devbox/projects/freebsd-rpi $ source pyvenv/bin/activate
    (pyvenv) user@host:~/devbox/projects/freebsd-rpi $ salt-master -d -c salt --log-file "./salt.log" --log-file-level "debug"

Provision the VM

::

    (pyvenv) user@host:~/devbox/projects/freebsd-rpi $ vagrant up

Crochet
-------

Create salt state files to install git.

::

    (pyvenv) user@host:~/devbox/projects/freebsd-rpi $ mkdir -p salt/states/{git,vim-lite}
    (pyvenv) user@host:~/devbox/projects/freebsd-rpi $ vim salt/states/top.sls
    base:
        'minion*':
            - git
            - vim-lite
    (pyvenv) user@host:~/devbox/projects/freebsd-rpi $ vim salt/states/git/init.sls
    git:
        pkg:
            - installed
    (pyvenv) user@host:~/devbox/projects/freebsd-rpi $ vim salt/states/vim-lite/init.sls
    vim-lite:
        pkg:
            - installed
    (pyvenv) user@host:~/devbox/projects/freebsd-rpi $ git add salt/states
    (pyvenv) user@host:~/devbox/projects/freebsd-rpi $ git commit -m "Add salt states"

Restart salt master

::

    (pyvenv) user@host:~/devbox/projects/freebsd-rpi $ pkill -F /Users/user/devbox/projects/freebsd-rpi/salt/var/run/salt-master.pid
    (pyvenv) user@host:~/devbox/projects/freebsd-rpi $ salt-master -d -c salt --log-file "./salt.log" --log-file-level "debug"

Apply the configuration.

::

    (pyvenv) user@host:~/devbox/projects/freebsd-rpi $ mkdir -p salt/var/log/salt
    (pyvenv) user@host:~/devbox/projects/freebsd-rpi $ salt -c salt '*' state.highstate

On the FreeBSD VM build a Raspberry Pi 2 bootable image with Crochet.

::

    (pyvenv) user@host:~/devbox/projects/freebsd-rpi $ vagrant ssh
    vagrant@freebsd:~ % sudo pkg install vim-lite
    vagrant@freebsd:~ % git clone https://github.com/freebsd/crochet.git
    vagrant@freebsd:~ % cd crochet

Copy the sample config file ...

::

    vagrant@freebsd:~/crochet % cp config.sh.sample config.rpi2.sh
    vagrant@freebsd:~/crochet % vim config.rpi2.sh

... and apply this diff

::

    vagrant@freebsd:~/crochet % diff config.sh.sample config.rpi2.sh
    34c34
    < #board_setup RaspberryPi2
    ---
    > board_setup RaspberryPi2
    55c55
    < #option ImageSize 3900mb # for 4 Gigabyte card
    ---
    > option ImageSize 3900mb # for 4 Gigabyte card
    97c97
    < #option Growfs
    ---
    > option Growfs
    115c115
    < #option User <username>
    ---
    > option User raspberry
    126a127
    > option SwapFile 768mb deferred file=/swapfile0
    160c161
    < #option Ntpd
    ---
    > option Ntpd
    164c165
    < #KERNCONF=MYCONF
    ---
    > KERNCONF=RPI2
    171c172
    < #FREEBSD_SRC=/usr/src
    ---
    > FREEBSD_SRC=/usr/src
    194c195
    < #IMG=${WORKDIR}/FreeBSD-${KERNCONF}.img
    ---
    > IMG=${WORKDIR}/FreeBSD-${KERNCONF}.img
    202c203
    < #FREEBSD_INSTALL_WORLD=y
    ---
    > FREEBSD_INSTALL_WORLD=y

Download sources, install u-boot-rpi2, and build an image

::

    vagrant@freebsd:~/crochet % sudo svnlite checkout svn://svn.freebsd.org/base/head /usr/src
    vagrant@freebsd:~/crochet % sudo pkg install -y sysutils/u-boot-rpi2
    vagrant@freebsd:~/crochet % sudo /bin/sh crochet.sh -c config.rpi2.sh

Take the resulting image, ``dd`` it to a micro SD card, insert into RPi2 and
boot it.


TODO
----

Make this an automated process that requires minimal manual intervention.