Seed snaps using MAAS

Or otherwise entitled: Where my week went....

MAAS quickly deploys machines by providing configuration to the fast path installer curtin. Snappy supports setting up a seed directory containing local snaps and snap assertions which snapd installs on first boot. This post below is a catalog of how I seeded some local snaps which got subsequently installed as the MAAS-deployed node booted.

TLDR: Use cloud-init :)

Snappy seed directory

As of snap version 2.30, seed directories can be provided to a system image by creating a seed directory in an image the looks like the following:

/var/lib/snapd/seed/seed.yaml  # yaml manifest of snaps to install
/var/lib/snapd/seed/snaps      # local *.snap files
/var/lib/snapd/seed/assertions # assert files for each snap
/var/lib/snapd/seed/generic.account-key  # an account assert file
/var/lib/snapd/seed/generic.account  # an account type assert file
/var/lib/snapd/seed/generic-classic.model  # a model type assert file

Shell: create the seed directory

mkdir -p /var/lib/snapd/seed/snaps /var/lib/snapd/seed/assertions;
cd /var/lib/snapd/seed/snaps;
# Download the published snaps and their related assert files
# core snap is required for seeding
snap download core; snap download <yourpublishedsnap> # optionally provide --edge or --beta channel;

# Move assert files downloaded into assertions subdir
mv *assert ../assertions;
cd ../assertions;

# Create model and account assertions
snap known --remote model series=16 model=generic-classic brand-id=generic > generic-classic.model;

ACCOUNT_KEY=`awk '/sign-key-sha3-384/{print $2}' generic-classic.model`;

snap known --remote account-key public-key-sha3-384=${ACCOUNT_KEY} > generic.account-key;

snap known --remote account account-id=generic > generic.account;

# Create the seed.yaml: the manifest of snaps to install
echo "snaps:" > /var/lib/snapd/seed/seed.yaml;
for assertfile in *assert; do
    awk '/snap-name/{SNAP=$2} /snap-revision:/ {VERSION=$2} END {printf " - name: %s\n   channel: stable\n   file: %s_%s.snap\n", SNAP, SNAP , VERSION} $assertfile >> /var/lib/snapd/seed/seed.yaml;
done;
# Note classic snaps require a "classic: true" attribute the seed file

Python: create the seed directory

Here's a script I wrote in python to setup a seed dir to install core and vlc

'''Simple script to create a snap seed directory.'''

import glob
import os
import re

from cloudinit.util import chdir, ensure_dir, subp, write_file


SNAP_SEED_DIR = '/var/lib/snapd/seed'


def download_snaps(snap_names, targetdir):
    """Download the provided snaps to targetdir.

    @param snap_names: List of snap names or urls to download.
    @param targetdir: The path where snaps should be downloaded.

    @return: Returns a tuple containing a list of snap filenames downloaded
        and a list of snap assert filenames downloaded.
    """
    commands = []
    env = os.environ.copy()
    env['SNAPPY_STORE_NO_CDN'] = '1'
    for snap in snap_names:
        try:
            commands.append(['snap', 'download', snap])
        except ProcessExecutionError as e:
            print("WARNING: Could not download snap '{snap_name}': {error}".format(
                snap_name=snap, error=str(e)))
    with chdir(targetdir):
        for cmd in commands:
            subp(cmd, env=env)
        return (glob.glob('*.snap'),  glob.glob('*.assert'))
    return ([], [])


def create_local_snap_seed(snap_names, targetdir):
    """Creates a snap mirror in targetdir.

    Downloads the snap_names from the snapstore and creates a local
    seed directory from which snaps will be auto-installed.
    """
    snap_names = sorted(snap_names)
    snapsdir = os.path.join(targetdir, 'snaps')
    ensure_dir(snapsdir)
    assertdir = os.path.join(targetdir, 'assertions')
    ensure_dir(assertdir)
    model_out, _ = subp([
        'snap', 'known', '--remote', 'model', 'series=16',
        'model=generic-classic', 'brand-id=generic'], capture=True)
    write_file(os.path.join(assertdir, 'generic-classic.model'), model_out)
    match = re.match(
        r'.*sign-key-sha3-384: (?P<account_key>[^\n]+).*',
        model_out, flags=re.DOTALL)
    account_key = match.group('account_key')
    account_key_out, _ = subp([
        'snap', 'known', '--remote', 'account-key',
        'public-key-sha3-384={key}'.format(account_key)])
    write_file(os.path.join(assertdir, 'generic.account-key'), account_key_out)
    account_out, _ = subp([
        'snap', 'known', '--remote', 'account', 'account-id=generic'])
    write_file(os.path.join(assertdir, 'generic.account'), account_out)
    snapfiles, assertfiles = download_snaps(snap_names, snapsdir)
    # move assert files to assertions dir
    for assertfile in assertfiles:
        os.rename(os.path.join(snapsdir, assertfile),
                  os.path.join(assertdir, assertfile))

    seedfile = os.path.join(targetdir, 'seed.yaml')
    content = ["snaps:"]
    for index, snapfile in enumerate(sorted(snapfiles)):
        # For classic snaps, append 'classic: true' to the specific section
        content.append(
            " - name: {snap_name}\n"
            "   channel: stable\n"
            "   file: {snap_file}".format(
                snap_name=snap_names[index], snap_file=snapfile))
    write_file(seedfile, '\n'.join(content))

if __name__ == '__main__':
    create_local_snap_seed(['core', 'vlc'], SNAP_SEED_DIR)

Seed snaps through curtin config

Curtin provides facility to write content into the target disk by specifying write_files configuration and late_commands. Below is an abridged configuration section which writes the python script above in curtin's write_files and runs it in late_commands.

write_files:
  setup_snap_seed:
    path: /tmp/create_snap_preseed.py
    content: |
      '''Simple script to create a snap seed directory.'''

      import glob 
      # rest of script
      ...

late_commands:
    run_snap_seed_setup: ["curtin", "in-target", "--", python3, /tmp/create_snap_preseed.py]

MAAS talking to curtin

MAAS allows you to provide custom curtin configuration to a matching node based on /etc/maas/preseeds/curtin_userdata_* files. Let's create a curtin preseed file for all amd64 xenial systems.

cd /etc/maas/preseeds/;
cp curtin_userdata curtin_userdata_ubuntu_amd64_generic_xenial
# 1. Add run_snap_seed_setup line to the existing late_commands section
# 2. Add write_files section included above

Now, anytime MAAS deploys ubuntu amd64 generic xenial systems, they'll all get vlc snaps installed. You can now watch all the movies you really to on any system your employer owns :).

Retrospective: better alternatives for snap installs

Since MAAS drives curtin for installs, seeding snaps in an image before the system boots is compelling in a couple of situations:

  • MAAS can deliver a private snap to the image which is not hosted in a public snap store (firmware updates, security fixes etc)
  • The deployed machines have limited network connectivity to the snap store behind a firewall
  • You have some private snap credentials you don't want exposed on the deployed machines, so you keep them only on your MAAS server and use them only from curtin.

If those cases don't necessarily apply, it is probably best to use cloud-init for the following reasons:

  • Since cloud-init runs after curtin MAAS-deployed nodes, you can leverage cloud-init's features 'for free'
  • Pre-seeded snaps can't actually install until snapd starts after networking is up. This is during cloud-init's init-network stage anyway.
    • By preseeding snaps, we have only shifted the actual snap download time into curtin's setup stage instead of cloud-init's.
  • Use a generic approach: Cloud-init's snappy module config supports a simple option to declare desired snappy packages in #cloud-config files which work on almost any cloud you need, including your MAAS managed hardware.
  • In contrast to seeding snaps, cloud-init takes care of installing snap-core dependency automatically.

Here is a sample #cloud-config file you could provide when deploying your instance in most major clouds.

#cloud-config
snappy:
    system_snappy: true  # allow snap installs on non-snappy systems
    packages: [canonical-livepatch]

Thanks for staying in touch, I'll get back to our regularly scheduled cloud-init posts in our next post 'cloud-init subcommands'.

blogroll

social