Cloud Infra, FreeBSD, OpenStack

Using release(7) to create FreeBSD images for OpenStack – Yes, you can do it!


During a recent BSDNow podcast, Allan and Kris mentioned that it would be nice to have a tutorial on how to create a FreeBSD image for OpenStack using the official release(7) tools.

With that, it came to me that: #1 I do have access to an OpenStack environment and #2 I am interested in having FreeBSD as a guest image in my environment. Looks like I was up for the challenge.

Previously, I’ve had success running FreeBSD 11.0-RELEASE on OpenStack with success but more could/should be done. For instance, as suggested by Allan,  wouldn’t be nice to deploy the latest code from FreeBSD ? Running -STABLE or even -CURRENT ? Yes, it would. Also, wouldn’t it be nice to customize these images for a specific need? I’d say ‘Yes’ for that as well.

With that, I thought I would give it a try, so without further ado, here’s how you
can bake a FreeBSD image using the official releng toolset.

After some research I found that the current openstack.conf file, located at /usr/src/release/tools/ could use some extra tweaks to get where I wanted. I’ve created and attached that to a bugzilla on the same topic. You can read about that here.

This diff brings the following changes:

  •  resize FreeBSD to the size of the OpenStack flavor (growfs).
  • speeds up the boot process by disabling sendmail.
  • allows an user to ssh as root with a public key.
  • make ssh respond faster by disabling DNS lookups (can be reverted).
  • enable DHCP on the vtnet interface.

Here’s our goal:

Create a FreeBSD image for OpenStack (KVM based) using the release(7) tools.

and as a bonus:

  • Be able to upload the image to OpenStack’s Glance and to manipulate a HEAT stack _from FreeBSD_


  • Make sure you have a system that can do a build(7) – disk space is important here, more CPU power helps too.
  • Patch the openstack.conf file with the new changes – you can get the file from here.
# $FreeBSD: releng/11.0/release/tools/openstack.conf 277458 2015-01-20
# 23:56:04Z gjb $

# Set to a list of packages to install.
export VM_EXTRA_PACKAGES="net/cloud-init
 devel/py-pbr devel/py-iso8601 \
 net/py-eventlet net/py-netaddr comms/py-serial devel/py-six \
 devel/py-babel net/py-oauth net/py-netifaces"

# Set to a list of third-party software to enable in rc.conf(5).
export VM_RC_LIST="cloudinit"


vm_extra_pre_umount() {
 #env ASSUME_ALWAYS_YES=yes pkg -c ${DESTDIR} delete -f -y pkg
 #rm ${DESTDIR}/var/db/pkg/repo-*.sqlite

 echo 'growfs_enable="YES"' >> ${DESTDIR}/etc/rc.conf

 # Enable sshd by default
 echo 'sshd_enable="YES"' >> ${DESTDIR}/etc/rc.conf

 # Disable DNS lookups by default to make SSH connect quickly
 echo 'UseDNS no' >> ${DESTDIR}/etc/ssh/sshd_config

 echo 'PermitRootLogin without-password' >>

 # Enable DHCP for the OpenStack instance
 echo 'ifconfig_DEFAULT="SYNCDHCP"' >> ${DESTDIR}/etc/rc.conf

 # Disable sendmail
 echo 'sendmail_enable="NO"' >> ${DESTDIR}/etc/rc.conf
 echo 'sendmail_submit_enable="NO"' >> ${DESTDIR}/etc/rc.conf
 echo 'sendmail_outbound_enable="NO"' >> ${DESTDIR}/etc/rc.conf
 echo 'sendmail_msp_queue_enable="NO"' >> ${DESTDIR}/etc/rc.conf

 # Openstack wants sudo(8) usable by default without a password.
 rm -f ${DESTDIR}/etc/resolv.conf

 # The console is not interactive, so we might as well boot quickly.
 echo 'autoboot_delay="-1"' >> ${DESTDIR}/boot/loader.conf
 echo 'beastie_disable="YES"' >> ${DESTDIR}/boot/loader.conf

 # Reboot quickly, Don't wait at the panic screen
 echo 'debug.trace_on_panic=1' >> ${DESTDIR}/etc/sysctl.conf
 echo 'debug.debugger_on_panic=0' >> ${DESTDIR}/etc/sysctl.conf
 echo 'kern.panic_reboot_wait_time=0' >> ${DESTDIR}/etc/sysctl.conf

 touch ${DESTDIR}/firstboot
 return 0


  • Fetch the FreeBSD source code and extract it under /usr/src
  • Once the code is in place, follow the regular process of build(7) and perform a `make buildworld buildkernel`
  • Change into the release directory (/usr/src/release) and perform a make cloudware

That’s it! This will generate a qcow2 image with 1.4G in size and a raw image of 2G.
The entire process uses the release(7) toolchain to generate the image and should work with newer versions of FreeBSD.

Tested with 11.0-RELEASE-p1 and also tested on OpenStack Mitaka.

Bonus Tips: Installing OpenStack cli tools.

Instal pip:

pkg install py27-pip

Install the OpenStack Python Client:

pip install python-openstackclient

Install the legacy OpenStack Clients (might be needed for some environments).

pip install python-heatclient
pip install python-nova client

Next, create an openrc file and source it. Here’s an example:

unset ${!OS_*}
export OS_AUTH_URL=
export OS_USERNAME=admin
export OS_PASSWORD=NotMyPassw0rd
export OS_PROJECT_NAME=admin
export OS_USER_DOMAIN_NAME=default

Then, source it:

source ~/

Right now, you have all you need to operate OpenStack. To test, let’s try the following:

  1. Upload our newly created FreeBSD image to OpenStack’s Glance.
  2. Spin up a stack that uses our FreeBSD image by invoking a Heat template
  3. Test the connectivity to the new stack
  4. Clean the stack

Let’s go!

With the image now fully created, upload it to glance.

[root@freebsd /usr/src/release]# cd /usr/obj/usr/src/release
[root@freebsd /usr/obj/usr/src/release]# glance image-create \ 
 --progress              \ 
 --disk-format qcow2     \
 --name FreeBSD-OpenStack\
 --file openstack.qcow2  \
 --container bare

Create a heat stack called “freebsd-image-test” using the freebsd-openstack.yaml file.

[root@freebsd ~]# openstack stack create -t ./freebsd-openstack.yaml freebsd-image-test

| Field               | Value                                      |    
| id                  | e217ba55-6892-4012-aebb-e8b2f7032df7       |    
| stack_name          | freebsd-image-test                         |    
| description         | This template generates a FreeBSD instance |    
|                     |                                            |    
| creation_time       | 2016-12-07T01:26:46                        |    
| updated_time        | None                                       |    
| stack_status        | CREATE_IN_PROGRESS                         |    
| stack_status_reason | Stack CREATE started                       |    

Now that the stack is created, let’s get the floating IP of the instance and ssh into it. For this example, I’m using a key named “cse-pubkey”. You can/should change that on the template to reflect your environment.

[root@freebsd ~]# openstack stack output show --all freebsd-image-test 
| Field               | Value                                   |             
| instance_private_ip | {                                       |             
|                     |   "output_value": "",          |             
|                     |   "output_key": "instance_private_ip",  |             
|                     |   "description": "No description given" |             
|                     | }                                       |             
| instance_public_ip  | {                                       |             
|                     |   "output_value": "",     |             
|                     |   "output_key": "instance_public_ip",   |             
|                     |   "description": "No description given" |             
|                     | }                                       |             

Test the connection to the instance.

[root@lppa240 ~]# ssh -i cse_id_rsa
The authenticity of host ' (' can't be established.
ECDSA key fingerprint is SHA256:LX06d48gTGV+SCrl3DJlWw5i4j7+IjaHDW6J2s/JwzQ.
No matching host key fingerprint found in DNS.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '' (ECDSA) to the list of known hosts.
FreeBSD 11.0-RELEASE-p1 (GENERIC) #1: Sat Dec  3 10:54:58 EST 2016

Welcome to FreeBSD!

Release Notes, Errata:
Security Advisories:
FreeBSD Handbook:
Questions List:
FreeBSD Forums:

Documents installed with the system are in the /usr/local/share/doc/freebsd/
directory, or can be installed later with:  pkg install en-freebsd-doc
For other languages, replace "en" with a language code like de or fr.

Show the version of FreeBSD installed:  freebsd-version ; uname -a
Please include that output and any error messages when posting questions.
Introduction to manual pages:  man man
FreeBSD directory layout:      man hier

Edit /etc/motd to change this login announcement.
root@freebsd-image-test-instance-qk6lpulwbxp4:~ # uptime
 1:30AM  up 3 mins, 1 users, load averages: 0.27, 0.30, 0.14

All set!

Cleaning the stack

[root@freebsd ~]# heat stack-delete freebsd-image-test



Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s