Cloud Infra, FreeBSD, OpenStack

Using release(7) to create FreeBSD images for OpenStack – Yes, you can do it!


During a recent BSDNow podcast, Allan and Kris mentioned that it would be nice to have a tutorial on how to create a FreeBSD image for OpenStack using the official release(7) tools.

With that, it came to me that: #1 I do have access to an OpenStack environment and #2 I am interested in having FreeBSD as a guest image in my environment. Looks like I was up for the challenge.

Previously, I’ve had success running FreeBSD 11.0-RELEASE on OpenStack with success but more could/should be done. For instance, as suggested by Allan,  wouldn’t be nice to deploy the latest code from FreeBSD ? Running -STABLE or even -CURRENT ? Yes, it would. Also, wouldn’t it be nice to customize these images for a specific need? I’d say ‘Yes’ for that as well.

With that, I thought I would give it a try, so without further ado, here’s how you
can bake a FreeBSD image using the official releng toolset.

After some research I found that the current openstack.conf file, located at /usr/src/release/tools/ could use some extra tweaks to get where I wanted. I’ve created and attached that to a bugzilla on the same topic. You can read about that here.

This diff brings the following changes:

  •  resize FreeBSD to the size of the OpenStack flavor (growfs).
  • speeds up the boot process by disabling sendmail.
  • allows an user to ssh as root with a public key.
  • make ssh respond faster by disabling DNS lookups (can be reverted).
  • enable DHCP on the vtnet interface.

Here’s our goal:

Create a FreeBSD image for OpenStack (KVM based) using the release(7) tools.

and as a bonus:

  • Be able to upload the image to OpenStack’s Glance and to manipulate a HEAT stack _from FreeBSD_


  • Make sure you have a system that can do a build(7) – disk space is important here, more CPU power helps too.
  • Patch the openstack.conf file with the new changes – you can get the file from here.
# $FreeBSD: releng/11.0/release/tools/openstack.conf 277458 2015-01-20
# 23:56:04Z gjb $

# Set to a list of packages to install.
export VM_EXTRA_PACKAGES="net/cloud-init
 devel/py-pbr devel/py-iso8601 \
 net/py-eventlet net/py-netaddr comms/py-serial devel/py-six \
 devel/py-babel net/py-oauth net/py-netifaces"

# Set to a list of third-party software to enable in rc.conf(5).
export VM_RC_LIST="cloudinit"


vm_extra_pre_umount() {
 #env ASSUME_ALWAYS_YES=yes pkg -c ${DESTDIR} delete -f -y pkg
 #rm ${DESTDIR}/var/db/pkg/repo-*.sqlite

 echo 'growfs_enable="YES"' >> ${DESTDIR}/etc/rc.conf

 # Enable sshd by default
 echo 'sshd_enable="YES"' >> ${DESTDIR}/etc/rc.conf

 # Disable DNS lookups by default to make SSH connect quickly
 echo 'UseDNS no' >> ${DESTDIR}/etc/ssh/sshd_config

 echo 'PermitRootLogin without-password' >>

 # Enable DHCP for the OpenStack instance
 echo 'ifconfig_DEFAULT="SYNCDHCP"' >> ${DESTDIR}/etc/rc.conf

 # Disable sendmail
 echo 'sendmail_enable="NO"' >> ${DESTDIR}/etc/rc.conf
 echo 'sendmail_submit_enable="NO"' >> ${DESTDIR}/etc/rc.conf
 echo 'sendmail_outbound_enable="NO"' >> ${DESTDIR}/etc/rc.conf
 echo 'sendmail_msp_queue_enable="NO"' >> ${DESTDIR}/etc/rc.conf

 # Openstack wants sudo(8) usable by default without a password.
 rm -f ${DESTDIR}/etc/resolv.conf

 # The console is not interactive, so we might as well boot quickly.
 echo 'autoboot_delay="-1"' >> ${DESTDIR}/boot/loader.conf
 echo 'beastie_disable="YES"' >> ${DESTDIR}/boot/loader.conf

 # Reboot quickly, Don't wait at the panic screen
 echo 'debug.trace_on_panic=1' >> ${DESTDIR}/etc/sysctl.conf
 echo 'debug.debugger_on_panic=0' >> ${DESTDIR}/etc/sysctl.conf
 echo 'kern.panic_reboot_wait_time=0' >> ${DESTDIR}/etc/sysctl.conf

 touch ${DESTDIR}/firstboot
 return 0


  • Fetch the FreeBSD source code and extract it under /usr/src
  • Once the code is in place, follow the regular process of build(7) and perform a `make buildworld buildkernel`
  • Change into the release directory (/usr/src/release) and perform a make cloudware

That’s it! This will generate a qcow2 image with 1.4G in size and a raw image of 2G.
The entire process uses the release(7) toolchain to generate the image and should work with newer versions of FreeBSD.

Tested with 11.0-RELEASE-p1 and also tested on OpenStack Mitaka.

Bonus Tips: Installing OpenStack cli tools.

Instal pip:

pkg install py27-pip

Install the OpenStack Python Client:

pip install python-openstackclient

Install the legacy OpenStack Clients (might be needed for some environments).

pip install python-heatclient
pip install python-nova client

Next, create an openrc file and source it. Here’s an example:

unset ${!OS_*}
export OS_AUTH_URL=
export OS_USERNAME=admin
export OS_PASSWORD=NotMyPassw0rd
export OS_PROJECT_NAME=admin
export OS_USER_DOMAIN_NAME=default

Then, source it:

source ~/

Right now, you have all you need to operate OpenStack. To test, let’s try the following:

  1. Upload our newly created FreeBSD image to OpenStack’s Glance.
  2. Spin up a stack that uses our FreeBSD image by invoking a Heat template
  3. Test the connectivity to the new stack
  4. Clean the stack

Let’s go!

With the image now fully created, upload it to glance.

[root@freebsd /usr/src/release]# cd /usr/obj/usr/src/release
[root@freebsd /usr/obj/usr/src/release]# glance image-create \ 
 --progress              \ 
 --disk-format qcow2     \
 --name FreeBSD-OpenStack\
 --file openstack.qcow2  \
 --container bare

Create a heat stack called “freebsd-image-test” using the freebsd-openstack.yaml file.

[root@freebsd ~]# openstack stack create -t ./freebsd-openstack.yaml freebsd-image-test

| Field               | Value                                      |    
| id                  | e217ba55-6892-4012-aebb-e8b2f7032df7       |    
| stack_name          | freebsd-image-test                         |    
| description         | This template generates a FreeBSD instance |    
|                     |                                            |    
| creation_time       | 2016-12-07T01:26:46                        |    
| updated_time        | None                                       |    
| stack_status        | CREATE_IN_PROGRESS                         |    
| stack_status_reason | Stack CREATE started                       |    

Now that the stack is created, let’s get the floating IP of the instance and ssh into it. For this example, I’m using a key named “cse-pubkey”. You can/should change that on the template to reflect your environment.

[root@freebsd ~]# openstack stack output show --all freebsd-image-test 
| Field               | Value                                   |             
| instance_private_ip | {                                       |             
|                     |   "output_value": "",          |             
|                     |   "output_key": "instance_private_ip",  |             
|                     |   "description": "No description given" |             
|                     | }                                       |             
| instance_public_ip  | {                                       |             
|                     |   "output_value": "",     |             
|                     |   "output_key": "instance_public_ip",   |             
|                     |   "description": "No description given" |             
|                     | }                                       |             

Test the connection to the instance.

[root@lppa240 ~]# ssh -i cse_id_rsa
The authenticity of host ' (' can't be established.
ECDSA key fingerprint is SHA256:LX06d48gTGV+SCrl3DJlWw5i4j7+IjaHDW6J2s/JwzQ.
No matching host key fingerprint found in DNS.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '' (ECDSA) to the list of known hosts.
FreeBSD 11.0-RELEASE-p1 (GENERIC) #1: Sat Dec  3 10:54:58 EST 2016

Welcome to FreeBSD!

Release Notes, Errata:
Security Advisories:
FreeBSD Handbook:
Questions List:
FreeBSD Forums:

Documents installed with the system are in the /usr/local/share/doc/freebsd/
directory, or can be installed later with:  pkg install en-freebsd-doc
For other languages, replace "en" with a language code like de or fr.

Show the version of FreeBSD installed:  freebsd-version ; uname -a
Please include that output and any error messages when posting questions.
Introduction to manual pages:  man man
FreeBSD directory layout:      man hier

Edit /etc/motd to change this login announcement.
root@freebsd-image-test-instance-qk6lpulwbxp4:~ # uptime
 1:30AM  up 3 mins, 1 users, load averages: 0.27, 0.30, 0.14

All set!

Cleaning the stack

[root@freebsd ~]# heat stack-delete freebsd-image-test


Cloud Infra, OpenStack

OpenStack Neutron Diagrams

<!– –>

TL;DR: Neutron diagrams depicted below are available on my Github repo.

During my presentation on Troubleshooting Neutron Networks (OpenStack Summit 2016 in Barcelona – details and video) there were many network diagrams used to illustrated the various scenarios with Neutron networks.

Many stackers were asking where to get these diagrams and as promised, here they are:


east-west DVR traffic


north-south traffic



east-west traffic with VXLAN

GitHub – dcasati/OpenStack/tree/master/OS2016BCN-Troubleshooting-Neutron

The diagrams can be open using and are made available under the Creative Commons License (CC by 4.0). You can show/hide the different layers and well as edit each component individually.

Have fun!

Cloud Infra, OpenStack, Uncategorized

OpenStack Summit Barcelona 2016


OpenStack Summit in Barcelona is now officially over. The Summit happened between October 25 and 28th in Barcelona, Spain.

This being my first Summit, I was excited about what to see and who I would meet during the conference 4 days.  The fact that I had the chance to deliver two presentations alongside my team members.

Since this was the  OpenStack Summit, we could not go on without, well having  OpenStack, so we brought one own. And it was great ! With us this year, we brought the Dell EMC VxRack Neutrino, a hyperconverged turn-key OpenStack solution. Neutrino really shined during the Summit as more and more Stackers were curious wether or not we  really had a running rack or just a gimmick blinking lights. For folks who are still wondering, we did have the real deal with running workloads on it.

Presentations –

With all of the activities during the Summit, it was hard to see all of them while on site, but I did manage to attend a few of them while also presenting.

Without further ado, here’s what I’ve presented and also some of the interesting presentations that I saw:

Accelerate the OpenStack RCA with Kibana and ElasticSearch (Details)

During this presentation, me and David Sanchez went on to talk about how difficult it is for teams and OpenStack practioners to fully have a Root Cause Analysis (RCA), in time and in a reproducible fashion. We’ve delve  the different aspects of a successful RCA and also how to move from the firefighting mode into a more proactive one.

Troubleshooting Neutron – Physical and Virtual (Details)

In this presentation we’ve went through two real scenarios of troubleshooting Neutron, showing the different components involved by following the data traffic trought the network.

Hands-On Workshops –

Hands-On Workshop: Learn About Microservices Architectures with Docker Swarm, Etcd, Kuryr, Neutron (Details)

    This hands-on workshop was presented by Fawad Khaliq from PLUMgrid who essentially did a great job explaining how Microservices work through theory and examples. During the course of this workshop, Fawad showed showed how one would increase and decrease a swarm cluster and how Docker networking and OpenStack Neutron interface through Kuryr.

Neutron Ninja 101: A Hands-On Workshop with Neutron Networking (Details)

    Becoming a Ninja seems very challening, but Phil Hopkins did a great job here to ensure that we would get there. This presentation was packed with good content and Phil was able to drive the conversation by making sure we all had the necessary networking and Linux foundation. I truly enjoyed his style of presenting and his very clear and concise examples. To my surprise, the last set of slides on this workshop were related to explaing some of the more complex aspects of the OpenStack overlay networking, which as great as our Troubleshooting Neutron: Physical and Virtual Networks presentation started right at this point.

A final cherry-on-top-of-the-cake moment happened after this presentation as I met James Denton whose book, Learning OpenStack Networking (Neutron), was mentioned as a excelent source of reference during our presentation.FullSizeRender.jpg

The highlight of an event like this must be the opportunity one has to meet with fellow Stackers and share stories, knowledge and experience. It’s the whole conversation that happens between the presentations and 1×1 engagement with others.

Happy to have the opportunity to travel to the Summit. See you next time.