The history of cloud-init and virtualization in Ubuntu
The story about the creation of cloud-init begins in October 2007. At the time, Ubuntu would have a “developer summit” shortly after each release where we would set the direction for the next release, brainstorm, plan, hack, design, and socialize. At this particular Ubuntu Developer Summit (UDS), we decided to make an effort to integrate virtualization in the upcoming Ubuntu Server release due out in April 2008¹. Since I had a little bit of virtualization experience, I was put in charge of it.
Our virtualization efforts had two main components: Ubuntu as the guest and Ubuntu as the host. Being the host means running virtual machines. Being a guest means running in a virtual machine.
Ubuntu as virtualization host
The leading technology for Linux virtualization at the time was Xen, while KVM had only been released earlier that same year. KVM also required an Intel CPU with VT-x capabilities. Nevertheless, we gambled that VT-x support would be commonplace soon, and decided to focus on KVM. I was a little apprehensive about this decision, but luckily the proponents prevailed.
If there ever was such a thing as “full stack engineering,” this was it. There was work to be done in the kernel. There was work to be done in kvm’s emulated hardware. libvirt, which provides a management layer for virtualization, needed work. virt-manager, a graphical interface for virtual machines needed work. Some of this involved piecing together improvements from many different sources and getting them to play nicely together. Some of it had to be done from scratch.
Looking back on it now, I’m surprised how far we got in that release cycle. Some of the notable changes were:
- The Ubuntu kernels got first class KVM support.
- Paravirtualized network and block device (virtio-net and virtio-blk) drivers were added KVM/QEMU.
- virt-manager was added to Ubuntu².
- We added virtio support to libvirt and virt-manager.
Ubuntu as virtualization guest
Running an operating system as a virtual machine (i.e. as a “guest”) should Just Work™. And for the most part, it does.
However, there were a lot of things we could do to make it as seamless as possible. For example, if you run Linux under VMWare, your experience is a lot better if you use a special graphics driver (avoiding the expensive graphics adapter emulation) and a special mouse driver that made the pointer move seamlessly in and out of the virtual machine’s “screen”. This is what the open-vm-tools package did.
The virtio-net and virtio-blk kernel drivers were also added to the Ubuntu kernel. They dramatically improved performance when running Ubuntu inside KVM.
I somehow also found time to create VMBuilder, a fully automated tool to create virtual machine images.
Ubuntu goes cloud
At some point in 2008 we started working on getting Ubuntu onto Amazon EC2. At the time, EC2 was based on Xen and regular users could not upload their own kernels. My team mate, Chuck Short, worked tirelessly with Amazon on getting the Ubuntu Xen kernel working on EC2. This apparently succeeded in October:
Eric Hammond was an early EC2 champion who had been building unofficial Ubuntu images for EC2 for a while. His experience and guidance proved tremendously helpful in getting Ubuntu running well on EC2.
VMBuilder fit nicely into this project and was used to build the official Ubuntu images for EC2 for years to come.
Eric was the one who told me about a common trick used on EC2: Fetch whatever was in the user-data and execute it. “user-data” is a blob of data you can provide to EC2 when you are launching a new VM. It is made available to running VM’s through a special service that lives at http://169.254.169.254/.
Thus, ec2-init was born:
ec2-init would also fetch SSH keys from the EC2 meta-data service. That’s all it did to begin with.
My memory gets kinda hazy at this point, to be honest. Or, rather, my memory does not match what I can see in the git history. I know that VMBuilder tried its best to build small images and I feel relatively confident that something would resize them on boot, but ec2-init didn’t start doing that until 2011:
…but I digress.
By mid 2009, I was still working very actively on all of the above, and none of it was my main objective at Canonical anymore. I was supposed to focus on getting Eucalyptus working in Ubuntu, but that was… well, a shit show.
We hired the inimitable Scott Moser to the Ubuntu Server team in June 2009. Scott soon got up to speed and took over the development of ec2-init later that year.
ec2-init was renamed to cloud-init in January of 2010.
cloud-init and OpenStack
As the embedded tweet above suggests, I moved on from Canonical a little while later. I might write an article about OpenStack and how that all happened, but today we’re focused on cloud-init, so I’ll stick to that.
The functionality provided by cloud-init was something you’d need on any cloud computing platform, so it was a natural fit for OpenStack, especially since it originally aimed for EC2 compatibility.
cloud-init was already used beyond Ubuntu on EC2, but the intimate relationship (by way of the people involved in both) between ec2-init and OpenStack made adoption skyrocket in the years following the creation of OpenStack.
cloud-init became the go-to for anyone wanting to customize virtual machines at boot time in OpenStack. This brought cloud-init to dozens of operating systems. This in turn made cloud-init the natural target for cloud providers who wanted to add support for their technology to many operating systems.
It was around the same time that cloud-init started supporting cloud-config, which is a declarative approach to some of the most common things people would do in their user-data scripts. E.g. creating users, adding ssh-keys, adding apt or yum repositories, installing packages, joining a Puppet master, etc. Great work, Scott :)
And that’s where we are now. From cloud-init’s REAMDE.md:
…and as recently as last year (i.e. 2022), the FreeBSD Foundation made cloud-init support a priority, so it’s still very much alive and well.
