After extensive SRU testing, Ubuntu published cloud-init version 17.1.27 to Xenial, Zesty, Artful and Bionic series. One notable feature in 17.1.27 is the automatic configuration of IPv6 adddesses associated with your EC2 instances.
What has cloud-init done for me lately?
You no longer need to provide your own network configuration to cloud-init describing IPv6, manually add IPv6 config to /etc/network/interfaces.d/50-cloud-init.cfg or /etc/network/50-cloud-init.yaml nor reboot your instance. Cloud-init now configures ipv4/ipv6 support for you automatically on instance first boot. Think of how much more productive your day just got without that extra reboot :).
The skinny on IPv6 in EC2
Amazon's EC2 platform supports associating IPv6 addresses to instances in addition to IPv4 configuration. At the time of this article, private IPv4 network association is still mandatory for EC2 instances to communicate with Amazon's internal services.
Once you've added an IPv6 CIDR to one of your virtual private clouds (VPCs) in EC2, cloud-init will discover and configure IPv6 automatically for any instance created using that VPC.
Trails and tribulations: The nitty gritty
Getting IPv6 support in cloud-init was not without a bit of code wrangling. Below are some of the hoops cloud-init jumps through to get to your network configuration.
IPv6 discovery meets chicken and egg
Due to EC2 security/provisioning limitations, there are only a couple options for an instance to discover it has an IPv6 association.
- Attempt a dhcp6-discovery on all NICs and wait for either addresses or a timeouts
- Query the metadata service and to see which NICs are configured for IPv6 and setup dhcp6 on them.
Option 1 is a non-starter because cloud-init can't waste precious boot cycles on a 5 minute dhcp6 discovery timeout for NICs that may not even be IPv6.
For option 2, how can we talk to the metadata service on the network before we actually have network set up?
Door #1 - Add random link-local address to our NIC and query EC2 metadata
DHCP4 discovery takes precious cycles we would rather not spend if can set our own static IP address. EC2's metadata has a link-local address of 169.254.169.254. Since link-local can't be forwarded by routers, we tried statically configuring a random link-local address to talk to metadata, but the service only responds to requests from the known source IPv4 address allocated to the instance. By comparison, setting a static link-local address before talking to metadata is exactly what DigitalOcean's datasource does.
Door #2 - Dhcp4 discovery on primary NIC, then query metadata
Cloud-init takes this approach because EC2 metadata service is the least expensive mechanism by which an instance can determine if it has IPv6 configured, and we can't talk to metadata without the known allocated IPv4 address. DHCP4 discovery costs us a bit of time, but Amazon's dhcp services are quick to respond w/ IP request. The metadata tells us which NICs need IPv6 though presence of network/interfaces/macs//ipv6s.
DCHPv4 discovery in a sandbox
We know on EC2 that our initial DHCPv4 discovery in init-local may not be the "final" network configuration because we only discover IPv6 configuration once we crawl metadata. So, we run our discovery in a temp directory sandbox to avoid side-effects produced by dhcp-script or precipitating dhcplease files on the filesystem. This contained environment allows us to consume the minimal information we need to talk to the metadata service: our IP, mask, broadcast and router. Anything else handled by the stock dhcp client is unnecessary baggage at this point in boot. Our dhcp4 discovery takes us 0.033 seconds (that is 33 thousandths!). Thanks Amazon for super responsive dhcp servers.
EC2 potential improvements
All told, this is a good win for EC2. No longer having to hack /etc/network/interfaces or provide #cloud-config gets folks back to a simpler start state for IPv6.
Additional performance gains could be made for EC2 in the following areas:
Dropping the initial dhcp4 discovery before talking to metadata.
- If the EC2 platform can tell our instance what IP it should have (via /sys, DMI info of environment vars), we can statically configure our address and crawl the metadata address without sandbox DHCP discovery. This would further expedite EC2 instance startup.
EC2 metadata needs to present enough information that we could statically configure the NICs without the need to talk to dhcp at all.
- EC2 metadata doesn't currently have enough info about the IPv4 IPv6 network config to statically setup routes, so we have to rely on dhcp4/6 clients.
- Caveat: EC2 platform may rely on that dhcp discovery for some service and security setup that prevents this from being a viable solution.
Thanks for sticking with me. Stay tuned for future posts about other cloud-init 17.1 features such as:
- using python-boto3 for speedy integration testing on EC2
- cloud-init CLI commands
- Making EC2 boots faster
- cloud-init cloud-config schema annotations