An API for simplified generation of cloud-init user-data documents.
- Introduction
- Installation
- Usage
- Known issues and limitations
- Getting help
- Contributing
- License
- Authors and history
cloud-init is the industry standard multi-distribution method for cross-platform cloud instance initialization. It is supported across all major public cloud providers (AWS EC2, Azure, Google Cloud, etc.), provisioning systems for private cloud infrastructure, and bare-metal installations.
When a cloud IAAS user creates a VM instance in the cloud (e.g., an EC2 instance), they can optionally provide a user-data document. This document is passed to cloud-init at VM boot time, which uses the document to, among other things, configure the VM for first-time use by installing packages, creating OS users, running custom scripts, etc.
Python package cloud-init-gen
provides a simple API for generation of a well-formed cloud-init user-data document, and rendering the
document into a format expected by cloud service providers (typically base-64).
Some key features of cloud-init-gen:
- Type inference from #-comment and MIME-style headers, as well as explicit MIME-type.
- Automatic re-encoding with #-comment or MIME Conent-Type, minimizing rendered document size.
- Transparent support for multipart cloud-init user-data documents, composed one part at a time.
- Automatic conversion from JSON-friendly structured data to YAML, typical for cloud-config document parts.
- Support for custom MIME types and additional MIME headers.
- Automatic compression with GZIP if necessary to keep rendered binary under 16KB (the maximum user-data size).
- Rendering to string, binary, or binary encoded as a base-64 string.
Python: Python 3.7+ is required. See your OS documentation for instructions.
The current released version of cloud-init-gen
can be installed with
pip3 install cloud-init-gen
Poetry is required; it can be installed with:
curl -sSL https://install.python-poetry.org | python3 -
Clone the repository and install cloud-init-gen into a private virtualenv with:
cd <parent-folder>
git clone https://github.com/sammck/cloud-init-gen.git
cd cloud-init-gen
poetry install
You can then launch a bash shell with the virtualenv activated using:
poetry shell
Let's say you want to use boto3 to create an EC2 instance on AWS, and you want to do a few things with cloud-init
- On every boot, before doing anything else, you want to clear
/var/log/per-boot.log
if it exists, and write the boot time to the newly cleared file - On the first boot, you want to install a few Debian/Ubuntu packages, and you want to install the latest version of Docker.
- You want to set up docker for authentication to AWS ECR in the same region
import shlex
import json
import boto3
from cloud_init_gen import CloudInitDoc
sess = boto3.session.Session()
aws_region = sess.region_name
sts = sess.client('sts')
resp = sts.get_caller_identity()
aws_account_id = resp['Account']
user_data = CloudInitDoc()
boot_hook = '''#boothook
#!/bin/bash
echo "Booted on $(date)" > /var/log/per-boot.log
'''
user_data.add(boot_hook) # MIME type text/cloud-boothook is infered from the "#boothook" header.
ecr_domain: str = f"{aws_account_id}.dkr.ecr.{aws_region}.amazonaws.com"
docker_config_obj = {
"credHelpers": {
"public.ecr.aws": "ecr-login",
ecr_domain: "ecr-login"
}
}
docker_config = json.dumps(docker_config_obj, separators=(',', ':'), sort_keys=True)
cloud_cfg = dict(
repo_update = True,
repo_upgrade = "all",
apt = dict(
sources = {
"docker.list": dict(
source = "deb [arch=amd64] https://download.docker.com/linux/ubuntu $RELEASE stable",
keyid = "9DC858229FC7DD38854AE2D88D81803C0EBFCD88"
),
},
),
packages = [
"jq",
"awscli",
"collectd",
"ca-certificates",
"curl",
"gnupg",
"lsb-release",
"docker-ce",
"docker-ce-cli",
"amazon-ecr-credential-helper",
],
runcmd = [
[ "bash", "-c", f"mkdir -p /root/.docker && chmod 700 /root/.docker && echo {shlex.quote(docker_config)} > /root/.docker/config.json && chmod 600 /root/.docker/config.json" ],
[ "bash", "-c", 'echo "it works!"' ],
],
)
user_data.add(cloud_cfg) # will be rendered as yaml with implicit MIME type text/cloud-config
print(f"Final user-data (text):\n====================\n{user_data.render()}\n====================")
print(f"Final user-data (base64):\n====================\n{user_data.render_base64()}\n====================")
ec2 = sess.client('ec2')
resp = ec2.run_instances(
...
UserData=user_data.render_base64() # boto3/EC2 expect user-data to be encoded with base-64
...
)
- Has not been tested for compatibility against cloud service providers other than AWS EC2.
Please report any problems/issues here.
Pull requests welcome.
cloud-init-gen is distributed under the terms of the MIT License. The license applies to this file and other files in the GitHub repository hosting this file.
The author of cloud-init-gen is Sam McKelvie.