Using Kubernetes Deployments to deliver SmartOS iPXE Infrastructure

Image for post
Image for post

A couple of times before, I wrote about my various approaches to using iPXE to boot SmartOS:

I recently elected to move this (rather small) part of my operation to Kubernetes for a couple of reasons, but mostly because I wanted a fast way to refresh the served version of SmartOS archive, and I wanted it to be quickly reproducible, something my previous approach to using configuration management for this task that became sort of arduous to keep up with:

Since my approach basically comes down to 2 components, an Nginx frontend, and a backend manager for pulling down and mounting the platform archive to somewhere nginx can reach it relative to the iPXE script, Kubernetes offered a pretty straightforward way to do this.

I only needed to create a base Docker image that would contain my iPXE script, and an informational webpage:

#!ipxe
dhcp
set base-url http://sdc-ipxe.east.gourmet.yoga
kernel ${base-url}/smartos/smartos/platform/i86pc/kernel/amd64/unix -B smartos=true,console=ttyb,ttyb-mode="115200,8,n,1,-"
module ${base-url}/smartos/smartos/platform/i86pc/amd64/boot_archive type=rootfs name=ramdisk
boot

and the Dockerfile to build this image:

FROM nginxCOPY index.html /usr/share/nginx/html/index.html
COPY smartos.ipxe /usr/share/nginx/html/smartos.ipxe

Build and push it to the registry.

The tricky part, and where Kubernetes could be the most help, would be where I pull down the archive, place it on a volume, and then mount it to my above Nginx container. I did this by having an InitContainer pull the archive and decompress it to a volume (I used a hostPath, but since this is not data I wish to persist between schedulings, an emptyDir would suffice as well):

apiVersion: apps/v1
kind: Deployment
metadata:
name: ipxe-deployment
labels:
app: ipxe
spec:
replicas: 2
selector:
matchLabels:
app: ipxe
template:
metadata:
labels:
app: ipxe
spec:
containers:
...
initContainers:
- name: config-data
image: ubuntu:xenial
command: ["/bin/sh","-c"]
args: ["apt update; apt install -y wget tar; wget
https://us-east.manta.joyent.com/Joyent_Dev/public/SmartOS/platform-latest.tgz; tar xvf platform-latest.tgz -C /data; mkdir /data/smartos; mv /data/platform* /data/smartos/platform"]
volumeMounts:
- mountPath: /data
name: ipxe-data
volumes:
- name: ipxe-data
hostPath:
path: /mnt/kube-data/ipxe/
type: DirectoryOrCreate
imagePullSecrets:
- name: regcred

The above has, before my Nginx container is deployed, declared that an InitContainer will run first to extract the archive to the volume, and once that has completed, will proceed to create the Nginx container:

spec:
containers:
- name: ipxe
image: coolregistryusa.biz/jmarhee/smartos-ipxe:latest
imagePullPolicy: Always
ports:
- containerPort: 80
volumeMounts:
- mountPath: /usr/share/nginx/html/smartos
name: ipxe-data

So, once the InitContainer completes, the archive volume will be mounted to /usr/share/nginx/html/smartos which is where the ipxe script expects the path to the kernel to be.

The last piece is the Kubernetes Service to expose this to the network, so, if you use a provider like Packet (see the above linked pieces for consuming this endpoint) to boot using Custom iPXE, you can boot from this endpoint:

---
kind: Service
apiVersion: v1
metadata:
name: ipxe-service
spec:
selector:
app: ipxe
ports:
- protocol: TCP
port: 8085
targetPort: 80
type: LoadBalancer

Further Reading

Written by

Systems Engineer

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store