Node Disks Manager: Operator + Node Agent for Local Disk Allocation

Description

This project demonstrates an operator + node-agent pattern for discovering node-local disks and coordinating their allocation.

node-disks-operator reconciles NodeDisk resources and applies label-driven policy.
node-disks-agent runs on every node, discovers local disk files, and updates node-observed status.
NodeDisk is cluster-scoped and represents one disk on one node.

Policy behavior:

when a NodeDisk has label localtest.example.com/auto-reserve=true, node-disks-operator sets:
- spec.desired.state=Reserved
- spec.desired.owner=auto
when the label is removed, auto-owned reservation is reverted to Available.

Status ownership is split:

agent writes status.node (phase, capacityBytes, lastSeenTime, message)
operator writes status.controller (accepted, observedGeneration, conditions)

Source code: github.com/martishin/k8s-node-disks-manager

Quickstart

1) Start local multi-node cluster

make minikube-up

* [node-disks-dev] minikube v1.36.0 on Darwin (arm64)
* Starting "node-disks-dev" primary control-plane node
* Starting "node-disks-dev-m02" worker node
* Starting "node-disks-dev-m03" worker node
* Starting "node-disks-dev-m04" worker node
* Done! kubectl is now configured to use "node-disks-dev"

2) Build and load images

make images-load

docker build -t node-disks-operator:dev -f Dockerfile.controller .
docker build -t node-disks-agent:dev -f Dockerfile.agent .
minikube -p node-disks-dev image load node-disks-operator:dev
minikube -p node-disks-dev image load node-disks-agent:dev

3) Install chart

make helm-install
kubectl -n node-disks-system get pods -o wide

NAME                                   READY   STATUS    NODE
node-disks-agent-fh2lq                 1/1     Running   node-disks-dev-m04
node-disks-agent-qfcfd                 1/1     Running   node-disks-dev-m03
node-disks-agent-zbtlz                 1/1     Running   node-disks-dev-m02
node-disks-operator-76cc5d65df-6wjbk   1/1     Running   node-disks-dev-m03

4) Verify discovery

kubectl get nodedisks -o wide

NAME                                 NODE                 DISK    DESIRED     PHASE
node-disk-node-disks-dev-m02-disk1   node-disks-dev-m02   disk1   Available   Available
node-disk-node-disks-dev-m02-disk2   node-disks-dev-m02   disk2   Available   Available
node-disk-node-disks-dev-m02-disk3   node-disks-dev-m02   disk3   Available   Available
node-disk-node-disks-dev-m03-disk1   node-disks-dev-m03   disk1   Available   Available
node-disk-node-disks-dev-m03-disk2   node-disks-dev-m03   disk2   Available   Available
node-disk-node-disks-dev-m03-disk3   node-disks-dev-m03   disk3   Available   Available
node-disk-node-disks-dev-m04-disk1   node-disks-dev-m04   disk1   Available   Available
node-disk-node-disks-dev-m04-disk2   node-disks-dev-m04   disk2   Available   Available
node-disk-node-disks-dev-m04-disk3   node-disks-dev-m04   disk3   Available   Available

Validate Reservation Flow

1) Pick and reserve a disk

ND_NAME=$(kubectl get nodedisks -o jsonpath='{.items[0].metadata.name}')
kubectl label nodedisk "$ND_NAME" localtest.example.com/auto-reserve=true --overwrite
kubectl get nodedisk "$ND_NAME" -o jsonpath='{.spec.desired.state} {.spec.desired.owner} {.status.node.phase} {.status.node.capacityBytes}{"\n"}'

Reserved auto Reserved 67108864

2) Check side effect on the node files

NODE_NAME=$(kubectl get nodedisk "$ND_NAME" -o jsonpath='{.spec.nodeName}')
DISK_ID=$(kubectl get nodedisk "$ND_NAME" -o jsonpath='{.spec.diskID}')
minikube -p node-disks-dev ssh -n "$NODE_NAME" "ls -1 /var/lib/node-disks-manager/disks | grep -E '^${DISK_ID}\\.img(\\.reserved)?$'"

disk1.img
disk1.img.reserved

3) Verify directly on node via SSH

minikube -p node-disks-dev ssh -n "$NODE_NAME" "ls -la /var/lib/node-disks-manager/disks"
minikube -p node-disks-dev ssh -n "$NODE_NAME" "cat /var/lib/node-disks-manager/disks/${DISK_ID}.img.reserved"

-rw-rw-r-- ... disk1.img
-rw-r--r-- ... disk1.img.reserved
-rw-rw-r-- ... disk2.img
-rw-rw-r-- ... disk3.img
reserved

4) Release and verify marker removal

kubectl label nodedisk "$ND_NAME" localtest.example.com/auto-reserve-
kubectl get nodedisk "$ND_NAME" -o jsonpath='{.spec.desired.state} {.spec.desired.owner} {.status.node.phase} {.status.node.capacityBytes}{"\n"}'
minikube -p node-disks-dev ssh -n "$NODE_NAME" "ls -1 /var/lib/node-disks-manager/disks | grep -E '^${DISK_ID}\\.img(\\.reserved)?$'"

Available  Available 67108864
disk1.img

Cleanup

make helm-uninstall
make minikube-delete

Local Development

make help
make tools
make codegen
make test