The situation
GPUs are the expensive part of an ML platform, and the classic mistake is leaving them running. I wanted to practise exactly that: a cluster whose GPU node group sits at zero until a job needs one.
I haven't applied this code to a real AWS account, so nothing has been created and no money has been spent. It is validated, linted and scanned on every push, and the README says so on its first line.
What I built
- A VPC across two availability zones, with private and public subnets and one NAT gateway, using the community VPC module.
- An EKS cluster whose API endpoint is public but limited to the IP addresses you list.
- A small always-on system node group on t3.medium machines.
- A GPU node group on g5.xlarge Spot instances, set to a minimum and desired size of zero. It uses the NVIDIA EKS image so the drivers are already installed, and it carries a taint so only GPU jobs land on it.
- A hardened test pod that runs nvidia-smi to prove a GPU works.
- A pipeline that runs Trivy first, then format, init, validate, tflint and kubeconform, with no AWS credentials at all.
How it flows
- Set your own IP
- Plan
- Apply
- Install the NVIDIA plugin
- Scale the GPU group up
- Run the job
- Scale to zero and destroy
Decisions I made and why
Zero by default
The GPU group starts empty, so it costs nothing until someone scales it. It is tagged so a cluster autoscaler could manage it, but I haven't added one, so scaling is manual today.
Make the unsafe thing impossible
The variable for allowed IP addresses has no default and refuses 0.0.0.0/0, so the cluster API can't be opened to the whole internet by accident.
Taint the GPU nodes
The taint keeps ordinary pods off expensive machines. Only pods that explicitly tolerate it can land there.
Spot, behind one variable
Spot is much cheaper and suits jobs that can restart. One variable switches the group to on-demand.
Accept three scan findings in writing
Trivy flagged the public API endpoint, open outbound traffic from the nodes and the default network ACLs. Each one is in the ignore file with the reason next to it. Ignoring a finding with a reason is normal. Ignoring it silently isn't.
Scan before init
After terraform init, Trivy started scanning the example folders inside the downloaded modules and failed. Running the scan first fixed it.
What changed
Every push is formatted, validated, linted and scanned without giving GitHub a single AWS credential, and the part that costs the most money is off by default.
What I’d do next
Remote state in S3 with locking, Karpenter or the cluster autoscaler, the NVIDIA device plugin installed by Terraform or GitOps instead of by hand, VPC endpoints instead of open outbound traffic, and a real apply in a sandbox account with a billing alert.
Stack Terraform, AWS EKS, VPC, NVIDIA GPU nodes, Spot instances, tflint, Trivy
Read the code and the pipeline runs: terraform-aws-eks-gpu on GitHub.