Gopala Krishna Akkineni

Home Work GPU platform for ML

GPU platform for ML

Giving data scientists the GPU capacity they need, without paying for GPUs nobody is using.

Where
State Farm
When
2025 to now
My role
DevOps Engineer (AWS/GCP)

The situation

Deep learning workloads need GPUs, and GPUs are both expensive to leave idle and painful to wait for. Data scientists want capacity on demand; the budget wants nothing running that isn’t doing work.

On top of training, the teams were moving toward LLM-based applications, which bring their own deployment and serving needs.

What I built

  • Orchestrated GPU and TPU-enabled clusters for deep learning workloads across AWS and GCP.
  • Automated resource monitoring and scaling for GPU workloads, so clusters grow for a job and shrink after it.
  • Integrated SageMaker, Bedrock and Hugging Face into the DevOps pipelines and worked with data scientists and ML engineers to deploy LLM-based apps to production.

How it flows

  1. Training job queued
  2. Scale up GPUs
  3. Train and track
  4. Register model
  5. Deploy and serve
  6. Scale down

Decisions I made and why

Scale on the work, not the calendar

Scaling GPU capacity around the actual jobs, instead of keeping a fixed fleet warm, is where most of the savings came from.

Measure first, then scale

Automated monitoring of GPU workloads showed when capacity sat idle and when it ran short, so scaling followed real usage instead of guesses.

Use managed ML services where they fit

SageMaker, Bedrock and Hugging Face cover a lot of ground. Plugging them into the existing pipelines was faster and safer than rebuilding what they already do.

What changed

Training and inference ran up to 30% faster, and automated scaling cut costs for GPU workloads by 15% while keeping performance where it needed to be.

What I’d do next

Show GPU spend per team and per model, so the people choosing instance sizes can see what their choices cost.

Stack GPU and TPU clusters, Kubernetes (EKS, GKE), SageMaker, Bedrock, Hugging Face

Details about State Farm’s systems are kept general on purpose. The approach, the tools and the results are from my work there.

Hiring for DevOps or platform work?

I’m open to DevOps, SRE and platform engineering roles. Email is the fastest way to reach me.