Using Claude in DevOps without pretending it's magic
Where an LLM helps a delivery pipeline, where it is the wrong tool, and the four architectural choices that decide which of those two you end up with.
Facts checked 7 August 2026 against the vendor's own documentation. Exam codes, curricula and product versions change — verify against the official page before you book anything.
- A single callOne request, one response. Most useful DevOps uses live here.
- A workflowMulti-step, but your code owns the control flow.
- An agentThe model decides what happens next. Justify this one.
Most “AI in DevOps” writing is either a product pitch or a demo that would fall over on contact with a real cluster. This is what we’ve found works, and what doesn’t.
Start by not building an agent
The most common mistake is reaching for the most complex thing available.
There’s a ladder, and you should stand on the lowest rung that solves your problem:
A single call. Classify this alert. Summarise this incident channel. Extract the failing test from this log. One request, one response. Most useful DevOps applications of an LLM live here, and they’re boring and reliable.
A workflow. Multi-step, but you own the control flow. Your code decides what happens next; the model does the language-shaped part of each step. Deterministic wherever determinism is free.
An agent. The model decides what to do next. Only justified when the task is open-ended and hard to specify in advance.
Before building an agent, answer four questions honestly. Is the task multi-step and hard to fully specify? Does the outcome justify the cost and latency? Is the model good at this task type? And the one people skip: can errors be caught and recovered from?
That last question is where DevOps differs from most domains. An agent with kubectl access and a wrong idea doesn’t produce a bad summary. It produces an outage.
Where it helps
Incident triage, not incident response. Summarising a 400-message incident channel into a timeline is a language problem, and models are good at it. Deciding to fail over a database isn’t, and shouldn’t be delegated.
Log and trace archaeology. “Find every error in this 50MB log that isn’t the known noisy one, group them, and tell me which started today.” Real work, low blast radius, easy to verify.
Post-incident writing. A draft postmortem from a timeline and a transcript saves an hour of the worst kind of writing. A human still owns the conclusions.
Reviewing infrastructure changes. Terraform plans and Kubernetes manifests are a good fit: dense, structured, full of consequential details that are easy to miss at 5pm. Treat the output as a second reviewer rather than as a gate.
Writing the boring code. The parser, the migration script, the fifteenth CronJob that looks like the previous fourteen.
Where it doesn’t
Anything where being confidently wrong is expensive and unverifiable. If you can’t cheaply check the answer, don’t generate the answer.
Deterministic work. If the inputs fully determine the output — routing, tallying, filtering, formatting — that’s code. We regularly find pipelines making a model call per item where a filter would do, at meaningful cost and latency, for a worse result. Count the model call sites in your pipeline and ask of each whether its inputs fully determine its output.
Production access without a gate. Which brings us to the architectural part.
Four choices that decide whether this works
Give it dedicated tools, not a shell
A bash tool gives a model enormous leverage and gives your harness an opaque command string. Every action looks identical to your code: a string.
Promote the actions you need to gate into dedicated tools with typed arguments. scale_deployment(name, replicas) is something you can validate, log, rate-limit and require approval for. bash -c "kubectl scale ..." isn’t.
The rule of thumb: start with bash for breadth in a sandbox, then promote to dedicated tools whenever you need to gate, render, audit or parallelise the action. Anything hard to reverse should be a dedicated tool.
Approval gates belong in the tool, not in the prompt
Don’t ask the model to confirm before doing something dangerous. Ask your code.
Gate inside the tool’s implementation: when the model calls delete_namespace, your handler prompts a human and returns “user declined” if they say no. The model can’t route around this, because the decision was never the model’s to make. A prompt instruction can be argued with. A function that returns a refusal can’t.
Reach for the right level of thinking
Current Claude models support adaptive thinking, where the model decides how much reasoning a task needs, combined with an effort setting that controls overall spend.
For DevOps work this maps cleanly. Classifying an alert is a low-effort task. Working out why a rollout is stuck across three interacting controllers isn’t. Setting effort per route, rather than picking one globally, is the cheapest optimisation available.
Don’t put credentials in the prompt
We cover this at length in our MCP post, but it bears repeating because the failure is silent and permanent. Prompts and messages persist in session history and get returned by the events API, so a key placed there is durably readable.
Use the vault mechanism, or keep the credential entirely host-side by making the authenticated call from your own orchestrator in response to a tool request. The container never needs to see it.
On agents that run unattended
If you do run an agent autonomously, on an overnight remediation loop or a scheduled audit, two things matter more than they do interactively.
Ground its claims. Models will report progress that outran what they verified. Requiring every progress claim to be tied to a tool result from the same session substantially reduces fabricated status. So does asking explicitly for faithful reporting: if tests failed, say so with the output; if a step was skipped, say that.
Tell it nobody is watching. An agent that pauses to ask “shall I proceed?” at 3am blocks until morning. If the run is autonomous, say so in the system prompt, and be explicit about which actions it may take without asking and which it may not.
What it adds up to
An LLM in a delivery pipeline behaves like a very good junior engineer with excellent recall, no judgement about consequences, and no ability to tell you when it’s out of its depth.
Give it the work you’d give that person: drafting, summarising, reviewing, the tedious mechanical changes. Keep the irreversible decisions with people, and put the gates in code rather than in prompts.
That framing has served us better than either the hype or the dismissal.
Next steps
Get help
Running this in production?
We operate Kubernetes and OpenShift for clients across the EU and the Gulf, and train the teams who inherit them. Platform assessments, migrations and hands-on enablement.
Talk to us