Stage 7 · Master
AI for SRE Workflows
Plug LLMs into the on-call life — ground models in your runbooks, build ChatOps copilots, cut alert noise with AIOps, and let AI draft the postmortem while you fix the outage.
8 modules48 lessons5.7h of reading48/48 live today
Prerequisite
Production experience.
What this course leaves you with
- Integrate LLMs into operations workflows
- Build AI-powered alerting and RCA tools
- Understand responsible AI in production systems
AIOpsLLMRAGChatOpsIncident Response
Why this stage matters — AI is integrated — validate and certify the broader stack.
Progress
0 of 48 lessons complete. Progress is stored locally in your browser so you can pick the path back up later.
0%
course completion
01
6 lessons
LLM Foundations for Operators
Just enough model literacy to use LLMs safely in production operations.
- 01How LLMs Actually WorkTokens, context windows, temperature, and why models hallucinate.7 min
- 02Embeddings & Semantic SearchVectors, similarity, and turning logs and docs into searchable meaning.6 min
- 03Prompting for Ops TasksSystem prompts, few-shot examples, and structured JSON output.7 min
- 04Tool Calling & Function APIsLetting a model run kubectl, query Prometheus, or open a PR safely.8 min
- 05Cost, Latency & Data PrivacyToken budgets, self-hosted vs API models, and keeping secrets out of prompts.6 min
- 06Failure Modes & TrustHallucination, prompt injection, and never trusting AI in a blast radius.6 min
02
6 lessons
RAG Over Your Ops Knowledge
Ground answers in your runbooks, docs, and past incidents instead of the open internet.
- 01Why RAG for OperationsGrounding, freshness, and citing the runbook instead of guessing.6 min
- 02Chunking Runbooks & DocsSplitting Markdown, wikis, and postmortems for high-signal retrieval.7 min
- 03Vector Databasespgvector, Qdrant, and Weaviate — indexing, metadata, and filtering.7 min
- 04Hybrid Retrieval & RerankingCombining keyword + semantic search and reranking for precision.8 min
- 05Indexing Past IncidentsTurning years of postmortems into an answerable knowledge base.7 min
- 06Evaluating Retrieval QualityRecall, groundedness, and catching confidently-wrong answers.7 min
03
6 lessons
ChatOps & On-Call Copilots
Bring the assistant into Slack, the terminal, and the incident channel.
- 01Slack & Teams BotsEvent APIs, slash commands, and streaming responses into a channel.7 min
- 02The On-Call CopilotAnswering "what does this alert mean?" with runbook-grounded context.8 min
- 03kubectl & Terraform CopilotsNatural-language ops with dry-run confirmation and read-only guardrails.8 min
- 04Agents With GuardrailsApproval steps, allow-lists, and blast-radius limits for autonomous actions.8 min
- 05Human-in-the-Loop DesignWhen to suggest, when to act, and how to keep the human accountable.6 min
- 06Auditing AI ActionsLogging every prompt and action so a copilot is debuggable and safe.6 min
04
6 lessons
AIOps: Detection & Triage
Use models to cut alert noise and find the signal faster than a human dashboard scan.
- 01Anomaly DetectionSeasonality, baselines, and catching drift metrics thresholds miss.8 min
- 02Alert Correlation & Noise ReductionGrouping a storm of alerts into one probable root cause.8 min
- 03Log Clustering & SummarizationCollapsing millions of log lines into the handful that matter.7 min
- 04Automated Triage & RoutingClassifying severity and paging the right team automatically.7 min
- 05Predictive & Leading SignalsForecasting disk fill, capacity, and saturation before they page.7 min
- 06AIOps PitfallsAlert fatigue 2.0, false confidence, and keeping models accountable.6 min
05
6 lessons
Incident Response With AI
Let AI handle the paperwork of an incident so responders can focus on the fix.
- 01Live Incident SummarizationReal-time "where are we?" summaries for a busy incident channel.7 min
- 02Automated Timeline GenerationReconstructing the sequence of events from chat, alerts, and deploys.7 min
- 03Root-Cause AssistanceCorrelating deploys, metrics, and changes to suggest likely causes.8 min
- 04Drafting the PostmortemGenerating a blameless first draft with contributing factors and actions.7 min
- 05Stakeholder CommsTurning technical detail into status-page and exec-ready updates.6 min
- 06Closing the Knowledge LoopFeeding resolved incidents back into the RAG index automatically.6 min
06
6 lessons
Running AI Reliably (LLMOps)
Treat the AI in your stack like any other production service — with SLOs.
- 01Prompt & Model VersioningGit for prompts, staged rollouts, and reproducible model config.7 min
- 02Evals & Regression TestingGolden datasets, LLM-as-judge, and catching quality regressions in CI.8 min
- 03Guardrails & SafetyInput/output validation, PII redaction, and prompt-injection defense.7 min
- 04Caching & Cost ControlSemantic caching, batching, and keeping the token bill predictable.7 min
- 05Serving & Latency SLOsInference servers, streaming, timeouts, and fallbacks under load.8 min
- 06Monitoring the AI ItselfTracking hallucination rate, drift, latency, and cost as first-class SLIs.7 min
07
6 lessons
Ops Agents & Automation Tools
Build AI-assisted operational tools that plan, call APIs, and require approvals before risky actions.
- 01Agent ArchitectureDesign planners, tool routers, memory stores, approval gates, and execution logs for operations agents.8 min
- 02Tool Schema DesignExpose kubectl, Terraform, PagerDuty, and GitHub actions through typed JSON schemas and strict validation.8 min
- 03Read-Only FirstSeparate observe, propose, and execute phases with dry-run commands, diffs, and human confirmation.7 min
- 04Workflow State MachinesUse durable state, retries, compensating actions, and timeout handling for multi-step automation.8 min
- 05Secrets & PermissionsScope service accounts, short-lived credentials, audit logs, and redaction for LLM tool calls.7 min
- 06Agent Test HarnessesReplay incidents with mocked tools, golden transcripts, expected actions, and failure injection.8 min
08
6 lessons
LLM Cost, Latency & Observability
Operate LLM-backed SRE workflows with measurable cost controls, latency budgets, and quality signals.
- 01Token BudgetingMeasure prompt, completion, retrieval, and tool-call tokens with per-feature budgets and alerts.7 min
- 02Model RoutingRoute requests across small, large, and local models using task complexity, latency SLOs, and fallback rules.8 min
- 03Semantic Cache DesignUse embeddings, similarity thresholds, TTLs, and invalidation rules to reduce repeated LLM calls.8 min
- 04Streaming & TimeoutsImplement streaming responses, cancellation, retries, circuit breakers, and graceful degradation for slow providers.7 min
- 05LLM TracingTrace prompts, retrieved chunks, tool calls, model parameters, latency, and errors with OpenTelemetry.8 min
- 06Quality DashboardsTrack groundedness, refusal rate, user feedback, escalation rate, and cost per resolved incident.7 min