The 3am database crash
No SaaS dashboard. No ticketing license. It ran on your own box — and every step left a record you can read later.
We speak ITIL — with receipts.
skops turns your machines into a self-healing, self-documenting operation. It watches for failures, fixes what it can, opens a ticket when it can't, learns the fix, and tells you how everything's doing — all on your hardware. No SaaS. No per-seat pricing.
pip install -e . && skops status
skops is not a new platform — it's the connective tissue that wires together tools you already run (alerts, scheduler, health checks, the knowledge graph) into a real ITIL loop. Everything that happens is recorded, validated by a manager agent, and retrievable. That's the "with receipts" part.
flowchart LR
EV["something breaks
(a service goes down)"] --> HEAL["skops tries to fix it
(restart / runbook / agent)"]
HEAL -->|fixed| LOG["logged + in the digest"]
HEAL -->|can't fix| TICK["opens an Incident
(a tracked ticket)"]
TICK -->|keeps happening| PROB["opens a Problem
(find the root cause)"]
PROB --> CHG["proposes a Change
(the permanent fix)"]
CHG --> CMDB["updates the CMDB
(the map of what you run)"]
CMDB --> RPT["daily/weekly digest
→ your phone"]
You don't need to know ITIL to use skops. But here's the whole idea in plain words — then the formal terms, if you're an ops pro.
Imagine you run a small shop with some machines in the back. Things break. ITIL is just a sensible, named way of dealing with "things break" so nothing falls through the cracks. That's it — it's a vocabulary + a set of habits, not software.
skops gives you those habits automatically. Every term on the right has a skops job attached to it.
"How bad is it?" is just Priority: P1 = everything's down · P4 = mildly annoying. And each ITIL process has a manager — a small AI agent with a personality suited to the job — that reviews the automation and asks the uncomfortable questions.
| Plain English | ITIL word | What skops does |
|---|---|---|
| "Something broke, deal with it now." | Incident | Detects it, tries to auto-fix, opens a ticket if it can't. |
| "This keeps breaking — why?" | Problem | Notices repeated failures, opens an investigation. |
| "Let's change something, carefully." | Change | Proposes the fix with a rollback plan; gets approval before prod. |
| "A thing we run and care about." | CI / CMDB | Keeps a live map of your machines & services. |
| "Our library of known issues + fixes." | KEDB | Remembers every fix so next time it's automatic. |
| "How are we doing?" | Reporting | Sends you a daily/weekly digest. |
No SaaS dashboard. No ticketing license. It ran on your own box — and every step left a record you can read later.
Remediation is tiered: L0 a plain restart, L1 a KEDB runbook retrieved by hybrid search, L2 an agent reasoning over the incident + runbook behind a confidence gate. Every successful fix becomes a new runbook — the library grows itself, and the next occurrence resolves one tier cheaper.
flowchart TD
A["health check fails"] --> B["sk-alert *.critical"]
B --> C{"known remediation?"}
C -->|"L0 restart"| R["systemd / monit restart"]
C -->|"L1 runbook"| RB["KEDB runbook
(hybrid search)"]
C -->|"L2 agent"| AG["agent reads incident
+ runbook, confidence gate"]
R --> V["verify (doctor.py)"]
RB --> V
AG --> V
V -->|ok| OK["close + grow KEDB"]
V -->|still broken| ESC["escalate → incident + human"]
OK -.->|each fix → new runbook| RB
sequenceDiagram
participant S as skscheduler
participant M as Manager agent
participant Q as ITIL store / CMDB
participant A as sk-alert / journal
S->>M: governance pass due
M->>Q: read open records in scope
M->>M: apply disposition
(skeptical / root-cause / cautious)
M->>Q: verify vs live reality
alt looks wrong
M->>Q: reclassify / promote / reopen
M->>A: alert + journal finding
else looks right
M->>Q: confirm + close + record evidence
end
The thing that makes auto-ops trustworthy is that nothing the automation does is taken on faith. Each ITIL process has a manager agent: a soul (a disposition that shapes its judgment), a tool scope (the only tools it may use), and a scheduled governance pass. They run after events and ask the uncomfortable questions.
At L0, one steward wears all the hats. On a single machine you run the Ops Steward — one pragmatic soul with all the ITIL tools. Level up to split the hats into the five dedicated managers below.
Pragmatic, thorough, calm under pressure — one person's worth of ITIL sense. "When in doubt: log it, escalate it, fix it."
Urgent, skeptical of "resolved." "Was the fix actually verified? Is this CI recurring?"
Analytical, root-cause-obsessed. Never satisfied with "we restarted it." "WHY did it fail?"
Cautious, governance-minded. "Every change needs a rollback plan. PIR is not optional."
Meticulous, accuracy-obsessed. "The CMDB is the truth — stale CIs are lies."
Communicative, trend-aware. Translates ops data into human language.
A cron job runs a script. A manager agent reads the situation and judges it through a consistent disposition — the Incident Manager is built to distrust "it's fine now," so it re-checks; the Change Manager is built to refuse changes without a rollback, so it flags them. The disposition is the control: it's how you get automation that argues with itself before it trusts itself.
skops runs the same model at every scale — you just turn on more of it. Start at L0 on one box; grow to L3 across a compliance-bound enterprise without re-architecting. Every level is the same ITIL loop and the same manager souls — only the adapters and the number of split-out agents change.
| Level | Context | Monitoring | ITIL scope | Managers | CMDB | ITSM |
|---|---|---|---|---|---|---|
| L0 — laptop | one person / machine | systemd + monit, auto-restart, weekly digest | minimal (alerts only) | one "Ops Steward" wears all hats | AGE graph | native |
| L1 — home | 1–5 machines | + Gatus + Healthchecks | incident + problem, standard changes | 5 managers split out | AGE (+ NetBox opt) | native (+ Icinga ingest) |
| L2 — work | team / prod | + skmon opt-in | + normal changes, async CAB, SLAs | full manager team | NetBox / iTop | iTop or Freshservice |
| L3 — enterprise | multi-team / compliance | full observability | full ITIL, ECAB, PIR, audit trail | full team + SLA roles | iTop / ServiceNow | ServiceNow / Freshservice |
Just me, one box? → L0: skops status, auto-restart, a weekly digest to Telegram.
Nothing else to configure. A few machines at home? → L1. A team in production? → L2.
Enterprise / compliance? → L3. You never rewrite anything to grow.
skops has a port for each axis and adapters behind it. Add a new backend by implementing the port — nothing else changes. Your monitoring feeds the loop; your ITSM and CMDB hold the state; remediation escalates from a restart to an agent.
cmdb_sync MERGEs Configuration Items
(Host · Service · Application · Agent) from live sources — service registry, heartbeat peers,
app.yaml, the skingest GPU cluster — into the AGE graph in skmem-pg. The Config Manager
continuously audits the graph against reality and reconciles drift. KEDB runbooks index into
the same Postgres (pgvector + mxbai), retrieved by hybrid_search_docs.
The canonical loop: a service dies; skops fixes it, notices it keeps happening, finds the cause, and ships the permanent fix — most of it while you sleep.
sequenceDiagram
participant Mon as Gatus
participant Heal as skhealer
participant KEDB as KEDB
participant IM as Incident Mgr
participant PM as Problem Mgr
participant CM as Change Mgr
participant You as Operator
Mon->>Heal: db.health = down (critical)
Heal->>KEDB: itil_kedb_search("db down")
KEDB-->>Heal: runbook: "restart skmem-pg"
Heal->>Heal: restart → doctor verify ✅
Note over IM: every 15m governance pass
IM->>PM: 3rd same-CI failure → promote
PM->>PM: RCA — backup starves DB memory
PM->>CM: propose CHANGE (stagger + mem limit)
CM->>You: CAB vote + rollback plan
You-->>CM: approve
CM->>CM: ansible apply → PIR (clean) ✅
sequenceDiagram
participant CFG as Config Mgr
participant SRC as live sources
participant AGE as AGE CMDB
participant Bus as sk-alert
CFG->>SRC: gather live state
CFG->>AGE: cmdb_sync.sync_all()
CFG->>AGE: drift_audit()
alt CI in CMDB, host dead
CFG->>Bus: alert: stale / orphaned CI
else live service not in CMDB
CFG->>AGE: add new Service CI
CFG->>Bus: alert: untracked service
end
flowchart TD
C["change proposed"] --> T{type?}
T -->|Standard| AUTO["auto-approve"]
T -->|Normal| CAB["CAB vote on coord board"]
T -->|Emergency| ECAB["ECAB → retro-review"]
AUTO --> PIR["Post-Impl Review"]
CAB --> PIR
ECAB --> PIR
PIR -->|caused incidents?| RB["rollback"]
PIR -->|clean| DONE["close + CMDB update"]
| You say… | skops does… |
|---|---|
| "something broke" | open / auto-heal an Incident |
| "it keeps breaking" | open a Problem (RCA) |
| "change something safely" | propose a Change (with rollback + approval) |
| "what do we run?" | query the CMDB |
| "what are the known issues?" | search the KEDB |
| "how are we doing?" | run a Report / digest |
For people who don't speak ITIL, skguide maps plain intents onto these flows.
skops installs into the ~/.skenv venv
and runs at L0 out of the box: one machine, one Ops Steward, auto-restarts + a digest.
skops is a Core capability. It consumes signals from the Compute tier (monitoring + model reasoning) and writes its state into Data (the AGE graph = CMDB · pgvector = KEDB / runbooks), orchestrated by shared platform services.
capauth — identityskmemory — memoryskmon / Gatus / Icinga — monitoringskmodel (Ollama) — agent reasoningskdata → skmem-pgskscheduler — governance jobssk-alert — bus + digestsskguide — ops onboardingSelf-healing, self-documenting, on your own hardware. We speak ITIL — with receipts.
"staycuriousANDkeepsmilin" 🐧