An AI agent fixed a broken OSPF adjacency in a four-router lab

The question

We gave an AI agent CLI access to a broken four-router lab, no hints, and watched whether it could find and fix a dead OSPF adjacency. The full transcript follows: every show command, every diagnosis, and the fix.

Setup

Four Cisco IOS XE routers (IOL 17.12) in a square, running in CML 2.10:

flowchart TD
  R1["R1<br/>Lo0 1.1.1.1"]
  R2["R2<br/>Lo0 2.2.2.2"]
  R3["R3<br/>Lo0 3.3.3.3"]
  R4["R4<br/>Lo0 4.4.4.4"]
  R1 ---|"10.1.12.0/24"| R2
  R1 ---|"10.1.14.0/24"| R4
  R2 ---|"10.1.23.0/24"| R3
  R4 ---|"10.1.34.0/24"| R3

Links are /24s (10.1.12.0, 10.1.23.0, 10.1.34.0, 10.1.14.0), loopbacks are 1.1.1.1 through 4.4.4.4, everything in OSPF area 0. The healthy baseline was verified before any break: all adjacencies FULL, every loopback in every routing table, cross-ring ping 100 percent.

The agent is Claude, connected to the routers through CML’s API (an MCP server that can run CLI commands). The method we held it to: show commands first, state the diagnosis, then fix with the smallest possible change, then prove recovery.

Three breaks, applied one at a time, each restored before the next:

1. Hello timer changed to 30 seconds on one side of the R1-R2 link.
2. Subnet mask changed to /25 on one side of the R2-R3 link.
3. MD5 authentication enabled on one side of the R3-R4 link.

Each break hit a different link. The R4-R1 link was never touched, which is what let us prove nothing else broke:

flowchart TD
  R1["R1"]
  R2["R2"]
  R3["R3"]
  R4["R4"]
  R1 -. "BREAK 1: hello 30 on R2" .- R2
  R1 ---|"never touched"| R4
  R2 -. "BREAK 2: mask /25 on R3" .- R3
  R4 -. "BREAK 3: MD5 on R3 only" .- R3

The run

Break 1: the timers

R1 lost its neighbor on Ethernet0/0. Interface up, so not physical. The tell came from comparing the same link from both sides:

R1# show ip ospf interface Ethernet0/0 | include Timer
  Timer intervals configured, Hello 10, Dead 40, Wait 40, Retransmit 5

R2# show ip ospf interface Ethernet0/0 | include Timer
  Timer intervals configured, Hello 30, Dead 120, Wait 120, Retransmit 5

OSPF requires matching hello and dead intervals on a segment. Mismatched timers mean each side rejects the other’s hellos and the adjacency starves. Fix: no ip ospf hello-interval on R2. Adjacency returned to FULL.

Break 2: the mask

R2 lost its neighbor on Ethernet0/1. Timers checked clean this time, so the agent did not just repeat its last diagnosis. It looked one line deeper and found the mismatch:

R2# show ip ospf interface Ethernet0/1 | include Internet
  Internet Address 10.1.23.2/24, Interface ID 3, Area 0

R3# show ip ospf interface Ethernet0/0 | include Internet
  Internet Address 10.1.23.3/25, Interface ID 2, Area 0

On broadcast networks the OSPF hello carries the network mask, and a mismatch rejects the hello. Fix: put R3 back to /24. Adjacency returned.

Break 3: the authentication

R4 lost its neighbor on Ethernet0/0. The log showed the dead timer expiring:

R4# show logging | include OSPF
*Aug 12 11:11:25.727: %OSPF-5-ADJCHG: Process 1, Nbr 3.3.3.3 on Ethernet0/0
  from FULL to DOWN, Neighbor Down: Dead timer expired

R4’s interface showed no authentication configured. R3’s did:

R3# show ip ospf interface Ethernet0/1 | include authentication|digest
  Cryptographic authentication enabled

R3 demands MD5 authentication; R4 sends plain hellos. Neither one accepts what the other sends, and the adjacency never comes up. Fix: remove the one-sided authentication (adding the matching key on R4 would work just as well; the right call in production depends on what your security policy says the link should be). The adjacency came back FULL. Every loopback returned to the routing table, and ping across the ring hit 100 percent.

Scorecard

What the test showed

The agent went 3 for 3 on classic single-fault OSPF breaks, and its method was sound. It compared both ends of the failing link and ruled things out based on what the output actually said. It did not fix by reboot or by shotgun.

These are textbook failures in a clean lab with one fault at a time. A live production network adds a change freeze, an on-call engineer who gets paged at 2 a.m., and real consequences if the fix is wrong. Passing this test is the minimum bar. It does not prove this works in production. A harder test with compound faults and misleading symptoms is on the list, and we expect the score to drop.

What production would need

Before an agent like this touches a network you are paid to keep up:

• Read-only by default. Everything in the diagnosis phase was show commands. The tooling should enforce that separation. Relying on trust is not enough.
• An approval step between diagnosis and fix. The agent states the hypothesis and the exact change; a human approves it.
• A rollback plan per change. Every fix here was one reversible line. Compound changes need staged rollback the agent can execute.
• An audit trail. The full transcript of this run was saved before this note was written. Demand the same from any tool that claims to do this.

Run this yourself

Everything above is downloadable: the exact lab and the exact prompt from this run.

The lab. ospf-break-fix.yaml is the topology exported from the machine this test ran on. Import it into CML, start it, and it boots with OSPF already converged. Four nodes, about 4 GB of RAM, fine on the free tier. Credentials are cisco / cisco.

The three breaks, each one line of config, each with its restore:

! Break 1, on R2: mismatched hello timer
interface Ethernet0/0
 ip ospf hello-interval 30

! Break 2, on R3: mismatched mask
interface Ethernet0/0
 ip address 10.1.23.3 255.255.255.128

! Break 3, on R3: one-sided MD5
interface Ethernet0/1
 ip ospf authentication message-digest
 ip ospf message-digest-key 1 md5 fieldtest

Apply one, wait up to 40 seconds for the dead timer, and go.

The prompt. This is the whole thing the agent was held to. Copy it:

You have CLI access to a small OSPF network in a lab. Exactly one thing is
broken: an adjacency that should be up is down. Find it and fix it.

Work in four phases and do not skip ahead.

PHASE 1: OBSERVE. Show commands only. No configuration commands, not even
ones you are certain about. Start by finding which adjacency is missing, then
gather evidence about that link specifically. Whenever you check something on
one end of a link, check the same thing on the other end before you draw a
conclusion. Most adjacency faults are disagreements between two routers, and
you cannot see a disagreement from one side.

PHASE 2: DIAGNOSE. State, in plain language:
  - which adjacency is down
  - what the output shows on each end
  - what you believe the cause is
  - what would have to be true for you to be wrong
Name the specific line of output that supports your conclusion. If two causes
fit the same evidence, say so and go back to Phase 1 for the command that
separates them.

PHASE 3: FIX. Propose the smallest reversible change that addresses the cause
you named, on one router. Write out the exact commands and the exact commands
that would undo them. Then stop and wait for me to approve before you apply
anything.

PHASE 4: PROVE. After the change, show that the adjacency is FULL, that the
routes came back, and that traffic passes end to end. Then confirm you did not
break anything else: check every other adjacency in the topology, not just the
one you fixed.

Rules that hold for the whole run:
- Never reboot, reload, or bounce an interface as a diagnostic step.
- Never change more than one thing at a time.
- If the evidence does not support a conclusion, say "I do not know yet" and
  keep looking. A wrong confident answer is worse than an honest gap.
- Do not tell me what you are about to do and then do it in the same breath.
  Phase 3 ends with you waiting.

The reasoning behind each rule, and a table for scoring the run, is in agent-prompt.md.

The harness. An instruction to stay read-only during diagnosis is not enough on its own. It has to be enforced as a control. A privilege-level-1 device account and tool-level allowlisting are what actually stop a model from configuring your network while it is guessing. setup.md covers the wiring three ways, cheapest first, including the version where you paste output by hand and skip the plumbing entirely.

A Claude Code skill. If you use Claude Code, save ospf-triage/SKILL.md and its reference/ospf-failure-modes.md into .claude/skills/ospf-triage/. It runs this method as a harness: show commands only, a stated hypothesis with its falsifier, and a proposed one-line fix it will not apply until you approve it. The failure-mode reference is worth reading on its own even if you never install the skill; it is what to compare across the two ends of a link, and the exact line of output that settles each case.

Also here: breaks.md with the restores and harder variants, and README.md tying it together.

When the single faults get easy, delete “exactly one thing is broken” from the prompt. Apply two breaks on different links and see what the reasoning does. That test is next on our list, and we will publish the transcript.