The Map Is the Product

How a multi-vendor assurance platform builds its network topology from raw exports, why alarm correlation is only as good as that map, and the five places where the map breaks. Written from the inside of one such build.

Halfway through a design review for a multi-vendor assurance platform, an architect asked a question that sounded simple: "Which layer builds the topology? Is it the probe, or does the probe only enrich the alarms?"

The room went quiet. Everyone had drawn the box. Nobody had opened it.

The question that stopped the meeting

Every assurance architecture I have seen has a box labelled inventory and topology. Alarms point at it. Performance points at it. The correlation engine, the service-impact calculator and the ticket router all point at it. It is the most-referenced box on the slide and the least-explained one, because the honest explanation is unglamorous: the topology is not collected from anywhere. It is reconstructed, table by table, from exports that were never designed to describe a path.

This article opens the box. I spent the last weeks reading a large vendor's assurance platform documentation end to end, then the hand-built scripts that fed it in a real multi-vendor mobile network in the Gulf region: five vendors across radio, microwave, IP access and optical, a few thousand sites, and a site database maintained by people. What follows is vendor-neutral by design. The mechanics are the same on any platform, because the physics of the problem is the same.

Nobody exports the topology

The first thing to unlearn, and this is what the design here assumes rather than a law of the field: the platform never reads the topology from the network as a whole. It reads from each vendor's element manager, and each element manager exports what it manages, in its own shape. A radio manager knows base stations, their ports, port MAC addresses and IP addresses. A microwave manager knows radios, radio links and a MAC forwarding table. An IP manager knows routers, interfaces, ARP and LLDP neighbours, tunnels and pseudowires. An optical manager knows cross-connections and signal flows per layer. None of them knows the path from a base station to the core, because that path crosses three of them. This is the opposite of general-purpose network discovery, which probes the live network to learn what is there [7] [8]; here the network is never asked anything.

So the collection layer is deliberately dumb. One collector per element manager, per data type, on a schedule. It parses a file or a REST response, maps the vendor's field names to a common set of tables, and writes rows. It computes nothing. That split is this design's: collectors parse and map, while enrichment and topology inference run as separate jobs. Other platforms draw the line elsewhere, and some enrich at collection time. The common tables that come out are boring and they are the whole game:

• NE: every managed element, its type, management IP, which manager owns it, and which site it stands on.
• PORT: every port with its MAC address, IP address list, VLANs and up/down state.
• LINK: A-end and Z-end of every intra-domain link the manager knows about.
• ARP and LLDP: who each interface has recently talked to, by IP and MAC, or by advertised neighbour identity [1] [2].
• MAC forwarding table: which MAC addresses a layer-2 node has learned on which port, with the VLAN.
• SERVICE: pseudowires, tunnels, VPNs and their hop lists, where the manager can export them [3].

And then two tables nobody can export: SITE and REGION. No element manager knows that base station NRE-0412 and microwave radio MW-0412-A stand on the same rooftop. That relationship lives in a site database, or in a naming convention, or in a spreadsheet, and someone maintains it by hand. In the network I looked at, the site identifier was parsed out of the equipment name by a rule ("strip the technology letter, replace the fourth character"), and the region came from a dictionary of three-letter prefixes copied into five different scripts. That is not a criticism of the engineers. It is the state of the art everywhere I have looked.

Restoration is a job, not a feed

With the raw tables loaded, a batch job runs, typically once a day at four in the morning, because it locks the topology data while it works. It has one goal: for every base station, produce a hop-by-hop path to the core, and along the way produce the physical links that no manager exported. It does this in three steps, each with a preferred method and a ladder of fallbacks.

Step 1: base station to first router

A base station port has a MAC address and an IP address. The access router's ARP table lists IP and MAC per interface [1]. When the two agree, the job creates a cross-domain link from the base station port to the router interface, and writes it into the LINK table with a flag that says generated. If the ARP table is missing, the ladder is: LLDP neighbours [2], then "same subnet, same site" as a weak guess, then the router's MAC forwarding table filtered by VLAN. If the base station export has no MAC at all, the platform can even fill it in backwards from the router's ARP, which is a compensation, not a discovery.

Step 2: through the transport

The best source for the path across the IP or packet-transport domain is the service itself: the pseudowire has a source and a sink, and the tunnel that carries it has a hop list [3]. When the manager exports both, the path is strongly evidenced - though freshness, protection switching and which way traffic is actually forwarded still have to be checked before anyone treats it as fact. When it exports neither, the job falls back to shortest path over the LINK table, from the access layer to the aggregation layer. That fallback has a quiet prerequisite that eats weeks: every node needs a network layer attribute (access, aggregation, core) so the algorithm knows where to start and where to stop. And when even the links are missing, the job inserts a virtual node with a question mark, so the path closes and the report can say "this hop was not observed."

Step 3: the transparent layers

Microwave and legacy SDH or MSTP transport are invisible to IP. They forward frames; they never answer ARP. From the router's point of view the base station is directly attached, which is exactly the hole in the map that costs you at three in the morning when a microwave link fails and six base stations alarm at once. The job closes it with the ugliest data on the list: it takes the base station's MAC and VLAN and looks for them in the microwave hub's MAC forwarding table; it applies a co-site rule (the base station and the first radio stand on the same site identifier); and then it walks the microwave manager's own link table between radios. Every one of those inputs is a place where a typo in a site identifier or a wrong VLAN silently deletes a hop.

Two numbers that tell the truth

The output of the job is two things: a physical topology of links, and one service path per base station, hop by hop, stored back into the inventory. The interesting part is that a good platform also measures itself, and the two numbers it produces are the most useful KPIs in the whole assurance stack:

• Restoration rate: base stations with a complete path, over all base stations. Broken down by scenario (radio to IP, radio to microwave to IP, radio to fixed access to IP).
• Coverage rate: nodes that sit on at least one path, over all nodes. A router that no path crosses is either idle or missing from the map.

Both numbers measure completeness, not correctness. A base station can have a complete end-to-end path that is simply the wrong path: built on a stale ARP entry, or on a shortest-path guess that never matched how traffic is actually forwarded. Restoration rate tells you the job found an answer everywhere; it does not tell you the answers are right. For that you need a third habit: sample a set of restored paths against the live network, track how old each input table was when the job ran, and keep a confidence flag on any hop that came from inference rather than from an exported link.

Behind the numbers sits a failure report, one row per base station that did not restore, with the algorithm that failed and the reason: "ARP entry does not find the port by identifier", "no shortest path found", "MAC matches multiple ports". Read a few hundred of these and a pattern emerges that is the most important sentence in this article: most restoration failures are naming mismatches, not missing hardware. The port exists. The ARP entry exists. They spell the router's name differently.

What correlation does with the map

Now the alarms arrive. The first thing the alarm pipeline does with each one is enrich it from the inventory: site, region, domain, network layer, and, critically, the alarm's position on a path and the identity of the link upstream of it. That last field is what turns a list into a graph.

The correlation rule is almost embarrassingly simple once the map exists: an alarm on a node whose upstream path contains a link that failed inside the time window is a child of the link alarm. The link alarm is the root cause. The group becomes one event, the event becomes one ticket, and the ticket goes to the transmission team rather than to six radio engineers. Service impact narrows sharply, though it does not fall out for free. What the map gives you is the set of resources potentially affected: count the sites, cells and subscribers behind the failed link, using the same paths. The rule generalises: rings, VNF-to-host layers in a cloud core, optical client-to-line layers, all the same walk over a different graph.

Without the map, the same seven alarms are seven tickets, six of them to the wrong team, and the transmission team learns about its own outage from the radio team's escalation. Every assurance vendor sells "AI correlation." Almost all of the value is in the graph, and the graph is the part nobody wants to talk about because building it is a data-quality project, not a machine-learning one.

Where the map breaks

Five gaps, and I have seen every one of them in production. They are ordered by how often they caused a restoration failure in the network I studied.

Notice that none of the five is an algorithm problem. The shortest-path code is fine. The MAC-table matcher is fine. What fails is the contract between the people who run the element managers and the people who run the platform, and that contract is usually a folder of scripts with "(1)" and "(2)" in the file names.

What I would build differently

If I were starting the inventory layer of an assurance platform tomorrow, I would spend the first month on nothing but the contract:

1. Define the eight tables once, vendor-neutral, with every column documented in plain English and a stated identifier rule. NE, PORT, LINK, ARP or LLDP, MAC, SERVICE, SITE, REGION. Everything downstream reads only these.
2. One collector per source, outside the element manager. The knowledge in those hand-built scripts (which command gives which field, which export contains the MAC, how the site identifier is derived) is the real asset. Move it into a documented field-mapping table per vendor and a small transformation job. Most of it is SQL over CSV.
3. A quality gate before restoration: row counts, null counts, foreign-key resolution, duplicates by identifier, and a diff against the previous run. Cheap, and it catches the naming mismatch a day before the correlation engine does.
4. LLDP before descriptions, router-side ARP before base-station-side ARP, and a probability column on every generated link.
5. Naming rules as data. The site-identifier rule and the region dictionary are configuration, stored once, with an owner. Not five copies in five scripts.
6. Publish the two numbers. Restoration rate and coverage rate on the same dashboard as alarm volume. When they drop, correlation is about to get worse, and now you know why.

One note on standards, because someone always asks whether this "follows eTOM." It does, at the process level: everything above is the TM Forum process called Resource Data Collection and Distribution, plus Manage Resource Inventory under Resource Management and Operations support [5]. But eTOM is a process map; it says nothing about columns. The data standard that goes with it is the information framework and its Resource Inventory API [6], and the honest position is that the eight tables above are a project format from which a standard view can be generated when a consumer needs one. On this project standards were treated as an export profile rather than the working store. That was an implementation choice: a standard information model can equally be the internal one, and on a greenfield build that is worth arguing for.

The five-minute version

I turned this article into a short animated explainer for a design review: where the data comes from, the three restoration steps, the two numbers, the correlation walk and the five gaps. It is vendor-neutral and free to reuse with attribution.

The correlation engine is not the product. The map is the product. The engine just reads it.

Mohamed Kadri operates and transforms Tier-1 telecom networks, and led a NOC's L2-to-L3 autonomy transition that automated 75 percent of Level-1 incidents. He now builds AI and cloud products, and maintains guardplane, an open-source control plane for agent fleets. LinkedIn · mkadri85.github.io

Quick questions and answers

Does the collector or probe build the network topology?

No. A collector copies raw tables from each element manager (elements, ports, links, ARP, LLDP, MAC tables, services) into the inventory and computes nothing. A separate restoration job reads those tables, matches them, and writes generated links and per-site paths back. Alarm enrichment is a third function that only reads the inventory.

What data do I need from each vendor to restore a service path?

From the radio manager: the base station list with site identifier, and each port's MAC and IP address. From the IP manager: elements, ports, intra-domain links, ARP or LLDP, and if possible pseudowire and tunnel hop lists. From the microwave manager: radios, radio links and the MAC forwarding table. From optical: cross-connections and per-layer signal flows. Plus a site and region table maintained by people.

How does the platform connect a base station to the right router?

It matches the base station port's IP and MAC against the router's ARP table. If ARP is unavailable it uses LLDP neighbours, then same-subnet matching, then the router's MAC forwarding table filtered by VLAN. The resulting link is stored with a flag marking it as generated rather than exported.

Why do microwave links not appear in ARP?

Microwave and legacy transport forward Ethernet frames transparently and do not answer ARP, so the router sees the base station as directly attached. The platform recovers the hidden hops by matching the base station's MAC and VLAN in the microwave node's MAC forwarding table, applying a co-site rule, and walking the microwave manager's own link table.

How do I know the restored topology is correct?

Two rates: restoration rate (base stations with a complete path over all base stations) and coverage rate (nodes on at least one path over all nodes), plus a failure report per base station naming the algorithm and reason. In practice most failures are identifier mismatches between tables, not missing equipment.

Why does alarm correlation depend on topology?

Because the core rule is a walk: an alarm on a node whose upstream path contains a failed link is a child of that link's alarm. Without paths, the engine has only timestamps and names to work with, and one microwave failure becomes six unrelated tickets sent to the wrong team.

Sources and references

Standards are cited for the mechanisms named in the text. The field observations are the author's own, from a multi-vendor mobile network in the Gulf region, anonymised to region level; vendor documentation was read but is not quoted.

1. D. Plummer, RFC 826: An Ethernet Address Resolution Protocol, IETF, 1982. Supports: the ARP table as the IP-to-MAC source used in restoration step 1 (sections: Nobody exports the topology; Restoration).
2. IEEE, IEEE 802.1AB-2016: Station and Media Access Control Connectivity Discovery (LLDP). Supports: LLDP neighbour tables as the preferred link source over description parsing (sections: Restoration; Where the map breaks).
3. S. Bryant and P. Pate, RFC 3985: Pseudo Wire Emulation Edge-to-Edge (PWE3) Architecture, IETF, 2005. Supports: pseudowire source and sink and the tunnel that carries it as the exact-path source in restoration step 2 (section: Restoration).
4. S. Nadas (ed.), RFC 5798: Virtual Router Redundancy Protocol (VRRP) Version 3, IETF, 2010, section 7.3. Supports: the virtual router MAC address 00-00-5E-00-01-{VRID} that defeats OUI-based vendor lookup (section: Where the map breaks, gap 4).
5. TM Forum, GB921 Business Process Framework (eTOM) suite. Supports: Resource Data Collection and Distribution and Manage Resource Inventory as the processes this pipeline implements (section: What I would build differently).
6. TM Forum, TMF639 Resource Inventory Management API, v4.0, and TMF642 Alarm Management API, v5.0. Supports: the standard views that can be generated from the working tables (section: What I would build differently).
7. UMBOSS, What is network discovery? Independent vendor blog. Supports: the observation that discovery protocols are implemented differently per vendor and that reconciliation against planned inventory is the last step (sections: Where the map breaks; What I would build differently).
8. Progress WhatsUp Gold, Best practices series: network discovery. Supports: scheduled rediscovery and credential and protocol coverage as the operational baseline (section: What I would build differently).
9. Narration script of the video above: Topology restoration and alarm correlation, script with timings.

Related: The Loop Meets the Microwave Network on what an agentic loop must reason about on the same backhaul, and Your AI Agents Need a NOC on the operating discipline that makes any of this safe to automate.