bgpviews
Design v1.0-rc1 Nothing is implemented yet. The design is frozen at v1.0-rc1; the implementation follows it.

Your routers. Your routing history. Standard files.

bgpviews turns one or two routers into your own small RouteViews: a single binary receives BMP, writes RouteViews-shaped MRT into a plain directory, and leaves a Parquet catalog beside it.

monocle, bgpkit-parser, bgpdump and DuckDB can read the output without a service in the middle. The result is an archive you own, on your own disk.

Control plane
One TOML file
Deployment
One binary
Output
MRT + Parquet
Telemetry
None by default
01 / OVERVIEW

The archive public collectors cannot give you

Public archives show what a few hundred routers saw. They cannot show what your routers saw, what your inbound policy dropped, or what your network looked like last Tuesday at 03:00. Building that privately normally means assembling parsers, writers, rotators and a database before the first useful file exists.

bgpviews makes that pipeline one program with one configuration file and one output directory. The archive stays where you run it, written and read locally, with no database server and no second daemon to operate.

Network operators

Keep your own routing history without running a data platform.

Researchers

Build an archive for vantage points that cannot be published.

IXPs

Offer members BGP history without operating a collector service.

Where it fits

bgpviews is a collector you run yourself: it takes BMP from your routers and writes files that existing tools already read. It sits beside the systems you have rather than replacing them, and it is at design stage today.

02 / MECHANISM

From BMP stream to readable archive

A router points BMP at the binary. Each client has one listening port and a source-IP allowlist. This boundary matters because BMP has no authentication of its own.

STEP 01 / RECEIVE

Accept BMP frames

The single binary receives router sessions. A raw session journal retains every accepted frame, including Peer Up and Peer Down, so replay is possible.

STEP 02 / SEPARATE

Read the per-peer header

The view comes from every message's per-peer header, never from configuration. Each configured view becomes its own collector named <client>-<view>.

STEP 03 / PUBLISH

Write standard artifacts

RouteViews-shaped MRT, JSON sidecars and a Parquet catalog land in a plain directory for existing tools to read.

Four views remain distinct

  • Adj-RIB-In pre-policy
  • Adj-RIB-In post-policy
  • Loc-RIB
  • Filtered Loc-RIB

Pre-policy and post-policy exports are separate collectors, making policy visible as data rather than something inferred later. A record whose view is not configured is counted and left in the journal. It is never written into a different collector.

03 / ON DISK

One directory existing tools already read

The output directory resembles a RouteViews collector. The Parquet catalog is a projection of the segment sidecars and can be rebuilt from them. Tombstones ensure that a deleted file is not later reported as lost.

Schematic archive tree

Fixed layout, one collector per view
archive/                              # one output directory
├── <client>-<view>/bgpdata/YYYY.MM/ # RouteViews layout, one collector per view
│   ├── UPDATES/updates.…HHMM.gz      # MRT, plus a sidecar JSON per file
│   └── RIBS/rib.…HHMM.gz             # derived snapshots, derivation version
├── coverage/<collector>/<stream>     # ranges: what covers what, and why not
├── sessions/<client>/<epoch>/        # every accepted BMP frame, replayable
├── catalog/                          # rebuildable Parquet projection
│   ├── segments   ├── sessions       #   peer_state, gaps, peers beside them
└── tombstones/                       # a deleted file never looks lost

Paths are the ones the design fixes, and a reader can rebuild everything below `catalog/` from the sidecars and the coverage log. Updates segments use a UTC-aligned cadence, gzip and the RouteViews directory layout, with a JSON sidecar per segment. RIB snapshots are periodic, derived from a live in-memory trie, and marked derived with a derivation version.

04 / CONTROL PLANE

One TOML file as the control plane

Behaviour lives in one TOML file. SIGHUP re-reads it. In this design the file is the whole control plane and there is no HTTP surface; the CLI is limited to running, inspection, verification and repair.

Configuration contract

Schematic TOML comments
# bgpviews.toml
# One file is the entire control plane.
# The overview does not yet define TOML key names.
#
# For each client, configure:
#   * one listening port
#   * a source-IP allowlist
#
# Views come from BMP per-peer headers,
# never from this configuration file.
# Reload with SIGHUP. The file is the control plane.

Start and inspect

Defined commands
bgpviews check
bgpviews run
bgpviews status

check validates the configuration and prints the router snippet and firewall rule to paste. status shows coverage and the evidence separating dead, idle and refused.

The TOML block is a configuration checklist, not a proposed key schema. Publishing invented keys here would imply an interface the overview does not define.

05 / COVERAGE

Coverage is a partition of time, and it is auditable

The archive does not report a status per interval and leave you to guess what the gaps mean. It keeps one ordered list of ranges per collector, and every instant of that collector's life falls in exactly one range. A reader can therefore tell “nothing happened” apart from “nothing was recorded”, which is the question an archive is actually asked.

Two kinds of range, and no third.
RangeWhat it tells a reader
segmentAn MRT file covers this range. A segment may contain zero records, which is how “we were watching and nothing happened” is written: the file exists and its sidecar says record_count=0.
gapNo file covers this range, and the range names why: no session, a view never observed, a peer that never completed its table, backpressure, a closed session, a removed view.

Ranges are contiguous and append-only. A closed range is never edited, never split and never removed, so a coverage file that disagrees with the MRT files beside it is a verify failure rather than a silent inconsistency.

Three states a segment can carry. Only the first two are publishable.
StatePublished meaning
completeThe range closed at its configured boundary, belongs to one session and one view, its journal is contiguous, and no reason applies. It never claims that loss at the exporter before its socket did not happen.
partialSomething named went wrong: a peer still syncing, a session that ended mid-interval, a restart, a refusal, a disk guard. The reason is in the sidecar, not inferred later.
withheldThe segment failed its own integrity check and sits in quarantine/. Its range stays in the timeline so the attempt is recorded, and no reader treats it as coverage.

A file that is absent, expired on purpose, or failed verification is three different things, and the archive says which: missing, a tombstone, and corrupt.

06 / CLI

Five commands, four for a healthy system

The full command surface. Catalog rebuilding is reserved for disaster recovery.
CommandPurpose
bgpviews runStarts bgpviews.
bgpviews checkValidates the config and prints the router snippet and firewall rule to paste.
bgpviews statusShows coverage and the evidence that separates dead, idle and refused.
bgpviews verifyReconciles the catalog against disk.
bgpviews rebuild-indexRebuilds the catalog. Disaster recovery only.
07 / CATALOG

Inspect the projection with DuckDB

The catalog contains segments, sessions, peer_state, gaps and peers. Because it is Parquet in a plain directory, DuckDB can read it without a bgpviews service.

Catalog inspection query

Illustrative path, no assumed columns
-- The overview names catalog entities, not file paths or columns.
-- This path is schematic; inspect the segment rows directly.
SELECT *
FROM read_parquet('archive/catalog/segments.parquet')
LIMIT 20;

The query intentionally avoids invented column names. Use verify to reconcile the catalog against disk; use rebuild-index only for disaster recovery.