IXSonar — BGP Visibility Analysis Across Internet ExchangesIXSonar — BGP Visibility Analysis Across Internet Exchanges

What it is

A network analysis tool that compares BGP route visibility across multiple Internet Exchange Points (IXs) — including different IX.br locations in Brazil, plus a few international IXs for comparison. It answers questions like: which prefixes announced at one IX also appear at another? How many keep the same AS path at both locations? How does that coverage vary across CDNs, access providers, banks, and government networks?

The practical goal behind the analysis is to show that, in most cases, it isn't worth a provider investing in long-distance physical connectivity to a far-away IX (e.g., running a link all the way to the São Paulo IX from Fortaleza) because most of the prefixes and networks available at a major IX are already replicated at regional IXs. The result is a historical dashboard that helps network teams make that decision with real data, understanding connectivity redundancy across locations.

Challenge

Data comes from RouteViews (University of Oregon), which collects route snapshots (RIBs) from routers around the world — but not every IX has full coverage: certain dates and times have capture gaps depending on the source router. Files came with missing data, inconsistent formatting between snapshots, and an enormous volume of information to process. The biggest technical challenge was finding a way to process that volume with minimal disk I/O and maximum speed, since the real bottleneck wasn't CPU — it was read/write throughput.

Solution

• A Rust binary that automatically downloads RIB snapshots from RouteViews, testing multiple days in parallel until a valid file is found, and applies configurable include/exclude filters over AS-PATH and BGP communities.
• Apache Parquet as the intermediate format — columnar, immutable files with built-in statistical summaries, chosen for drastically reducing disk I/O compared to alternatives like SQLite.
• Apache Arrow to move data directly in memory (RAM) between pipeline stages, avoiding unnecessary disk serialization/deserialization.
• Embedded DuckDB to compare IX pairs — prefix overlap, AS path consistency, and per-category network metrics — processing global, IPv4, and IPv6 dimensions in parallel, all via direct SQL over the Parquet files.
• Python scripts that read the final results and generate a self-contained HTML dashboard with interactive charts, served by a small Flask server.

Result

A complete pipeline — from raw data collection to final visualization — capable of efficiently processing large volumes of BGP data on both disk and memory, delivering historical visibility into connectivity redundancy across Brazilian and international IXs.

Tech Stack

• Rust (tokio, reqwest, rayon, embedded DuckDB)
• Python (pandas, matplotlib, Flask)
• Apache Parquet as intermediate format
• Apache Arrow for in-memory data transport
• Google Charts for the dashboard

Technical Highlights

• Automatic, resilient download of historical BGP RIBs from RouteViews, testing multiple days until a valid snapshot is found
• Handling of incomplete data and inconsistent formatting across different collections/routers
• Use of Parquet + Arrow to minimize disk I/O and maximize processing speed at scale
• Configurable filter engine with AND/OR logic over BGP communities and AS-PATH
• Route overlap comparison done entirely in SQL over Parquet files, via embedded DuckDB
• Native Rust parallel processing across the three analysis dimensions (global, IPv4, IPv6)
• Network classification by business category (CDN, ISP, financial, government) to segment the analysis