TREED: A Tree-Structured Diagram Benchmark for Multimodal Graph Reconstruction

Description

TREED is a curated benchmark for evaluating topology reconstruction in multimodal foundation models. It provides tree-structured diagram images, ground-truth graph annotations represented as structured edge lists, model responses, and code. TREED operationalizes structural abstraction by preserving graph topology while removing semantic and stylistic information, with connectivity derived from symbol positions using a modified spanning-tree procedure. Diagrams are derived from diverse electrical-system datasets and standardized to isolate topology from semantic and visual cues, enabling reproducible evaluation of graph reconstruction and diagram understanding. The repository also includes the TREE-EDs dataset release (47 images) annotated using the Label Studio platform. Additional electrical-system datasets for benchmarking and dataset construction are catalogued at: https://mayot0.github.io/TopDat-ED/ Characteristics: • Images: 7,544 • Resolution: 393×403 • Node Range: 4–20 • Graph Type: Undirected • Ground Truth: Structured Edge Lists • Source Datasets: 7 (see references and links) • Metrics: GED, WL, Spectral, Path, Node Count Difference (NCD), and Tree Edge Deviation (TED) • Quality Control: Automated verification

Steps to reproduce

To render images and generate ground-truth graph annotations from electrical diagrams, a Docker-compatible dataset_construction pipeline is included with TREE-EDs as an example dataset. Links to the additional electrical datasets used to create TREED are also provided. Once downloaded, these datasets can be placed in the data/ directory to reproduce or extend the benchmark. Dataset Stats: Dataset name (Format): Number of files selected from dataset (4 to 20 nodes), Number of unique classes TREE-EDs (Pascal VOC): 45 files, 2 classes AMSNet (BBox JSON): 626 files, 39 classes CGHD (Pascal VOC): 1733 files, 29 classes Ci2N (LabelMe JSON): 1498 files, 30 classes Circuit Recognition (Pascal VOC): 188 files, 9 classes Netlistify (YOLO TXT): 3443 files, 16 classes SESYD (GOM XML): 10 files, 16 classes

Institutions

• Universidad de Oviedo

  Asturias, Oviedo
• Robert Gordon University

  Scotland, Aberdeen

Categories

Related Links

Licence