PlanSpot: One-Shot Symbol Spotting on CAD Drawings

Problem

Construction estimators count symbols and measure runs across drawing sets of 300+ sheets. Every project legend defines its own symbols, so a detector trained on fixed classes misses the long tail. Drawings are also unlike photos: a full sheet is about 10,800 × 7,200 px, symbols are 20–60 px, and lines are 1–3 px wide.

Approach

Frozen DINOv2-S runs on the drawing upsampled 2× in overlapping tiles, so small symbols span several patches. The exemplar template is taken in context from the drawing’s own feature map, and the drawing is embedded in all 8 rotations and mirrors so symbols match in any orientation. A pixel NCC matcher over the same poses is the baseline. Both are scored on the same FloorPlanCAD tiles, with thresholds tuned on separate validation drawings.

How It Works

1. 1
   ExemplarThe user boxes one symbol on the drawing.
2. 2
   Dense featuresDINOv2-S patch features at 2× scale in overlapping 518 px tiles, for each of 8 poses.
3. 3
   MatchMean cosine similarity between the exemplar’s in-context patch template and every window, then peaks and NMS.
4. 4
   Cut-offA global threshold tuned on validation drawings, or an adaptive cut at the largest gap in each query’s scores.
5. 5
   EvaluatePrecision, recall and F1 at IoU ≥ 0.5 by symbol size and class, with splits by source drawing.

What I Built

• A one-shot spotter on frozen DINOv2 with tiled upsampling, in-context templates and 8-pose matching, plus a pixel NCC baseline.
• A benchmark on FloorPlanCAD with drawing-level splits, size buckets and count error, plus an offline study of per-query cut-offs.
• A DINOv2 + LoRA fusion segmenter for thin walls, with an ablation against a pixel U-Net, frozen features, and a clDice topology loss.
• A static results viewer on Hugging Face Spaces and a local Gradio demo.

Impact

On the same held-out tiles, DINOv2 reached F1 0.489 vs 0.466 for pixel matching, with much higher precision (0.78 vs 0.50) and less than half the count error. The classical baseline still won on small symbols, because CAD symbols are exact copies. A per-query cut-off lifted pixel matching to F1 0.555, which showed the threshold matters as much as the backbone, and that recall (~0.4) is the next bottleneck. For wall segmentation, LoRA on DINOv2 reached F1 0.761 vs 0.704 for a pixel U-Net and 0.731 for frozen features, and a clDice loss cut wall-piece count errors by a further 23% at a cost of 1.3 F1 points.