Radio-coverage main checkout: implemented work and evidence limits
Read-only assessment of current code, selected recorded experiments and missing completion evidence.
Historical pre-gate checkpoint
Preserved as draft evidence on manager instruction after the user paused substantive work. Scope approval and manager start authorization were required before resuming; both were subsequently recorded for v002. Findings below reflect only the read-only inspection already performed; study completion and acceptance remain unapproved. No further investigation, shared index/log edits or Git actions were performed after the pause instruction.
Scope and evidence classes
Inspected /Users/gumanist/projects/radio-coverage read-only, current HEAD 623ec672e36b4131abfdc7326e6f7e0713bc4408. Tracked working-tree files were clean; untracked artifacts include runs/poc/ and Sionna arrays/results. The source record preserves 24 selected original files (196,372 bytes), SHA256, absolute origin paths, HEAD and per-file Git status; full status and capture mapping are in its provenance JSON. Captured PoC metrics are untracked local files, not HEAD artifacts. No source-repository files were changed.22
Independently checked here: source-code presence/behavior by inspection, result JSON values/schema and selected prose–JSON consistency. Reported internal experiments: all accuracy, timing, convergence and flight-log conclusions; not rerun. Vendor/dataset claims: no new independent checks of WinProp, DJI, Sionna or dataset licensing. Assumptions/analysis: completion recommendations below, not user decisions. No raw flight logs, credentials, large arrays, dataset archives or checkpoints were read/imported.
What demonstrably exists
- A controller-to-drone geometric baseline implements free-space path loss plus classic Bullington equivalent knife-edge diffraction, radial raster evaluation, earth curvature, terrain/building collision handling. The service produces best-band link margin, Fresnel diagnostics and minimum altitude. The FastAPI app calls
rcm.service; its inspected path does not loadrcm_mlcheckpoints. This is an implemented baseline application, not a demonstrated ML-integrated product.123 - ML research code implements least-squares B0 distance, B1 multi-wall and B2 street-canyon features; U-Net and a RadioUNet architecture port; geometry, physics-feature and residual variants; threshold scaling; full validation/best-checkpoint selection; alternate-target evaluation; optional bounded residual/gate/off-transmitter crop settings. Their presence is not evidence every option was validated.456
- The official dataset split is encoded as 501/100/99 map groups. Site-selection code uses outdoor coverage thresholds and greedy multi-site selection; its three-site reference is greedy, not the global optimum. Negative regret versus that reference is therefore possible. Exact-best and top-5 metrics use sorted indices, so tied sites can affect reported hit rates.207
What recorded experiments establish
| Recorded experiment | Supported interpretation | Practical limit |
|---|---|---|
| Seed-0 128 px, 12 epochs, 501 training maps: geometry clipped DPM RMSE 2.353 dB; hybrid 1.994 dB. IRT2 RMSE 3.884 vs 3.752 dB. | Physics residual improves this recorded DPM fit by 0.359 dB; cross-simulator improvement is smaller (0.132 dB). | Both result JSONs untracked; different historical commit fields; this pair alone does not establish convergence, statistical significance or altitude transfer. |
| 99 maps × 80 sites, −110 dB: DPM reference, hybrid one-site regret 0.14 pp, geometry 0.41 pp; hybrid three-site regret 0.15 pp. | Strong recorded simulator-based site-ranking utility. | 256 m flat tiles, 1.5 m Tx/Rx, 25 m buildings, 5.9 GHz; not measured RF or passive detection. |
| IRT2 reference at −110 dB: calibrated tracer 0.54 pp regret, hybrid 0.94 pp; DPM used as predictor 0.94 pp. | Choice of reference materially changes rankings. | Does not show either simulator is physically correct. |
| Sionna subset: 20 maps × 80 sites, −110 dB, DPM regret 1.27 pp; IRT2 regret 1.24 pp. | Recorded tuned Sionna setup is weaker at this selection task than several cheaper methods. | Coarse 8 m receiver grid; narrow setup and subset; not a universal Sionna conclusion. |
The metrics above match selected captured JSON entries and the site-selection note. Runtime is experiment-specific; advertised ~0.04 s for 80 learned maps is GPU evaluation here, not end-to-end scene acquisition or deployment latency.14159108
The convergence report preserves one 2 km artificial mosaic, 350 fixed outdoor receivers and a −127.2 dB floor. Increasing tracer interactions 2→4 changes coverage at 100–200 m from 54% to 58%, and leaves 200–400 m at 15% and 400–800 m at 1%. Sionna's settings likewise leave 400–800 m at 1%. This supports numerical coverage stability for those sampled settings and calibration, not physical far-field accuracy.1211
The same study's 1,402 WinProp cases contradict a broad reading of earlier statements that street-level signal “really dies” within ~200–300 m: IRT4 coverage exceeds IRT2 by 17.1 pp at 100–150 m, 18.8 pp at 150–200 m and 16.4 pp at 200–260 m. The later note explicitly attributes likely pessimism to IRT2 calibration and recommends refitting ≥4-interaction references against IRT4/DPM. The 260–400 m sample is tile corners (17,769 pixels); there is no WinProp evidence for 400–800 m. Cross-document claims should follow these later qualifications.131116
The flight-log note reports 54 DJI logs, full-level links ~97% of the time and only six degraded flights (~three minutes); controller position was assumed at home, while signal fields were quantized/noisy. Reported AUC was distance-only 0.82 versus geometric baseline 0.77. This suggests limitations in the baseline and measurement design; it is not a usable detector calibration dataset. Only the aggregate note was captured; these values were not checked against private logs.17
Missing completion evidence and next action
- Reconcile the original ML task with actual artifacts. It asks for full-resolution reproduction, convergence and 3-seed comparisons over 50/150/500 maps.
rcm_ml/README.mdprovides a proposed 256 px/50-epoch recipe; recipes do not establish completed runs. The inspected main checkout contains PoC 128 px records and an early 256 px run; no full-protocol completion evidence was identified in this bounded pass. The original task's raw/clipped scale and split statements were superseded by the corrections in the MPS note. Reconcile other worktrees/history before concluding the original task failed or authorizing costly reruns.1819161415 - Choose and validate target geometry. Street-level simulator agreement does not establish pilot-to-drone altitude behavior or passive detector coverage. Define the intended output (path gain, received power, reliable link or detection probability), frequencies, heights and measurable target first. A useful next research test is recalibration against IRT4/DPM with ≥4 interactions and evaluation on held-out scenes; a useful measurement campaign records actual controller/detector position, antenna orientation/height and adequate weak-signal events. These are recommendations, not accepted scope.111716
- Separate research acceptance from integration. Establish reproducible run provenance (including untracked metrics/checkpoints), agreed held-out error/coverage metrics, uncertainty and failure boundaries. The application then requires an explicit model-loading/inference contract and a measured comparison against the existing baseline; detector integration additionally needs detector-specific observations and detection ground truth.326
Checks performed: curated copies hashed; selected metrics cross-read against captured source JSON; code inspected without imports or execution. Source tests exist but were not run, and checkpoint availability/quality was not verified. Knowledge validator is run separately; coordinator owns generated indexes and integration. No claim of scientific verification or independent study review is made. Next action: coordinator combines worktree/history evidence, resolves target output/geometry, and writes bounded acceptance criteria before implementation or new training.
Assessment integration
The coordinator resumed under approved v002, rechecked all four HEADs (unchanged), and integrated these bounded findings into the idea assessment. This file is a completed code/result inspection note; experimental performance has not been reproduced. Its earlier checkpoint describes the historical pause.
-
Exact captured original
rcm/propagation.py; seesourcesfor capture path. ↩ -
Exact captured original
rcm/service.py; seesourcesfor capture path. ↩↩ -
Exact captured original
app/main.py; seesourcesfor capture path. ↩↩ -
Exact captured original
rcm_ml/baseline.py; seesourcesfor capture path. ↩ -
Exact captured original
rcm_ml/models.py; seesourcesfor capture path. ↩ -
Exact captured original
rcm_ml/train.py; seesourcesfor capture path. ↩↩ -
Exact captured original
rcm_ml/cell_placement.py; seesourcesfor capture path. ↩ -
Exact captured original
docs/research/05-cell-placement.md; seesourcesfor capture path. ↩ -
Exact captured original
runs/mps/cell_placement.json; seesourcesfor capture path. ↩ -
Exact captured original
runs/mps/cell_placement_sionna.json; seesourcesfor capture path. ↩ -
Exact captured original
docs/research/07-far-convergence.md; seesourcesfor capture path. ↩↩↩ -
Exact captured original
runs/convergence/report.json; seesourcesfor capture path. ↩ -
Exact captured original
runs/convergence/winprop_rings.json; seesourcesfor capture path. ↩ -
Exact captured original
runs/poc/geom_s128_m501_seed0/test.json; seesourcesfor capture path. ↩↩ -
Exact captured original
runs/poc/hybrid_s128_m501_seed0/test.json; seesourcesfor capture path. ↩↩ -
Exact captured original
docs/research/03-mps-runs.md; seesourcesfor capture path. ↩↩↩ -
Exact captured original
docs/research/00-flight-logs-vs-baseline.md; seesourcesfor capture path. ↩↩ -
Exact captured original
docs/TASK-claude-code-01-ml-on-mps.md; seesourcesfor capture path. ↩ -
Exact captured original
rcm_ml/README.md; seesourcesfor capture path. ↩ -
Exact captured original
rcm_ml/data.py; seesourcesfor capture path. ↩ -
Exact captured original
tests/test_rcm_ml.py; seesourcesfor capture path. -
Local source record and its provenance JSON, captured 2026-10-10 UTC (2026-10-09 America/Los_Angeles). ↩