More Works from Our Lab

PointMAC: Meta-learned Adaptation for Robust Test-time Point Cloud Completion

Meta-auxiliary test-time adaptation that refines each input individually, producing sample-specific completions instead of generic ones.

NeurIPS 2025

No Adaptation Without Observation: Observability-Constrained Test-Time Prompt Tuning for LiDAR Semantic Segmentation

IROS 2026

Linlian Jiang1,2, Wentao Ju1, Sadman Rakib Pinon1,2, Jianwei Xian1, Zhixiang Chi3, Xinxin Zuo1*, Yang Wang1,2*
1Concordia University   2Mila – Quebec AI Institute   3University of Toronto
*Corresponding authors
Paper Code arXiv

A wise man, therefore, proportions his belief to the evidence.

David Hume
Overview of observability-constrained test-time prompt tuning
Under partial LiDAR sensing, pseudo-label reliability is spatially heteroscedastic. Existing TTA methods update globally shared parameters uniformly, so gradients from weakly supported regions corrupt the representation and fragment thin structures. We instead compute a geometry-aware observability score that quantifies sensing reliability in LiDAR scenes and use it to modulate prompt-based adaptation within a frozen backbone, so that parameter updates are driven only by geometrically supported evidence.

Abstract

LiDAR semantic segmentation often degrades under real-world deployment due to evolving sensing conditions, while collecting new annotations for retraining is impractical. Test-time adaptation (TTA) updates model parameters online using pseudo-label supervision, but directly applying standard TTA strategies to LiDAR data is challenging. Because pseudo-label reliability is spatially heteroscedastic under range-dependent sparsity and occlusion, uniform updates on globally shared parameters can inject unstable gradients and destabilize adaptation. We propose a geometry-constrained test-time prompt tuning framework for LiDAR semantic segmentation. Our method estimates per-location sensing reliability from depth-consistent beam terminations and neighborhood support, and uses it to reweight spatial supervision. Adaptation is confined to lightweight prompt adapters inserted into a frozen backbone, with spatial gating to prevent unreliable regions from perturbing globally shared representations. A temporally smoothed prototype alignment strategy further stabilizes online updates by accumulating reliable semantic evidence over time. Experiments on standard LiDAR benchmarks demonstrate improved adaptation stability and segmentation performance under deployment variations without additional annotations.

Method

Overview of the geometry-constrained test-time adaptation framework
Geometry-constrained test-time adaptation. Observability or is computed from beam-termination and neighborhood support (A), modulates prompt-induced residual updates within a frozen backbone (B), and regularizes adaptation through temporally smoothed prototype alignment (C). Only the lightweight prompt adapters receive gradients — the backbone stays frozen, so adaptation costs 0.72% of the model parameters.

Results

Qualitative comparison on SemanticKITTI and nuScenes
Qualitative comparison on SemanticKITTI (Rows 1–3) and nuScenes (Rows 4–6). From left to right: RGB image, SFCNet, FRNet, ours, and ground truth. Our method produces cleaner planar regions, less fragmentation on thin structures, and more coherent boundaries. Blue and Red boxes highlight representative differences.
Visualization of explicit observability modeling
Visualization of explicit observability modeling. (A) Observability map from two viewpoints, encoding per-cell geometric support. (B) Ground truth. (C) Without observability modeling, weakly supported regions yield fragmented predictions. (D) Our method, driven by geometrically supported cells, recovers a coherent pedestrian structure.

BibTeX

@article{jiang2026noob,
  title={No Adaptation Without Observation: Observability-Constrained Test-Time Prompt Tuning for LiDAR Semantic Segmentation},
  author={Jiang, Linlian and Ju, Wentao and Pinon, Sadman Rakib and Xian, Jianwei and Chi, Zhixiang and Zuo, Xinxin and Wang, Yang},
  journal={arXiv preprint arXiv:2606.30937},
  year={2026}
}