Summary
This paper aims to improve vectorized HD map construction for autonomous driving. Inspired by the global feature association in traditional offline HD mapping, the proposed MapUnveiler processes input frames in a clip-based manner and hopes to resolve occlusions using information from previous frames. Built up MapTRv2, MapUnveiler introduces clip tokens together with the Inter-clip and Intra-clip Unveiler modules to update the map queries with temporal information. Experiments on nuScenes and Argoverse2 datasets demonstrate the superior performance of the proposed method, especially on highly-occluded scenes.
Strengths
1. The idea of incorporating and aggregating clip-level information for online vectorized HD mapping is reasonable and is more akin to how humans drive. The proposed method has more thoughtful designs than early works such as StreamMapNet to better handle occlusions and incorporate long-range information.
2. The proposed MapUnveiler obtains state-of-the-art results in various experimental settings. The improvements over previous methods are especially prominent in the large 100mx50m setting and the highly-occluded scenes collected by the authors.
3. Extensive ablation studies enumerate the choices of almost all hyper-parameters or model components, which helps better understand and break down each element's contributions.
Weaknesses
1. The clarity of the method description is poor, making it very hard to thoroughly understand the proposed architecture. Details are discussed below:
- The method explanation is not self-contained: i) The Inter-clip Unveiler section refers to the TTM and directly skips all details. There is no information at all about how is the compact memory token generated from the denser map queries; ii) The "loss" section refers to MapTRv2 and again skips all details. The authors should not assume the general audience to be aware of the concrete details of TTM and MapTRv2. The core formulation of these components should be elaborated with texts or equations, while full details can go to the appendix.
- The definitions of the temporal window T and the stride S are unclear. Based on the text descriptions and the common definition of stride, my understanding of "T=3 and S =2" is that "each clip has 3 frames, and every two consecutive frames have a temporal gap of 1." However, the symbols in L177-178 seem to suggest other meanings of T and S.
- The description of the inference mechanism is also vague. Is the MapUnveiler executed per frame or per clip? Figure 2 seems to suggest the per-clip inference where the predictions of T frames are obtained together. If this is the case, does it hurt the actual response frequency?
In short, Section 3 of the paper lacks significant details, and I cannot properly understand MapUnveiler's exact formulation. Given that the authors answer "No" to Question 5 of the Checklist, I have to raise concerns about the paper's reproducibility.
2. There is no detail on how the pre-training and fine-tuning are conducted. Do you initialize the MapNet by training MapTRv2? If this is the case, how are the training epochs split for the MapNet pre-training and the end-to-end MapUnveiler fine-tuning? If the 24/6 epochs for nuScenes/Argo2 are only for the fine-tuning stage, then the comparisons in the main table are unfair, as other methods in the table have not fully converged.
3. The main comparison results are incomplete. Most previous papers provide the nuScenes results of both short and long training schedules, but the main table only presents short-schedule results. Considering the last question about the pre-training and fine-tuning, the authors should complement the table with long-schedule results to show that MapUnveiler can obtain consistent performance boosts when all the methods are fully converged. This concern is backed up by the fact that MapUnveiler's improvement is much smaller on Argo2 compared to nuScenes -- based on my empirical experience, previous methods like MapTRv2 and its followups converge faster on Argo2, and training for 6 epochs is close to convergence. This probably suggests that the large performance gaps on nuScenes come from unfair training settings.
4. Your interpretation of StreamMapNet and SQD-MapNet's Argo2 training epochs is wrong. These two methods employ a different frame sampling strategy at training time compared to MapTRv2, but their effective number of training samples is the same as MapTRv2. Therefore, the claim about the "longer training schedules" in the main table's caption is misleading.
5. The venues in the main table are not accurate. HIMap[49] and MGMap[24] are accepted by CVPR2024, and the information was already available at the time of NeurIPS submission. Furthermore, a recent HD map construction method, MapTracker[A], also studies temporal modeling and should be very relevant, but it is missing in the discussion and related works.
[A] MapTracker: Tracking with Strided Memory Fusion for Consistent Vector HD Mapping, arXiv:2403.15951
Questions
The paper studies an important problem (temporal information association) in online HD map construction and proposes a reasonable method. However, the poor clarity and the potentially incomplete/unfair comparison results raise serious concerns about the paper's quality and reproducibility. My current rating is reject, and I will consider changing the score if the main weaknesses are properly addressed.