4D Panoptic Scene Graph Generation

We are living in a three-dimensional space while moving forward through a fourth dimension: time. To allow artificial intelligence to develop a comprehensive understanding of such a 4D environment, we introduce 4D Panoptic Scene Graph (PSG-4D), a new representation that bridges the raw visual data perceived in a dynamic 4D world and high-level visual understanding. Specifically, PSG-4D abstracts rich 4D sensory data into nodes, which represent entities with precise location and status information, and edges, which capture the temporal relations. To facilitate research in this new area, we build a richly annotated PSG-4D dataset consisting of 3K RGB-D videos with a total of 1M frames, each of which is labeled with 4D panoptic segmentation masks as well as fine-grained, dynamic scene graphs. To solve PSG-4D, we propose PSG4DFormer, a Transformer-based model that can predict panoptic segmentation masks, track masks along the time axis, and generate the corresponding scene graphs via a relation component. Extensive experiments on the new dataset show that our method can serve as a strong baseline for future research on PSG-4D. In the end, we provide a real-world application example to demonstrate how we can achieve dynamic scene understanding by integrating a large language model into our PSG-4D system.

Paper

Similar papers

Peer review

Reviewer 4B6f7/10 · confidence 4/52023-07-05

Summary

This paper offers a novel perspective on understanding a 4D environment via the introduction of 4D Panoptic Scene Graph (PSG-4D). PSG-4D transforms 4D sensory data into nodes, representing entities with precise location and status data, and edges that capture temporal relationships. The authors have developed a high-quality PSG-4D dataset, which includes 3K RGB-D videos with a total of one million frames. Each frame comes with 4D panoptic segmentation masks and detailed, dynamic scene graphs. In order to solve PSG-4D, they propose PSG4DFormer, a model capable of predicting panoptic segmentation masks, tracking masks across the time axis, and generating corresponding scene graphs via a relation component. Extensive experiments demonstrate that their method forms a robust baseline for PSG-4D. Furthermore, the paper provides a real-world application example demonstrating how dynamic scene understanding can be achieved by incorporating a large language model into the PSG-4D system.

Strengths

1. This paper is significant for several reasons: it formulates a new problem, annotates a novel dataset, and provides simple baselines. These aspects could advance future research and encourage applications in robotic systems. Even though the research appears to be a logical progression from panoptic scene graph generation and panoptic video scene graph generation, the introduction of a new data modality provides potential for a broader range of applications. SGG in 4D space is an area that has not been extensively explored, to my knowledge. 2. The dataset selection and construction pipeline is detailed and appears to be reasonable. 3. The paper is well articulated and easy to comprehend. The incorporation of the large language model (LLM) in the proposed application seems to be an intriguing and viable addition.

Weaknesses

1. I have a significant concern with the Experimental section of the paper. The task of Scene Graph Generation is intricate, particularly when extended to a 4D scenario. In building a pipeline to solve this problem, it is reasonable to use various components each serving a unique role. However, it remains unclear which component holds the most promise for future research focus. While the ablation study in Table 2 gives us insights into the depth map data and naïve temporal modeling, it would be valuable for the authors to provide more detailed analysis and insights. 2. In Line 112, there is a minor formatting error: 'M' should be formatted as an equation rather than plain text.

Questions

Please refer to the weaknesses.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

4 excellent

Contribution

3 good

Limitations

The authors mentioned potential challenges and societal impact.

Authorsrebuttal2023-08-19

Additional Analysis

We sincerely thank the reviewers for their detailed feedback and recognition of our paper's potential impact. We appreciate the positive remarks regarding the significance, novelty, and presentation of our work. Below, we address your concerns: **1. Detailed Analysis in Experimental Section:** Over the past weeks, we've enriched our ablation studies. While Table 2 highlights: - A superior performance with RGB-D input over Point Cloud input. - The beneficial influence of the additional depth channel compared to RGB input alone. - The critical role of temporal attention in our relation model. We aim to provide a more extensive analysis on the PSG4D-GTA dataset encompassing: - **Correlation between Segmentation, Tracking, and SG Performance**: The subsequent table presents intermediate results, showcasing frame-level segmentation results (IOU) and video-level segmentation results that incorporate tracking (vIoU). - IoU evaluates all frames individually using a traditional semantic segmentation protocol. - vIoU assesses video segmentation performance, calculating the highest volume IOU for a GT mask tube. Our findings indicate that: - Improved image segmentation tends to enhance vIoU and SG performance. - The relation recall performance gap is narrower than that of video segmentation. For instance, while Mask2Former significantly boosts video segmentation performance, the subsequent relation modeling stage reduces this gain. This emphasizes the need for improved relation modeling strategies. - We also note that vIoU might not adequately support relation modeling, highlighting a need for community-driven advancements in video segmentation performance. - Considering the limitations of current methods, we contemplate organizing competitions to promote this new task and establish benchmarks. | Method | Input | IoU | vIoU | R/mR@20 | | --- | --- | --- | --- | --- | | DKNet | PC | 54.77 | 27.30 | 4.33 / 2.10 | | Mask2Former | RGB | 62.08 | 30.72 | 4.40 / 1.42 | | Mask2Former | RGBD | 70.23 | 37.22 | 6.68 / 3.31 | - **Class-level Segmentation Analysis**: - Our results display that the DKNet, reliant solely on point clouds, struggles with mean IoU performance, primarily due to challenges in segmenting diverse objects. - Performance disparities are evident for small objects like cups and chairs when solely using point clouds, while larger or structured objects such as persons and potted plants exhibit better segmentation. | Method | Input | mIoU | person | car | potted plant | bottle | cup | chair | door | | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | | DKNet | PC | 13.68 | 93.41 | 51.06 | 80.67 | 5.28 | 1.95 | 22.29 | 2.31 | | Mask2Former | RGB | 40.22 | 95.24 | 57.73 | 77.90 | 43.91 | 45.36 | 59.17 | 52.01 | | Mask2Former | RGBD | 45.14 | 96.39 | 58.45 | 83.21 | 48.35 | 51.04 | 62.23 | 59.14 | - **Class-level Relation Analysis**: - Our analysis indicates that while relations like **`talking to`** and **`looking at`** are effectively recognized since they involve persons (easily segmented objects), challenges arise for relations like **`holding`** and **`opening`**, especially with the DKNet. This pinpoints a need for strategies addressing relations involving smaller objects. - Also, the performance of `talking to` and `looking at` is also far from saturated. | Method | Input | talking to | looking at | holding | opening | | --- | --- | --- | --- | --- | --- | | DKNet | PC | 14.67 | 7.68 | 0.34 | 0 | | Mask2Former | RGB | 18.59 | 9.47 | 2.31 | 0 | | Mask2Former | RGBD | 19.24 | 12.03 | 2.31 | 0 | We trust these in-depth analyses will illuminate promising directions for subsequent research building on our foundation. **2. Formatting Error on Line 112:** We sincerely apologize for this oversight and thank you for bringing it to our attention. The formatting error will be rectified in the final manuscript. Once again, we're grateful for your constructive feedback and the time invested in reviewing our work.

Area Chair YhZH2023-08-21

Can you please check the rebuttal comments?

Dear reviewer, The authors have provided a response to your comments. Can you please take a look and accordingly comment, and updated your review? Thanks, -Area Chair

Reviewer z3XZ5/10 · confidence 4/52023-07-05

Summary

This paper introduces the concept of the 4D Panoptic Scene Graph (4D-PSG) as an approach for achieving a high-level visual understanding of 4D scenes. Additionally, the authors have developed two datasets annotated with 4D-PSG information. Furthermore, the authors propose a PSG4DFormer method for constructing the 4D-PSG from video data. The utilization of the 4D-PSG has the potential to enhance a robot's ability to comprehend its environment and appropriately respond to it.

Strengths

1. This paper is well written and easy to follow. 2. This paper contributes to the open source community they would release dataset, codebase and robot module.

Weaknesses

1. The novelty of this work is limited. The 4D-PSG is a simple extension from the existing 3DSG. The proposed PSG4DFormer is also a simple combination of the 4D Panoptic Segmentation Model and the Relation Model. 2. Although the authors mentioned the potential application of the 4D-PSG, no experiments are provided to substantiate this claim. The authors really need to add these experiments. Also, a strong baseline should be provided for comparison.

Questions

None

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

4 excellent

Contribution

3 good

Limitations

The authors have discussed the societal impact.

Area Chair YhZH2023-08-21

Can you please check the rebuttal comments?

Dear reviewer, The authors have provided a response to your comments. Can you please take a look and accordingly comment, and updated your review? Thanks, -Area Chair

Reviewer CrvQ5/10 · confidence 2/52023-07-07

Summary

The author introduces a new task called the 4D Panoptic Scene Graph (PSG-4D), which focuses on predicting 4D panoptic scene graphs from RGB-D video sequences. To support this task, the author provides a unique PSG-4D dataset, encompassing diverse viewpoints through two subsets: a third-view synthetic subset (PSG4D-GTA) and an egocentric real-world subset (PSG4D-HOI), each containing detailed frame labels To address the PSG-4D task, the author proposes a unified two-stage model, the PSG4DFormer, which comprises a feature extractor and a relation learner.

Strengths

- The paper is generally well written and easy to follow. - The new task/datasets seem novel, and well supported by the proposed PSG4DFormer framework.

Weaknesses

- Less relevant to NeurIPS. Perhaps venues like CVPR/ICCV/SIGGRAPH would be a better fit? - While I recognize the dataset is one the major contribution here, the novelty of this work is not entirely clearly to me. - Justification for the choice of dataset composition described in Section 4 seems insufficient. Why we choose this specific composition of dataset beyond convenience? Do we know model trained/evaluated on this dataset can generalize nicely to other use cases (or a specific vertical of applications?) - The experiments seem to be rather limited as only one baseline, though recent and relevant, is evaluated on this very proposed dataset. Are there smaller, ad-hoc dataset that can be used to evaluate the proposed framework to support its external validity?

Questions

See those raised in Weaknesses.

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

4 excellent

Contribution

3 good

Limitations

Yes, though I’d still would like to hear from the author about the generalizability (even for a specific domain) and external validity of the proposed dataset and framework.

Reviewer 2QCE8/10 · confidence 4/52023-07-07

Summary

This paper introduces a new task of scene graph generation that predicts 4D panoptic scene graphs from RGB-D video sequences. This paper also provides a high-quality PSG-4D dataset including PSG4D-GTA with third-view synthetic data and PSG4D-HOI with egocentric real-world data. As the third contribution, this paper also proposes a unified two-stage model to showcase real-world scenarios.

Strengths

1. The new task introduced by this paper is very important. It will help to enable embodied AI to better understand the 4D dynamic real world with complex object-object relations. 2. As illustrated in Tab. 1, the collected dataset features dynamic scenes with much richer data type than previous datasets, covering both egocentric and third-view data. If released, this new dataset will greatly benefit the community. 3. In section 7, the paper also illustrates how to deploy the PSG-4D model in a real-world application by interacting with large language models. This shows the potential of exploiting the introduced dataset to solve practical problems in the real world.

Weaknesses

1. The introduced datasets are based on existing datasets (ie, SAIL-VOS 3D and HOI4D datasets). Therefore, they only cover synthetic data for third-person views. This may cause the imbalanced development of algorithms for egocentric and third-view data. 2. L187-190 briefly mentions how the relation annotations are first automatically prepared and then reviewed and revised. Could the authors elaborate and share a bit more whether more design choices have been made to make the process more efficient or this is a very tedious/time-consuming task (if so, what is the average/total time needed for each scene or all scenes)?

Questions

It may be a bit easier for readers to digest the table if the numbers from the best-performing model can be highlighted.

Rating

8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

4 excellent

Limitations

The authors have adequately addressed the limitations and broader impacts.

Reviewer PY8p7/10 · confidence 5/52023-07-10

Summary

This paper proposes a new task, named 4D panoptic scene graph generation (PSG-4D). At the same time, a dataset, including PSG4D-GTA and PSG4D-HOI, is collected for this task, and a benchmark is provided to evaluate the performance of this task. PSG-4D offers a more comprehensive scene understanding, bringing more thought and possibilities for the practical implementation of scene understanding. The paper shows the potential application of PSG-4D in Robot Deployed scenarios by combining it with LLM, which is really meaningful.

Strengths

1. A new task, dataset, and benchmark are proposed to solve the PSG-4D problem, which provides a more comprehensive scene understanding in the real world. 2. This paper shows the potential applications of the PSG-4D task, making it appear more practically significant.

Weaknesses

The sample size of the dataset and the number of relationship labels are somewhat limited, and the recall provided by the benchmark is really low, which may limit its practical application.

Questions

Please refer to the weaknesses.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.

Soundness

3 good

Presentation

3 good

Contribution

4 excellent

Limitations

Not apply.

Reviewer CrvQ2023-08-15

I thank the author for their response. However, without seeing the claimed addition (e.g., the Digital Twin dataset) to this dataset, I still find the work having room for improvement in terms of data source diversity and a more comprehensive evaluation on the experimental side. I'll keep my score as is. That said, given the overall positive reviews of other reviewers, I believe this paper has a fair chance of getting accepted, which despite my reservation above, would still be a valuable contribution to the community.

Authorsrebuttal2023-08-19

Thank you for your feedback!

Thank you for your feedback and for taking the time to review our paper. We genuinely appreciate your insights and the constructive nature of your remarks. We are also happy to update our added experimental analysis within the [comment to Reviewer 4B6f](https://openreview.net/forum?id=GRHZiTbDDI&noteId=NZP8e51cJt). Regarding the data source diversity concern, we are committed to enhancing our work in this direction. Apart from the addition of the Digital Twin dataset, would you be kind enough to provide any specific suggestions or pointers that you believe would address this aspect further? We believe your insights will be instrumental in refining our approach and expanding the scope of our evaluation.

Reviewer 2QCE2023-08-17

Thanks for the responses! After reading all reviewers' comments and the responses from the authors, I am leaning towards keeping my original rating.

Reviewer PY8p2023-08-18

The novel task presents a potential direction for completely understanding scenes with scene graphs. However, I still have concerns about the dataset being potentially challenging to follow due to limited scale. Nonetheless, I maintain my acceptance score.

Program Chairsdecision2023-09-21

Decision

Accept (spotlight)

© 2026 NYSGPT2525 LLC