Streaming Long Video Understanding with Large Language Models

This paper presents VideoStreaming, an advanced vision-language large model (VLLM) for video understanding, that capably understands arbitrary-length video with a constant number of video tokens streamingly encoded and adaptively selected. The challenge of video understanding in the vision language area mainly lies in the significant computational burden caused by the great number of tokens extracted from long videos. Previous works rely on sparse sampling or frame compression to reduce tokens. However, such approaches either disregard temporal information in a long time span or sacrifice spatial details, resulting in flawed compression. To address these limitations, our VideoStreaming has two core designs: Memory-Propagated Streaming Encoding and Adaptive Memory Selection. The Memory-Propagated Streaming Encoding architecture segments long videos into short clips and sequentially encodes each clip with a propagated memory. In each iteration, we utilize the encoded results of the preceding clip as historical memory, which is integrated with the current clip to distill a condensed representation that encapsulates the video content up to the current timestamp. After the encoding process, the Adaptive Memory Selection strategy selects a constant number of question-related memories from all the historical memories and feeds them into the LLM to generate informative responses. The question-related selection reduces redundancy within the memories, enabling efficient and precise video understanding. Meanwhile, the disentangled video extraction and reasoning design allows the LLM to answer different questions about a video by directly selecting corresponding memories, without the need to encode the whole video for each question. Our model achieves superior performance and higher efficiency on long video benchmarks, showcasing precise temporal comprehension for detailed question answering.

Paper

References (99)

Scroll for more · 38 remaining

Similar papers

Peer review

Reviewer sqPz6/10 · confidence 4/52024-07-12

Summary

This paper proposes VideoStreaming to understand arbitrary-length videos by streamingly encoding video tokens and adaptively selecting a constant number of them. Through careful designs, including a small LLM streaming encoder, prefix task, modified attention, memory propagation, and Gumbel-Topk token selection, the method successfully achieves superior performance on long video benchmarks, showcasing precise temporal comprehension for detailed question answering.

Strengths

1. This paper has a strong motivation, aiming to solve the significant computational burden caused by the large number of tokens extracted from long videos. This is an important issue for LMM to understand long videos. 2. The methodology is novel. First, the paper utilizes the causal LLM to satisfy the streaming and query-related vision summarization with explicit timestamp cues. Second, the paper designs a prefix task that naturally compresses the vision tokens during auto-regressive training. Lastly, through memory propagation and adaptive selection, the model successfully uses only the contextually related tokens to forward to the LLM, thus reducing the computational and memory cost due to the long kv cache. 3. The paper is well written and easy to follow.

Weaknesses

My main concerns are as follows: A. In common short/medium video length benchmarks, VideoStreaming could be unnecessary; B. For long video length benchmarks, the paper does not provide enough experiments; C. Promising experiments, like real-time interaction, are completely ignored in this paper. Specifically, A.1. This paper actually uses much more data during training compared to LLaVA-Next-Video: - LLaVA-Next-Video-7B: 558K for MLP + 860K for MLP & LLM - VideoStreaming-8.3B: 790K for MLP + 763K for small LLM + 345K for all VideoStreaming is trained with 480K more data, which cannot be ignored. However, note that LLaVA-Next-Video-7B has slightly better results on VideoChat-GPT and even employs a much simpler pooling strategy to encode video tokens. Given that VideoChat-GPT's average duration is 1.81 minutes (see Table 1), we can infer that when the short video is less than 2 minutes, VideoStreaming may not offer advantages. A.2. For medium-length videos, such as EgoSchema (average 3 minutes), the paper seems to have missed the results of VideoChat2 mistral-7B, which shows an accuracy of 54.4%, much higher than the 44.1% reported in the paper. Although it is not entirely fair since VideoChat2 mistral-7B uses even more training data, its model employs a simple Q-former with 16 segments and 16 frames to handle the video. Therefore, I also suspect that VideoStreaming may not present advantages in medium-length videos. Here I have omitted the discussion of MovieChat-1K since its question setting does not really relate well to minute-long videos. B. Therefore, where VideoStreaming might truly excel is in handling very long videos. However, it is very disappointing that the paper does not present enough experiments on this. The ablations have been done on medium-length videos, and the authors do not provide any competitive baselines (with the same training data and procedure) in Table 7. C. In fact, VideoStreaming completely ignores a promising application, which is long-term, real-time streaming QA, such as GPT-4o (concept). The method can significantly reduce the kv cache consumption during long video streaming and maintain high-speed response even if a long sequence has been input. D. The demo shown in Figure 5(b) can be misleading. If you try the text-only Vicuna 13b (https://chat.lmsys.org/) and ask, "What challenges does Philippe Petit face in his quest to walk between the Twin Towers?" it can produce even better responses, such as "getting access to the top of the Twin Towers," "creating the perfect wire," "practicing the walk," and "keeping the walk a secret." This suggests that the question you provided may only prompt the LLM to recall its text pretraining data. I also doubt that the model can precisely select the relevant clips in a 2-hour movie, as that would be an extraordinary feat even for GPT-4o or Gemini-1.5-Pro. Despite the above weaknesses, I still acknowledge the novelty of the methodology. I will decide on my final rating after reading the rebuttal.

Questions

1. Would it be possible to use the previous layers of Vicuna to replace the extra introduced (half) Phi-2, which has 1.3B parameters that cannot be ignored? 2. Should L127-L129 be correct? StreamingLLM [a] has demonstrated the importance of initial tokens. I think it is your designed prefix task that restricts the streaming encoder from achieving this. 3. The method still needs to segment clips. Why not directly operate on the frames with a moderate FPS (e.g., 2)? This should make the method more general. Or will this also incur a huge memory cost for the streaming encoder since it is still at the billion-level? 4. The ablation in Table 8 is hard for me to understand. First, what is the performance if there is no memory and no selection? Second, the performance variance in MovieChat is so huge. Are these results from different trained models, or just the same trained model but different inference? [a] Efficient Streaming Language Models with Attention Sinks, ICLR 2024.

Rating

6

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

The limitations are discussed in the supplementary material: (1) simply uniformly sampling frames to form a set of short clips for memory-propagated streaming encoding rather than using adaptive segmentation; (2) the number of tokens may not be sufficient to represent clips with abundant visual content and intensive temporal dynamics. These are indeed limitations and are well discussed. However, as mentioned in the 'weakness' section, most limitations pertain to the experimental setting.

Authorsrebuttal2024-08-12

Additional Response to Reviewer sqPz

Dear Reviewer sqPz, We sincerely thank you again for your great efforts in reviewing this paper, especially for the precious advice that has helped us improve the quality of this paper significantly! In response to your suggestions, we have supplemented the experiments and discussions in the rebuttal. If there are any further clarifications or additional experiments you would like us to address, please do not hesitate to let us know. Best, Paper 6273 Authors.

Reviewer sqPz2024-08-12

Many thanks for the detailed response

> A1: ... The response somehow works for me. Let's focus more on the long-form videos. However, I do think that in a future version, you should provide an ablation study on short-term videos to show whether your method decreases performance compared to the simple pooling strategy. > A2 & A3: ... Nice & important results. Many thanks for that. > A4: ... The recommendation is to validate on some online/streaming action detection benchmarks, or follow Streaming Vid2Seq [a]/VideoLLM-online [b] on CVPR 2024. [a] Streaming Dense Video Captioning. arXiv: 2404.01297. [b] VideoLLM-online: Online Video Large Language Model for Streaming Video. arXiv: 2406.11816. > A5: ... Now the example is less misleading. Thanks for that. However, I wonder if there are instances, such as a movie title at the beginning frame, where knowledge could still be leaked. Could you provide non-movie examples? If the rebuttal cannot provide an anonymous link, please include them in a future version. > A6: ... This does not convince me. I suspect the grounding is cherry-picked and there are many false positives and true negatives. However, I understand that this is what the demo needs. > A7 & A9: ... Thanks for that. The results are consistent with my estimate. Overall, I appreciate the hard work of the authors. Most of my concerns have been addressed, and I plan to increase my rating score.

Authorsrebuttal2024-08-13

Dear Reviewer sqPz: We are delighted to hear that most of your concerns are addressed! We will include the ablation study on short-term videos, the experiments on online/streaming benchmarks, and non-movie examples in the revised version. Regarding your concerns in A5 and A6, there is no movie title at the beginning frame, only the visual tokens and question tokens are input to the LLM to produce the answer. For grounding, there exist reasonable alternatives to be selected in the movie, but the propagated memory allows the model to select a subset to answer the questions. We will present more detailed grounding results in the revised version. Many thanks again for your very constructive comments, which have helped us improve the quality of the paper significantly. Best, Paper 6273 Authors.

Reviewer 9Ye56/10 · confidence 4/52024-07-13

Summary

This paper introduces VideoStreaming, a Vision-Language Large Model (VLLM) designed for comprehensive video understanding. It effectively manages the challenges of computational overhead associated with processing long videos by innovatively using a constant number of video tokens. VideoStreaming demonstrates superior performance and efficiency on long video benchmarks, excelling in detailed question answering with accurate temporal comprehension.

Strengths

1.The Memory-Propagated Streaming Encoding technique effectively captures long-term video content while maintaining a manageable computational footprint by propagating memory through sequential segments. 2. Adaptive Memory Selection strategically reduces the number of tokens used in processing, selecting only the most relevant memories for generating responses, which enhances the model's efficiency and precision. 3. Demonstrated superior performance on long video benchmarks indicates the model's capability to handle complex video understanding tasks effectively, setting it apart from existing models.

Weaknesses

1. The core strategy still relies on processing video through key clips, which missed some relevant works, such as "LGDN: Language-guided denoising network for video-language modeling" (NeurIPS 2022). 2. The method for selecting key clips could potentially lead to redundancy, especially if multiple similar clips have high relevance scores. Integrating a regularization function to promote diversity in key clip selection could help mitigate this issue. 3. In scenarios involving multiple queries about a video, the model may need to repeatedly extract key clips for each question, potentially affecting efficiency. The paper could benefit from a discussion on optimizations that reduce the computational overhead in such use cases.

Questions

Please see weaknesses. I will raise my score if the author well address my concerns.

Rating

6

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

None.

Authorsrebuttal2024-08-12

Additional Response to Reviewer 9Ye5

Dear Reviewer 9Ye5, We sincerely thank you again for your great efforts in reviewing this paper, especially for the precious advice that has helped us improve the quality of this paper significantly! In response to your suggestions, we have supplemented the discussions and clarifications in the rebuttal. If there are any further clarifications or additional experiments you would like us to address, please do not hesitate to let us know. Best, Paper 6273 Authors.

Reviewer 9Ye52024-08-13

Thank you for the detailed responses. My concerns have been addressed, and I would like to raise my score.

Authorsrebuttal2024-08-13

Dear Reviewer 9Ye5: We are delighted to hear that your concerns are addressed! Many thanks again for your very constructive comments, which have helped us improve the quality of the paper significantly. Best, Paper 6273 Authors.

Reviewer QjSL5/10 · confidence 4/52024-07-15

Summary

This paper presents an advanced vision-language model designed to handle the complexities of understanding arbitrary-length videos. The model addresses the computational challenges posed by long video sequences by implementing two core techniques: Memory-Propagated Streaming Encoding and Adaptive Memory Selection. These innovations enable the model to efficiently encode long videos into a constant number of video tokens, preserving both spatial and temporal dynamics. Extensive experiments demonstrate that VideoStreaming achieves superior performance and higher efficiency on long video benchmarks, showcasing its ability to provide precise temporal comprehension for detailed question answering. The contributions include a novel approach to long video understanding, efficient encoding and memory selection strategies, and comprehensive evaluations that highlight the model’s capabilities and advantages over existing methods.

Strengths

- The paper addresses a significant and challenging research problem: long video understanding. - The proposed strategy is technically sound. - The paper conducts sufficient experiments on varied benchmarks, utilizing six video datasets of different durations. - The use of an additional small encoder, such as Phi-2, as a streaming encoder is an interesting approach.

Weaknesses

- Summarization Tokens: The use of summarization tokens is unclear, as it avoids attention between the TT tokens and the clip TN tokens. This design choice needs better justification and clarification. - Small Language Model for Streaming Encoder: While the use of a small language model for the streaming encoder is interesting, there are no experiments demonstrating the necessity of language reasoning for visual-dominant dependencies (mainly visual embed. outputs). Additionally, the model is still quite large (2.7 billion parameters). A more efficient alternative, such as a shadow layer transformer, should be considered. - Paper Writing: The proposed technical strategy is difficult to understand and lacks readability. It would benefit from clearer explanations and more detailed descriptions. - Memory-Propagated Strategy: The memory-propagated strategy is similar to previous strategies in video understanding, such as those in [1]. The paper should differentiate its approach more clearly. [1] Memory-Augmented Dense Predictive Coding for Video Representation Learning - Ablation Studies: The ablation studies are not comprehensive, with only Table 8 presented. There is a lack of convincing studies on several designs, such as summarization tokens. More detailed ablation studies are needed to validate the design choices. - Baseline Comparisons: In the main tables, the paper does not include recent instruction-tuning baselines such as LLaMA-VID. Many baselines, like LangRepo and LLoVi, are mainly modular methods based on image captioners like LLaVA, making the comparisons unfair. Including more relevant and recent baselines would provide a clearer picture of the paper's contributions. - Progressive Training: The progressive training strategy is not innovative and can be found in previous works [1, 2]. The paper should highlight any unique aspects of its approach. [1] LLaMA-VID [2] A Simple Recipe for Contrastively Pre-training Video-first Encoders Beyond 16 Frames

Questions

- Performance on Egoschema: I am very curious about the performance on the Egoschema dataset. As it is an egocentric video dataset, and since the pretraining dataset does not include egocentric videos, how does the model achieve a significant zero-shot performance (44.1) comparable to the finetuned MC-ViT-L baseline (44.4)? Could you provide more insights or experiments that explain this result? - Visualization in Figure 5(b): The accurate grounding along a video exceeding one hour is impressive. However, I wonder if the LLM already recognizes specific information from the query, such as "Philippe Petit" and "Twin Towers." Additionally, the grounding intervals and the text outputs do not seem to have solid associations. Could you clarify how the grounding intervals are determined and ensure that they are directly linked to the text outputs? Are there any qualitative or quantitative measures used to validate these associations?

Rating

5

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

Please refer to the weakness & questions section.

Reviewer FUM46/10 · confidence 3/52024-07-16

Summary

In this paper, a vision-language model, named VideoStreaming, is proposed. It adopts the memory based architecture to understand long video. Specifically, it segments a long video into multiple short clips and encodes each clip sequentially. During this encoding process, it exploits the memory feature to propagate the information from previous clips. Also, it develops the memory selection strategy to find out question-related memory among all encoded memories. Extensive experimental results on various VQA datasets show that the proposed algorithm achieves better performances than exisiting methods.

Strengths

- The paper is well-written and easy to understand - The motivation is reasonable. Also, the proposed algorithm technically sounds. - The proposed algorithm achieves

Weaknesses

I could not find the critical flaws in this work. Even though the motivation is not very new in long video understanding, the proposed memory-based framework is reasonable and well-designed. - It would be better to provide the comparsion on IntentQA since it is also widely used for LVQA. - In L201, the authors said 'we develop a variant of Gumbel-Softmax, denoted as Gumbel-TopK(),...' However, there is no detailed description on Gumbel-TopK in the paper and Gumbel-TopK trick is already widely used. Thus, I would recommend using other expression, instead of 'develop.' Otherwise, it would be better to explain Gumbel Top-k in detail.

Questions

In overall, I think that the motivation of the paper is reasonable. Also, the proposed memory propagating encoding framework and memory selection method have some contribution to the field. Therefore, I'm positive about this work. Please find my concerns in the weakness section.

Rating

6

Confidence

3

Soundness

2

Presentation

3

Contribution

2

Limitations

Yes, L574-581

Authorsrebuttal2024-08-12

Additional Response to Reviewer FUM4

Dear Reviewer FUM4, We sincerely thank you again for your great efforts in reviewing this paper, especially for the precious advice that has helped us improve the quality of this paper significantly! In response to your suggestions, we have supplemented the experiments and explanations in the rebuttal. If there are any further clarifications or additional experiments you would like us to address, please do not hesitate to let us know. Best, Paper 6273 Authors.

Reviewer FUM42024-08-14

I would like thank the authors for their effort for rebuttal. I have read the response from authors as well as the reviews from other reviewers. The rebuttal has resolved most of my concerns. Therefore, I decided to keep my original rating.

Authorsrebuttal2024-08-14

Dear Reviewer FUM4: We are delighted to hear that your concerns are addressed! Many thanks again for your very constructive comments, which have helped us improve the quality of the paper significantly. We will include the experiments and clarifications in the revised version. Best, Paper 6273 Authors.

Reviewer QjSL2024-08-12

Post-response

I would like to thank the authors for their hard work and appreciate the honest acknowledgment and significant updates during the rebuttal phase. which have mainly addressed my concerns. Additionally, I suggest the authors to include more discussion on recent works related to long video streaming optimization in their revision, such as [1], which maintaining long context with real-time efficiency. Moreover, recent benchmarks like [2] might offer a better evaluation choice for long videos. [1] Chen, Joya, et al. "VideoLLM-online: Online Video Large Language Model for Streaming Video." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024. [2] Fu, Chaoyou, et al. "Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis." arXiv preprint arXiv:2405.21075 (2024). Considering these points, I would like to raise my rating to borderline accept.

Authorsrebuttal2024-08-12

Dear Reviewer QjSL: We are delighted to hear that your concerns are addressed! Many thanks again for your very constructive comments, which have helped us improve the quality of the paper significantly. We will include detailed discussions with recent works like [1] and incorporate the results on more recent benchmarks like [2]. Best, Paper 6273 Authors.

Program Chairsdecision2024-09-25

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC