VideoDirectorGPT: Consistent Multi-scene Video Generation via LLM-Guided Planning

Recent text-to-video (T2V) generation methods have seen significant advancements. However, the majority of these works focus on producing short video clips of a single event (i.e., single-scene videos). Meanwhile, recent large language models (LLMs) have demonstrated their capability in generating layouts and programs to control downstream visual modules. This prompts an important question: can we leverage the knowledge embedded in these LLMs for temporally consistent long video generation? In this paper, we propose VideoDirectorGPT, a novel framework for consistent multi-scene video generation that uses the knowledge of LLMs for video content planning and grounded video generation. Specifically, given a single text prompt, we first ask our video planner LLM (GPT-4) to expand it into a 'video plan', which includes the scene descriptions, the entities with their respective layouts, the background for each scene, and consistency groupings of the entities. Next, guided by this video plan, our video generator, named Layout2Vid, has explicit control over spatial layouts and can maintain temporal consistency of entities across multiple scenes, while being trained only with image-level annotations. Our experiments demonstrate that our proposed VideoDirectorGPT framework substantially improves layout and movement control in both single- and multi-scene video generation and can generate multi-scene videos with consistency, while achieving competitive performance with SOTAs in open-domain single-scene T2V generation. Detailed ablation studies, including dynamic adjustment of layout control strength with an LLM and video generation with user-provided images, confirm the effectiveness of each component of our framework and its future potential.

Paper

Similar papers

Reviewer Ycer5/10 · confidence 3/52024-05-09

Summary

This paper presents VideoDirectorGPT, a multi-scene video generation framework focusing on generation consistency, which leverages the knowledge embedded in LLMs for video content planning and grounded video generation. VideoDirectorGPT consists of two stages. In the first stage, given a single text prompt, GPT-4 is employed as a video planner to craft a video plan, a multi-component video script with multiple scenes to guide the downstream video synthesis process. In the second stage, a novel grounded video generation module named Layout2Vid is proposed to render videos based on the generated video plan. Layout2Vid is built upon the off-the-shelf T2V generation model ModelScopeT2V, thus enabling Layout2Vid to be trained solely on layout-annotated images. Experiments demonstrate that VideoDirectorGPT can generate high-quality videos with good consistency.

Rating

5

Confidence

3

Ethics flag

1

Reasons to accept

The idea of using LLMs as a video plan generator is straightforward and technically sound, and empirically works quite well.

Reasons to reject

- VideoDirectorGPT is only compared against ModelScopeT2V. The authors should also benchmark other state-of-the-art baseline alternatives such as Phenaki. - The technical contribution of the paper is limited since the main video generation module, Layout2Vid, is built upon the off-the-shelf T2V generation model ModelScopeT2V. - Only 5 crowd-annotators from AMT were used, which makes the human evaluation results less reliable.

Authorsrebuttal2024-06-04

A Gentle Reminder for Response

We thank the reviewer for their time and effort in reviewing our paper. We hope that our response has addressed all the questions and hope that the reviewer can consider revising the score based on our response. We are also happy to discuss any additional questions. With sincere regards, The authors

Reviewer Ycer2024-06-05

I would like to thank the authors for providing additional qualitative and quantitative results. After reviewing the authors' rebuttal and other reviewers' comments, I am keeping my original score. I have a similar concern with reviewer ts3J about the novelty and technical contribution of the paper. The proposed model is built on top of the off-the-shelf T2V generation model ModelScopeT2V, and the technical contribution is thus limited. Increasing the number of human annotators from 5 to 10 is useful. However, in my opinion, it is still too small a sample size to be reliable.

Authorsrebuttal2024-06-05

Response to Reviewer Ycer's Comment

Thanks for your questions. We provide response and clarification as below: >1. The proposed model is built on top of the off-the-shelf T2V generation model ModelScopeT2V, and the technical contribution is thus limited. - We would like to respectfully disagree with the statement that building on top of T2V generation foundation models could lead to the claim that technical contribution is limited. As an analogy, many LLM researchers usually build their methods on top of existing language models (e.g., Llama) instead of training such foundation models from scratch. Similarly, the training cost of T2V generation models like ModelScopeT2V is too high to be affordable for most of the researchers. In addition, our method is model agnostic/easy to adapt to other video generation backbones. Therefore, we kindly request the reviewer to reconsider this statement. >2. Increasing the number of human evaluators from 5 to 10 is useful. However, in my opinion, it is still too small a sample size to be reliable. - We would like to bring to the reviewer’s attention that using less than 10 or around 10 human evaluators is very common in related works in NLP [1], multi-modal AI [2], diffusion models [3], and LLM+diffusion works [4, 5], and these papers are well received at top conferences and journals such as EMNLP, ICLR, ICCV, and TMLR. These works include ControlNet [3], which won the best paper award in ICCV 2023, and MaRVL [2], the best paper in EMNLP 2021. We hope our clarification is helpful. Please let us know if you have any other questions. Thanks! [1] Cho, Sangwoo, et al. "Toward Unifying Text Segmentation and Long Document Summarization." Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. 2022. [2] Liu, Fangyu, et al. "Visually Grounded Reasoning across Languages and Cultures." Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. 2021. [3] Zhang, Lvmin, Anyi Rao, and Maneesh Agrawala. "Adding conditional control to text-to-image diffusion models." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023. [4] Lian, Long, et al. "LLM-grounded Video Diffusion Models." The Twelfth International Conference on Learning Representations. 2023. [5] Lian, Long, et al. “LLM-grounded Diffusion: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Models.” Transactions on Machine Learning Research (TMLR)

Reviewer tgxZ7/10 · confidence 2/52024-05-11

Summary

The authors proposed a two-stage text-to-video pipeline, VideoDirectorGPT, with two major components: (1) a video planner and (2) a video generator. The video planner takes a video text prompt to generate video plans using GPT4, which also groups the same entities across scenes and generates layout (bounding box) of entities in addition to background and scene description. THe video planner is finetuned from ModelScopeT2V for training temporally consistent entity grounding component. Note that only 13% of total model weights are updated for this component. Experiment results show that the proposed VideoDirectorGPT outperforms ModelScopeT2V in single-scene and multi-scene settings on both automatical and human evaluation. Overall, the authors did a great job evaluating the video generator component.

Rating

7

Confidence

2

Ethics flag

1

Reasons to accept

1. Comprehensive evaluation of generated video generator, including automatic evaluation and human evaluation with ablation studies. 2. Proposed method can take not only textual input but also image + text input. 3. The two-stage pipeline allows users to modify each of the components, such as background or bounding boxes.

Reasons to reject

Not that I can think of. Basically, all of my questions are answered in the appendix.

Questions to authors

Comments/Suggestions: It would be helpful to have an example of full video planner output in the appendix.

Reviewer ts3J4/10 · confidence 5/52024-05-17

Summary

This paper proposes VIDEODIRECTORGPT for video generation. It comprises two stages: the first stage is an LLM planner that generates the scene descriptions, the entities with their respective layouts, the background for each scene, and consistency groupings of the entities with the given prompt; the second stage is a video generator mainly based on the weights of ModelScopeT2V. The authors claim their experiments demonstrate that their proposed framework improves layout and movement control in both single- and multi-scene video generation and can generate multi-scene videos with consistency, while achieving competitive performance with SOTAs in open-domain single-scene T2V generation. Ablation studies confirm the effectiveness of each component of the proposed framework.

Rating

4

Confidence

5

Ethics flag

1

Reasons to accept

1. The paper is well-written and easy to understand. 2. The experiments are sufficient in many aspects, including single-scene and multi-scene video generation, with metrics like FID, FVD, IS, and human evaluations.

Reasons to reject

1. The authors mentioned that "We provide code and generated video examples in the supplementary material," but I did not find where the supplementary material is. Thus, I am concerned about the completeness of this work, as well as the reproducibility and impact of the proposed method due to the lack of related code and models. 2. The application is not novel at all. It mainly relies on GLIGEN and ModelScopeT2V. One image version work is [1]. Thus, this work is a little bit below the bar of CoLM. 3. For a video generation paper, related videos should be appended to demonstrate the real performance. As we all know, some quantitative evaluation metrics are inconsistent with real performance, as many studies have proved. 4. There is a lack of related ablation experiments on different language models and whether to fine-tune them. At an LLM conference, the authors should focus more on the LM itself instead of video generation. [1] LLM-grounded Diffusion: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Models, https://llm-grounded-diffusion.github.io/

Questions to authors

Refer to weaknesses. The authors' response addresses my first reject reasion, which is appreciated. However, after carefully checking the code, I maintain my reservation regarding the heavy reliance on GLIGEN and ModelScopeT2V, which dominates the described method. Moreover, the emphasis on video rather than LM suggests that this work falls short of the expectations set by the CoLM standards.

Reviewer ts3J2024-06-05

After carefully reviewing the authors' response and considering other reviewers' comments, I have decided to maintain my rating at [4. rejection] for the following reasons: 1. **Heavy reliance on GLIGEN and ModelScopeT2V:** Despite the authors' claims of significant contributions, the code indicates a primary dependence on these existing models or methods. 2. **Poor organization:** Many important evaluation and ablation studies are relegated to the supplementary materials, which undermines the persuasiveness of the main sections. Given that supplementary materials will not be published, this is a significant issue. 3. **Misalignment with conference scope:** As a work focused on video generation, it is not well-suited for a language modeling (LM) conference. Although the authors reference many CoLM criteria, visual LMs are different from diffusion model technologies. This paper primarily centers on video generation with diffusion models rather than LMs. It may be more appropriate for the authors to reorganize their work and submit it to a vision-focused conference.

Authorsrebuttal2024-06-05

Response to Reviewer ts3J's Comment

>1. Heavy reliance on GLIGEN and ModelScopeT2V: Despite the authors' claims of significant contributions, the code indicates a primary dependence on these existing models or methods. - As we already clearly elaborated in our previous responses, our method is agnostic to the backbone model. To make our pipeline work for the purpose of experimentation, we need to choose one specific video generation model to build upon (e.g., ModelScopeT2V, the best open-source T2V model at the time), and it's easy to adapt our framework to future video generation backbones. As an analogy, LLM researchers usually build their methods on top of existing language models (e.g., Llama), which also indicates a primary dependence on these models. Thus, we believe it’s unfair to criticize a paper due to the foundation model it uses. >2. Poor organization: Many important evaluation and ablation studies are relegated to the supplementary materials, which undermines the persuasiveness of the main sections. Given that supplementary materials will not be published, this is a significant issue. - Firstly, our paper contains thorough analysis of how to use LLMs to help video generation, so we put the most important results in the main paper, and relatively less important ablation studies in the appendix (similar to all other papers submitted to COLM and all other conferences with 8-10 page limits). If the reviewer believes that it’s worthwhile to bring some ablation studies from the appendix back to the main paper, we are very willing to use the extra page given in the camera-ready version for this purpose. - Secondly, **we have emailed COLM PCs last night, and they told us to share the clear confirmation that the supplementary material will be made public along with the paper.** Considering all these aspects, we strongly believe that this should not be a “significant issue”. >3. Misalignment with conference scope: As a work focused on video generation, it is not well-suited for a language modeling (LM) conference. Although the authors reference many CoLM criteria, visual LMs are different from diffusion model technologies. This paper primarily centers on video generation with diffusion models rather than LMs. It may be more appropriate for the authors to reorganize their work and submit it to a vision-focused conference. - As we already mentioned in our previous responses, our focus is on how to use LLMs to help the video generation task. Using visual LMs could be one technical design choice, and using LLMs+diffusion could be another design choice. We chose our approach because diffusion models are the most popular framework for modern image/video generation currently. Given that both methods are under the topic of using LLMs to help video generation, we strongly believe that the specific technical design choices to tackle this problem should not be the key point to distinguish if a paper should be incorporated into COLM.

Reviewer HYoe6/10 · confidence 3/52024-05-20

Summary

This papar propose video-director-GPT, a framework to generate consistent multi-scene videos. The method includes a GPT-4 planning process and a layout2vid module to generate video based on the plans. The experiments show that the quality is improved.

Rating

6

Confidence

3

Ethics flag

1

Reasons to accept

1. the idea and the method is clear and well-motivated 2. the evalution is strong, including automatic and annoatation evaluations.

Reasons to reject

1. the experiments are somehow limited, I understand the fact that running LLM servers might be difficult, still, only using GPT-4 as the planner without any other alternative LLMs might weaken the results. 2. If I'm not mistaking any details, the GPT-4 generated plans are later grouped into a shared representation based on exact match. Is this a method that weakens the power of LLMs? The reasoning, continuous semantics are all discarded then?

Questions to authors

Please see the comments above.

Authorsrebuttal2024-06-04

A Gentle Reminder for Response

We thank the reviewer for their time and effort in reviewing our paper. We hope that our response has addressed all the questions and hope that the reviewer can consider revising the score based on our response. We are also happy to discuss any additional questions. With sincere regards, The authors

Reviewer HYoe2024-06-05

I'm curious what is the random accuracy in ActionBench-directions? I'm not sure if llama models can handle such task? I believe I've asked this question above

Authorsrebuttal2024-06-05

Response to Reviewer HYoe's Comment

Dear reviewer HYoe, thanks for your questions. We provide response and clarification as below: >I'm not sure if llama models can handle such task? - Thanks, we would like to bring your attention to our previous response to W1 (https://openreview.net/forum?id=sKNIjS2brr&noteId=UaqWRftSIi), we have shown the comparison between LLaMA2, GPT-3.5-turbo, and GPT-4 in Table 9 of Section H.1. - From the result in Table 9, we can see that fine-tuned LLaMA2-13B performs in-between GPT-3.5-turbo and GPT-4 (GPT-3.5-turbo 49.0 v.s. LLaMA2-13B 53.3 v.s. GPT-4 59.8). >I'm curious what is the random accuracy in ActionBench-directions? - As described in the paragraph “ActionBench-Direction prompts” on page 20 of our paper, for each object, we give prompts that include four movement directions: ‘left to right’, ‘right to left’, ‘top to bottom’, and ‘bottom to top’. Therefore, for a random guess, the movement accuracy on the ActionBench-directions dataset should be around 25%. We hope our clarification is helpful. Please let us know if any part of our response is still unclear to you. Thanks!

Reviewer HYoe2024-06-07

Thanks for the answer.

Reviewer TCNG6/10 · confidence 5/52024-05-23

Summary

This paper introduces VideoDirectorGPT, which leverages large language models (LLMs) to derive object/layout plans for multi-scene long video generation. Specifically, the video planner expands the input caption into scene descriptions, including entities with bounding boxes and the overall background. Then, the proposed Layout2Vid can control explicit spatial layout and temporal coherence during video generation.

Rating

6

Confidence

5

Ethics flag

1

Reasons to accept

- This paper is well-written and easy to follow. - The idea of adopting LLMs to derive scene description/layout is well-motivated, which has the potential to benefit the planning of long video generation. - They provide detailed ablation studies and comprehensive visualized examples in the Appendix.

Reasons to reject

- Since there are several previous works [1,2] adopting LLMs to improve the layout planning of T2I, the novelty of this project can be the issue. Furthermore, the proposed Layout2Vid is an extension of the existing GLIGEN [3]. I am afraid that this project can not achieve the bar of a top-tier conference. - Though the proposed video planner can craft the plan, Layout2Vid only generates 16 frames (2-4 seconds) as the output video. Can this short clip actually contain multiple scenes (e.g., different entities and backgrounds)? I am worried about the practicality of the proposed framework. - Apart from those successful cases, failure examples are also crucial to discuss, which can guide future research on this topic. **Reference** - [1] LayoutGPT: Compositional Visual Planning and Generation with Large Language Models - [2] LLM-grounded Diffusion: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Models - [3] GLIGEN: Open-Set Grounded Text-to-Image Generation

Questions to authors

Please see Reasons To Reject

Authorsrebuttal2024-06-04

A Gentle Reminder for Response

We thank the reviewer for their time and effort in reviewing our paper. We hope that our response has addressed all the questions and hope that the reviewer can consider revising the score based on our response. We are also happy to discuss any additional questions. With sincere regards, The authors

Authorsrebuttal2024-06-04

Response to updated reviewer comment

**Part1 response to updated reviewer comment:** >Reviewer ts3J: "The authors' response addresses my first reject reason, which is appreciated. However, after carefully checking the code, I maintain my reservation regarding the heavy reliance on GLIGEN and ModelScopeT2V, which dominates the described method.” To begin with, we would like to re-state that in addition to the first rejection reason, the third and the fourth rejection reason were already covered in our first rebuttal response. We respectfully disagree with the assertion that our primary contribution relies heavily on GLIGEN and ModelScopeT2V. As we mentioned in the rebuttal, our paper contribution consists of three parts: (1) LLM for video plan generation, (2) Layout-guided video generation with consistency control, as well as (3) Novel evaluation metrics and datasets for measuring layout and consistency accuracy for (multi-scene) video generation, as detailed below: (1) Firstly, we would like to emphasize that a substantial part of our work focuses on introducing the use of LLMs to generate complex, useful plans for multi-scene video generation. We start from the motivation/observation that LLMs (GPT-4) can generate bounding box movements with physical world understanding (Fig. 12), then design a multi-component video plan (Sec. 3.1) via in-context learning (Fig. 7 and Fig. 8). Finally, we conducted thorough analysis of the effect of different LLMs (Table 9), different layout control methods (Table 11), as well as error analysis (Table 8). We believe our paper has covered the analysis of LLMs in a relatively complete way. If you have any other questions or concerns, we are more than willing for open discussion. (2) Secondly, we would like to highlight that our method is agnostic to the backbone model. To make our pipeline work, we need to build on one specific video generation model (e.g., ModelScopeT2V, the best open-source T2V model at the time). However, it's easy to adapt our framework to future video generation backbones. In addition, adopting a gated self-attention module from GLIGEN is only one part of our architecture design. We also made a non-trivial effort to enhance it with text+image embedding conditioning, proposed sharable embedding for multi-scene consistency control, and creatively trained the video generation module in a parameter-efficient way with only images (Sec. 3.2). We also did extensive ablation studies of our model architecture design (Table 4, 7, 10, 12). Therefore, stating that our method "relies heavily on GLIGEN and ModelScopeT2V'' is not well-supported. (3) Thirdly, we provide comprehensive qualitative + quantitative experiments including novel evaluation metrics (movement direction accuracy / entity consistency) / new datasets (ActionBench-Direction/Coref-SV) for video generation, which could be beneficial for future works on layout-guided and multi-scene video generation. ---------------------------- **Part2 response to updated reviewer comment:** >Reviewer ts3J: “Moreover, the emphasis on video rather than LM suggests that this work falls short of the expectations set by the CoLM standards." **We strongly believe that our novel framework leveraging LMs for video generation aligns well with CoLM’s interests.** Specifically, our work addresses the topic of “17. LMs on diverse modalities and novel applications,” as outlined in the CoLM call for papers (https://colmweb.org/cfp.html). ```17. LMs on diverse modalities and novel applications: visual LMs, code LMs, math LMs, and so forth, with extra encouragement for less studied modalities or applications such as chemistry, medicine, education, database, and beyond.``` Moreover, the CoLM homepage (https://colmweb.org/index.html) mentions: ```CoLM is an academic venue focused on the study of language modeling, broadly defined, with the goal of creating a community of researchers with expertise in different disciplines, focused on understanding, improving, and critiquing the development of LM technology.``` **These underline the relevance of our contribution to the CoLM community.** **We kindly ask for a reconsideration of the review and rating in light of these points.**

Program Chairsdecision2024-07-10

Decision

Accept

© 2026 NYSGPT2525 LLC