DDCoT: Duty-Distinct Chain-of-Thought Prompting for Multimodal Reasoning in Language Models

A long-standing goal of AI systems is to perform complex multimodal reasoning like humans. Recently, large language models (LLMs) have made remarkable strides in such multi-step reasoning on the language modality solely by leveraging the chain of thought (CoT) to mimic human thinking. However, the transfer of these advancements to multimodal contexts introduces heightened challenges, including but not limited to the impractical need for labor-intensive annotation and the limitations in terms of flexibility, generalizability, and explainability. To evoke CoT reasoning in multimodality, this work first conducts an in-depth analysis of these challenges posed by multimodality and presents two key insights: "keeping critical thinking" and "letting everyone do their jobs" in multimodal CoT reasoning. Furthermore, this study proposes a novel DDCoT prompting that maintains a critical attitude through negative-space prompting and incorporates multimodality into reasoning by first dividing the reasoning responsibility of LLMs into reasoning and recognition and then integrating the visual recognition capability of visual models into the joint reasoning process. The rationales generated by DDCoT not only improve the reasoning abilities of both large and small language models in zero-shot prompting and fine-tuning learning, significantly outperforming state-of-the-art methods but also exhibit impressive generalizability and explainability.

Paper

References (75)

Scroll for more · 38 remaining

Similar papers

Peer review

Reviewer ndag5/10 · confidence 4/52023-06-12

Summary

This paper explore utilizing the rationales to achieve the multimodal reasoning based on LMs. The author analyzes the challenges on using the LM to perform the multimodal reasoning such as Hallucination problem. Based on the preliminary observations, the authors propose jointly exploit the reasoning ability in LLMs and the image understanding capability of visual question-answering models for general multimodal rationale generation. Experimental results demonstrate that the proposed method named DDCoT is effective from the experimental results.

Strengths

This paper clearly expresses their motivation and analyzes the common problems when exploiting language models for multimodal reasoning. 1. The motivation is clear and the analysis of hallucination problem is comprehensive. 2. The performance of the proposed DDCoT is demonstrated to be superior to some common baselines.

Weaknesses

The method proposed in this paper seems to be a simple performance combination of large language model and existing visual question answering model, and mainly utilizes the ability of large language model to decompose questions. Such a kind of method has already been explored in the research direction of using large language models to call external tools, such as Visual Chatgpt, Augmented Language Models. 1. Although the logic of the proposed method is clear, the technical novelties seem to be insufficient. The proposed method is similar to the Self-asking COT, or ``Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLP’’. For the multimodal information interaction, this paper does not present the detailed description. 2. The claims in lines 75-76 show that this paper is the first to explore the zero-shot multimodal reasoning. However, they do not give the enough analysis of this point and do not consider the performance of pretrained VLMs such as OFA or Flamingo. In addition, similar methods such as ModCR[1], Mini GPT-4, BLIP-2, and others should be compared. 3. The proposed module Rational-Compressed Visual Embedding (RCVE) aims to compress visual input embeddings according to the multimodal rationales by filtering visual features. I am confused by this calculation. For example, Formula 1, it adopts global visual features as query vectors and rational information as Key and Value. This process is the integration of different information, which could be not in line with its purpose. 4. Deep-Layer Prompting (DLP) is similar to the Prompt tuning v2 method [2]. The whole approach should be improved. Missing the corresponding citation. [1] Liu X, Ji K, Fu Y, et al. P-tuning v2: Prompt tuning can be comparable to fine-tuning universally across scales and tasks[J]. arXiv preprint arXiv:2110.07602, 2021. [2] Guo J, Li J, Li D, et al. From Images to Textual Prompts: Zero-shot VQA with Frozen Large Language Models[J]. arXiv preprint arXiv:2212.10846, 2022.

Questions

See the above strength and weakness parts

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

2 fair

Presentation

3 good

Contribution

2 fair

Limitations

None

Reviewer aDZq6/10 · confidence 4/52023-06-29

Summary

This paper proposes a novel DDCoT prompting that maintains a critical attitude through negative-space prompting and incorporates multimodality into reasoning by first dividing the reasoning responsibility of LLMs into reasoning and recognition and then integrating the visual recognition capability of visual models into the joint reasoning process.

Strengths

- The authors deeply analyze the challenges and insights in multimodal CoT for the rationale generation. - The authors propose a novel DDCoT prompting to maintain a critical attitude and identify reasoning and recognition responsibilities through the combined effect of negative-space design and deconstruction. - The experiments show the superiority of the proposed method.

Weaknesses

- Are the analysis results of the single case in section 3.1.1 and 3.1.2 applicable to most samples in the data set? In addition to chatgpt, whether gpt-3 can reach similar conclusions? - On what basis are the three principles in 3.2 considered? - Why the “uncertainty” are called as a negative space? - The statement ‘’learnable prompts to facilitate the alignment of visual and linguistic semantics at a shallow level but also utilizes explicit rationale to jointly encode multimodality by learning different prompts for each encoder layer.’’ should cite related papers. - There are some minor errors in this paper. For example, “exploring to introduce programming approach” should be “exploring to introduce a programming approach”.

Questions

See weakness.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

4 excellent

Contribution

3 good

Limitations

See weakness.

Reviewer kQGJ6/10 · confidence 4/52023-07-04

Summary

The work focuses on solving multimodal reasoning task. In the zero-shot setting, the work prompts LLM to conduct step-by-step reasoning. To avoid hallucination due to the lack of image features, LLM is asked to leave the answer to sub-questions as “uncertain” if they involve images. Then the corresponding sub-questions are answered by the visual components. In the fine-tuning setting, the generated rationales are used to train an LM augmented with a visual encoder. Experiments show that the elicited multimodal rationales can improve performance in both settings.

Strengths

- Originality: The work adopts careful prompt engineering that encourages LLM to (1) offload the step of visual recognition to the visual component and (2) discard errors in the intermediate results. Both are shown to be effective in mitigating hallucination and robust respectively. - Quality: The work investigates both zero-shot evaluation and fine-tuning to validate the effectiveness of the generated multimodal rationales. Comprehensive experiments and ablation studies are conducted to justify the design choices. - Clarity: The paper is well-written with illustrative running examples and figures. The intuitions behind each design choice are clearly stated, which helps readers to understand both the technical problems and motivations. - Significance: The proposed method brings significant improvement to a particular dataset, and also shows better generalization.

Weaknesses

1. The key ideas the paper tries to convey are somewhat scattered. The prompt engineering part and the visual component can be individually stand-alone as independent work. Right now, it is also hard to tell which part matters more. Perhaps the authors can add experiments where (1) a baseline model is trained on the rationales generated by the proposed method and (2) the proposed augmented model is trained on human-annotated rationales. 2. Only one dataset is chosen. And many questions in ScienceQA do not require images to be answered indeed. I would encourage the authors to explore more suitable datasets.

Questions

1. How do you sample the rationales for fine-tuning? Do you only sample those leading to the correct predictions?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

2 fair

Limitations

The authors have clearly stated the risks/biases brought by LLMs which are the common problems for LLMs.

Reviewer oVaJ6/10 · confidence 3/52023-07-06

Summary

The paper proposes Duty-Distinct Chain-of-Thought (DDCoT) Prompting for multimodal reasoning problems (e.g. VQA). Despite CoT's success in language-only reasoning problems, authors argue that multimodal reasoning challenges CoT as the rationale part is sensitive to the input information. Since image caption is the only source of image information for LLM, once the caption is generated poorly, rationale will intensity hallucinations. To this end, DDCoT proposes two insights: "keep critical thinking" and "disentangle reasoning and recognition". Specifically, the pipeline is like: 1. Given input question, zero-shot prompt LLM: “please think step-by-step and deconstruct the question down to necessary sub-questions" 2. Determine if each subquestion can be answered without visual information with zero-shot prompt: “Assume that you do not have any information about the picture, try to answer the sub-question and formulate the corresponding sub-answer as ‘Uncertain’ if the sub-question cannot be determined”. This helps deal with hallucination (i.e. LLM makes up facts about images). 3. Use an off-the-shelf VQA model to answer each subquestion "with negative space" (my guess is, each subquestion with 'Uncertain' as answer?) 4. Given subquestions and answers, LLM is prompted to perform CoT reasoning, generating a rationale and a final answer. The paper also considers a finetuning setup with some architecture inventions. Results on ScienceQA show - zero-shot prompting: 1-3% improvement from previous few-shot CoT baselines - finetuning: more impressive improvements and generalization from baselines. Analysis on visual information source, rationale generation process, fine-tuning components, and explainability are also conducted to further justify DDCoT.

Strengths

- Multimodal reasoning (e.g. VQA) is an important research direction, and LLM prompting is an emerging and promising approach to it. The paper highlights limitations of existing CoT-based methods for VQA (sensitive to caption; rationale does not improve performance), and propose reasonable solutions to these problems. - DDCoT proposes an interesting heuristic way to combine LLM and off-the-shelf VQA models as a tool: LLM decomposes questions, decide which parts to call VQA model, and aggregates information back for answer. The approach could be potentially valuable for other tasks (involving video, code, webpages...) - The consideration of both prompting and finetuning setups add to technical depth. - Experiments show improvements, and their design looks solid. I appreciate comprehensive and diverse ablations and analysis.

Weaknesses

- The presentation is a bit poor. The concrete problem is just "how to best put visual information in text, so that rationale can help", and the main insight is just "use VQA models across subquestions to best induce visual information, instead of off-the-shelf captioning". But it takes me some time to get it. I don't think "Flexibility/generalizability/explainability" makes too much sense to me --- I don't think few-shot CoT is really labor-intensive (just 3 examples??), inflexible, or hard to explain. The main motivation should be performance and generalization. 3.1.1 seems commonly known and a bit redundant. So intro, 3.1, 3.1.1, 3.1.2 are like four different stories of motivation, and a bit confusing to me. Why not sticking to one single story, instead of writing a story in 4 different ways/views/places? - Continuing on the presentation issue, I don't get Figure 1,2,3,4,5 --- all are examples, and in fact, Figure 5 should better be the teaser given it actually tells a bit about how the method works. Fig 1's example doesn't really tell much, as MM-CoT and UnifiedQA's rationales are omitted. etc. - Continuing on the presentation issue, some terms are confusing (or at least lack explaining), e.g. "negative space prompting". - While finetuning performances seem stronger, prompting performance is only slightly better than few-shot CoT baselines, and there are not a lot of prompting baselines to begin with. In the long run, seems prompting performance will be more important than finetuning performance (one can also imagine multi-modal GPT-4 just solves ScienceQA and beats every model, making it less worthwhile to study...) - The results are only on ScienceQA, not only how general the findings are, given original CoT is evaluated on various problems, and multimodal CoT should be able to solve a lot of different tasks as well.

Questions

I think the paper can benefit from better writing, and experiments from different datasets.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

1 poor

Contribution

2 fair

Limitations

Appendix E talks about hallucination and social bias.

Reviewer ka685/10 · confidence 4/52023-07-06

Summary

1. The paper studied the challenges and limitations in rationale generation for multimodal problems. Then the authors propose a Duty-Distinct Chain-of-Thought Prompting (DDCoT) to collect language-related or visual-related information and select valid information to generate the rationale. 2. The rationale can be used in zero-shot and fine-tuning settings for question answering. The authors designed a fine-tuning framework with deep-layer prompting and rationale-compressed visual embedding. 3. The experiment results demonstrate the effectiveness of the rationale generated by the proposed method with SOTA results on the ScienceQA benchmark.

Strengths

1. The proposed prompting design to separate the text reasoning and visual information for multi-modal QA problems is novel. 2. The result on the ScienceQA dataset is significant in zero-shot and fine-tuning settings, the ablation shows the effectiveness of different parts of the model.

Weaknesses

1. Some experiment settings are not clearly explained or confused. See questions. 2. It's better to validate the effectiveness of 'Integrate to Joint Reasoning' part of prompting in the ablation if it's an important part of the method.

Questions

1. The baseline(B) model in Table 2 is not explained. 2. Does 'B+our img' in Table 2 use rational-compressed visual embedding or not? What is the difference between 'B+our img' and 'w/ our R'? In line 308, it seems that the 'B+our img' includes rational-compressed visual embedding. 3. Does 'w/o DLP' in Table 3 include RCVE?

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

2 fair

Contribution

3 good

Limitations

The limitation is not discussed in the paper. Societal impact is discussed.

Reviewer oVaJ2023-08-16

Thanks

increased my score from 5 to 6 in light of the rebuttal

Authorsrebuttal2023-08-17

Thank you for your dedicated time and effort in reviewing our paper!

Thank you for your dedicated time and effort in reviewing our paper! Your insightful comments and appreciation serve as invaluable contributions that enhance the quality of our work.

Reviewer ka682023-08-18

Thanks to the authors for the clarifications! However, the reviewer still has the following concerns: 1. Why only evaluate on ScienceQA benchmark? There are several visual reasoning that rationale generation may help such as A-OKVQA [1]. The authors propose a rationale-focus method. I believe it's better to discuss what kind of problems and datasets it suits. 2. The contribution is somewhat limited as the technical difference between RCVE module and former visual-text attention methods such as Q-former in BLIP2, perceiver in Flamingo, etc, seems limited [2,3,4]. Also, the novelty of DLP module, as mentioned by reviewer ndag. [1] A-OKVQA: A Benchmark for Visual Question Answering using World Knowledge [2] BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models [3] Flamingo: a Visual Language Model for Few-Shot Learning [4] Perceiver: General Perception with Iterative Attention

Authorsrebuttal2023-08-19

Response to concern #1: Benchmark

Thank you for the response and for posting the concerns to discuss! > Why only evaluate on ScienceQA benchmark? There are several visual reasoning that rationale generation may help such as A-OKVQA [1]. The authors propose a rationale-focus method. I believe it's better to discuss what kind of problems and datasets it suits. > - We evaluate on ScienceQA benchmark because (1) it is used to diagnose the **multi-hop reasoning ability and interpretability** when answering multimodal science questions. Such requirements of multi-hop reasoning and interpretability enable ScienceQA suitable for evaluating the ability in multimodal rationale generation (the rationale reveals **the interpretable process of multi-hop reasoning**) and the rationale’s effects in multimodal reasoning. (2) “the goal of SCIENCEQA is to aid development of a reliable model that is capable of generating **a coherent chain of thought** when arriving at the correct answer” (copy from ScienceQA paper). As we employ CoT prompting to generate rationales and use rationales as a kind of CoT to arrive at answers, ScienceQA is pretty suitable for evaluating our DDCoT prompting’s effectiveness. (3) ScienceQA features **rich** domain diversity (natural science, social science, and language science), context diversity, and level diversity with different grade-level science exams, and thus has multiple splits. These features are suitable for evaluating our method **in different and general situations**. - Although ScienceQA is the most suitest dataset, we agree with the reviewer that our rationale generation could be applied to other suitable datasets and tasks as well (reviewer oVaj and kQGJ also pointed out this). We have conducted experiments on NoCaps for the captioning task and MSVD-QA for the video question answering task, leading to performance improvement. Please see our responses to Reviewer oVaj or kQGJ for detailed results and analysis.

Authorsrebuttal2023-08-19

Response to concern #2: Technical Difference

> The contribution is somewhat limited as the technical difference between RCVE module and former visual-text attention methods such as Q-former in BLIP2, perceiver in Flamingo, etc, seems limited [2,3,4]. Also, the novelty of DLP module, as mentioned by reviewer ndag. > Thank you for your rigorous consideration. And we respond to our technical contributions and differences from former works in below three different aspects: - **Our contributions and technical novelties are far more than proposing modules (RCVE+DLP) for fine-tuning learning** (”Utilization for Fine-tuning Learning” in Sec 3.3). Instead, ”Utilization for Fine-tuning Learning” **is only one part of one of our three core contributions**, which is proposed to validate that our rationales are not only helpful for multimodal reasoning in zero-shot prompting but also in fine-tuning learning. Our core contributions and novelties are: 1. We are the **first** to study and achieve **zero-shot multimodal rationale generation**, considering flexibility, generalizability, and explainability. We deeply analyze the challenges of **putting visual information in the text to generate rationales** (Sec 3.1) and further **induct the critical principles** (Lines 187-190) in generating flexible and generalizable multimodal rationales using LLMs. These challenges (such as the discovery that LLMs’ hallucinations are intensified with interleaved multimodal inputs), analysis, and principles are not fully explored by previous works. 2. We propose zero-shot DDCoT prompting to generate multimodal rationales (Sec 3.2), which consists of three steps: (1) decompose the question into sub-questions **with negative-spacing prompting by considering the uncertainty.** Negative-space prompting with uncertainty brings 7.73% performance improvement. Please refer to Table 1 and Table 2 in Appendix, (2) visual recognition to obtain visual complements, and (3) **integration to joint reasoning** (see first-round “Rebuttal by Authors” for the importance of this design). The former methods do not explore these highlighted aspects. 3. We propose the utilization of rationales to improve LMs’ multimodal reasoning (Sec 3.3). To show the effectiveness and generalizability, we achieve multimodal Ireasoning in both settings: (1) **zero-shot prompting** and (2) **finetuning learning** (*the contribution mentioned by the reviewer*). Our methods significantly improve performance in both settings while exhibiting impressive generalization ability. - Regarding the finetuning modules themself, our RCVE and DLP are also different from former visual-text attention methods. 1. Q-former consists of two transformer submodules: image transformer and text transformer with share self-attention layers. The design is in order to make Q-former applicable to multiple different pre-training tasks simultaneously. In contrast, our approach utilizes a single transformer with cross-attention layers to compact visual embeddings from rationale input. And our finetuning modules are directly supervised by the training objectives of the downstream task. 2. Perceiver in Flamingo maps diverse-sized feature maps into a few visual tokens, disregarding the text context. In contrast, our RCVE first engages with the rationales generated by our DDCoT approach, and subsequently compresses visual features, taking into account the guidance provided by the rationales. 3. While DLP is a common design in prompt learning, it diverges in role in our work from previous research. Our DLP not only facilitates the alignment of vision and language modalities at a shallow level but also collaborates with RCVE to utilize rationales for joint multimodal encoding at every layer. - Our proposed rationale generation can be compatible with vision-language models, such as the mentioned Flamingo and recent Mini GPT-4. We first generate rationales by our DDCoT prompting and employ the rationales as extra inputs to Flamnigo and Mini GPT-4. Our DDCoT prompting significantly improves Flamingo and Mini GPT-4 by 11.14%, and 10.96%, respectively. | | NAT | SOC | LAN | TXT | IMG | NO | Avg | | --- | --- | --- | --- | --- | --- | --- | --- | | Flamingo | 21.89 | 52.41 | 20.27 | 23.50 | 39.11 | 19.02 | 27.87 | | Flamingo with our rationales | 39.20 | 48.93 | 30.90 | 39.68 | 45.81 | 32.40 | 39.01 | | Mini GPT-4 | 43.83 | 48.59 | 43.36 | 55.01 | 42.84 | 41.67 | 44.71 | | Mini GPT-4 with our rationales | 57.37 | 62.32 | 46.82 | 65.91 | 56.72 | 48.57 | 55.67 |

Reviewer ndag2023-08-22

Official Comments from ndag

The rebuttal addresses my comments. The paper does have merits. Therefore, I am now tending to accept the paper.

Program Chairsdecision2023-09-21

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC