Gold-YOLO: Efficient Object Detector via Gather-and-Distribute Mechanism

Summary

In this paper, the authors proposed a Gather-and-Distribute mechanism (GD) for efficient information exchange in YOLOs by globally fusing multi-level features and injecting the global information into higher levels. The proposed GD-YOLO architectures show good results compared with the existing YOLO series. The authors also presented a pre-training method, where we pre-train the backbone on ImageNet 1K using the MAE method, which improves the convergence speed and accuracy of the model.

Strengths

1. In this paper, the authors proposed a gather-and-distribute mechanism to replace the traditional FPN structure. By using this unified module to gather and fuse information from all levels and subsequently distribute it to different levels, they can avoid the loss of information inherent in the traditional FPN structure and also enhance the neck’s partial information fusion capabilities without significantly increasing latency. 2. The paper is clear and organized, and easy to follow. 3. Authors have provided a comprehensive comparison between the proposed model and YOLOs.

Weaknesses

1. The motivation is a little bit unclear. The advantage of this GD mechanism compared with traditional FPN are not very clear. 2. Based on the results, the tradeoff between accuracy and latency is pretty similar compared with baselines (YOLOv8).

Questions

If the best trade-off between latency and accuracy is the main goal of this method, isn't fairer to compare with YOLOv8 instead of YOLOv6, since YOLOv8 has the closest latency with GDYolo?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

1. The idea is pretty interesting but the advantage of this proposed method doesn't show a very convincing result.

Summary

In this research, the authors propose a novel Gather-and-Distribute (GD) mechanism implemented through convolution and self-attention operations. This mechanism, incorporated into the GD-YOLO model, significantly enhances multi-scale feature fusion capabilities and achieves a remarkable balance between latency and accuracy across all model scales. Notably, the researchers introduce MAE-style pretraining to the YOLO-series models for the first time, enabling unsupervised pretraining benefits. The GD-YOLO-N variant achieves exceptional results, with a 39.9% Average Precision (AP) on the COCO val2017 dataset and a remarkable 1030 Frames Per Second (FPS) on a T4 GPU. These results surpass the previous state-of-the-art model, YOLOv6-3.0-N, with a similar FPS by 2.4%.

Strengths

GD-YOLO is a quite impressive work, I think as it globalized the features from local and can retrieve features without tagging other layers feature with it.

Weaknesses

1. While GD-YOLO-N cannot use the proposed backbone for limited capacity, what will be the approaches for mobile deployment as your main objective is YOLO for mobile deployment? 2. GD-YOLO-N implementation is not quite understandable while it misses a few parts like MIM and seems pretty same as YOLOv6 with EWA and AAT as stated. 3. Comparison with the Transformer based model could add a bit more value to this paper. 4. Abbreviations should be added after first introduction and then using the abbreviation is the ideal practice. For instance, in line 46, it should have been “... use Feature Pyramid Network (FPN) and…” instead of just FPN. 5. Line 45 to 54: This para basically represents the contribution part, which can be presented in bullet terms. 6. Line 58: “...SOTA YOLOv6…” : As per my knowledge, at this point, YOLOv8 is the SOTA. I would request for a checkup and update of this from your end. 7. Line 74-75: What strengths YOLOv8 takes from their predecessor, it is not mentioned. 8. Line 94: “In this study, we will…..” - it is recommended to use present tense, instead of future tense. 9. Table 1: Best values should be bolded. Minor: 1. Line 32: “...detectors.Despite…, Line 138: “...alignment module(Low-FAM)...” - Spacing issue. 2. Line 257: Double comma issue.

Questions

1. Why need to use LAMB Optimizer while already using SGD and manual scheduled Learning rate? 2. If you use EMA, then why again, LAMB is needed?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

2 fair

Presentation

2 fair

Contribution

3 good

Limitations

Limitations of GD-YOLO not added; however, I would suggest adding it.

Summary

This paper studies the problem of efficient object detector and proposes the Gather-and-Distribute mechanism (GD) mechanism to alleviate the information fusion problem. The experiments on the COCO dataset demonstrate the effectiveness of the proposed method.

Strengths

+ This paper studies an important topic, efficient object detection, and achieves a great balance between accuracy and speed in its results. + The ablation studies have been provided to verify the effectiveness of the proposed module.

Weaknesses

- The structural design of this paper is quite confusing in some aspects, such as the choice of where to inject information. For example, in Low-GD, semantic information is only injected into P3 and P4, while one would expect that global semantic information could also benefit the P5 branch. Similarly, in High-GD, information is only injected into N4 and N5. The authors should provide the rationale and advantages of this design choice to address these doubts. - In line #157, the authors mention they were inspired by [28]. However, the injection design is more closely related to Topformer, which bears greater relevance to the structure of the current paper. The authors should consider discussing and citing Topformer as another source of inspiration or relevant related work. [a] TopFormer: Token Pyramid Transformer for Mobile Semantic Segmentation, CVPR 2022.

Questions

Please refer to the Weaknesses.

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

2 fair

Limitations

None

Summary

In this paper, the authors proposed an efficient object detection network to make a new trade-off between efficiency and effectivity. In the framework, a lightweight adjacent-layer fusion module termed as gather-and-distribute (GD) mechanism is proposed to take place the conventional neck module in general detectors. Experimental results on COCO built-on the YOLO-series off-the-peg detectors demonstrate the effectiveness of the proposed method.

Strengths

-The bilateral interaction of information in multi-layer layers is interesting, and I like the work that simple changes can bring significant improvement for foundational tasks. -The experimental setup of ablation studies on COCO is helpful for follow-up work in the future.

Weaknesses

The main weakness of this paper is the limited technical novelty and the relatively inadequate experiments. First, although the authors try to explain that the proposed neck module of the GD mechanism can improve the detection accuracy of the detection task and benefit the computational efficiency, the novelty of the proposed module can be seen as a compromise similar to the stack of the existing technologies. Additionally, the experiment is carried out only on YOLO-series methods, and there is a lack of testing and verification on other datasets. This paper needs further modification in terms of layout design, e.g., the fonts in the Fig.4 are generally small and unclear, which is very difficult for review. Page 3 Line 82-83, Improving ->can improve, introduce -> introduced.

Questions

The innovation of the proposed method needs to be improved comprehensively, and there is a lack of more substantial analysis and understanding in terms of innovation in technology. Then, it is necessary to verify the scalability and robustness of the proposed method in more experiments and even more tasks. In addition, the writing and layout concept of the article still needs to be improved.

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

2 fair

Presentation

2 fair

Contribution

3 good

Limitations

Applicable

Summary

The paper presents a real-time object detection method for the YOLO series, introducing a 'Gather-and-Distribute' (GD) mechanism. Despite achieving good results on the COCO dataset, the paper lacks significant novelty and doesn't significantly advance multi-scale feature fusion or FPN-based methods. A deeper comparison with prior work could strengthen its scientific contribution.

Strengths

1. The paper exhibits commendable clarity with well-written content and lucidly presented figures, making it easy to understand. 2. Performance-wise, the proposed contribution demonstrates impressive results on the COCO dataset in terms of both accuracy and computational efficiency. 3. The introduction of the 'Gather-and-Distribute' (GD) mechanism is particularly striking. This method enhances multi-scale feature fusion capabilities, striking an excellent balance between latency and accuracy across different model scales.

Weaknesses

Weaknesses: 1. Despite achieving strong results on the COCO dataset in terms of accuracy and efficiency, the paper lacks substantial scientific novelty. The method, while technically sound, doesn't significantly advance the field. 2. The manuscript's focus is on real-time object detection and multi-scale features for the YOLO-Series. However, the related work section needs to delve more into multi-scale features. 3. The concepts of Low/High-FAM and the lightweight adjacent layer fusion (LAF) module are not new in the field, having been discussed in M2Det [1] and [2] respectively. Suggestions: 1. The authors need to restructure the related work section to better represent and compare with multi-scale features[1] or FPN-based methods [2-8]. 2. Adding more comparative analyses and surveys related to multi-scale features and FPN-based methods will establish their work as more than a minor modification of previous works. 3. Despite YOLO-Series being known for speed and efficiency, GD-YOLO appears to be slower than the baseline (YOLOV6: v3). The authors should address this discrepancy. 4. To reiterate, the work seems to be more application-focused with minimal contribution to the field. I recommend the authors address the points above to enhance their scientific contribution, which may change the review score. Otherwise, the work might not meet the standards of a top-tier conference. [1] Zhao, Q., Sheng, T., Wang, Y., Tang, Z., Chen, Y., Cai, L., & Ling, H. (2019, July). M2det: A single-shot object detector based on multi-level feature pyramid network. In Proceedings of the AAAI conference on artificial intelligence (Vol. 33, No. 01, pp. 9259-9266). [2] Chen, P. Y., Hsieh, J. W., Wang, C. Y., & Liao, H. Y. M. (2020). Recursive hybrid fusion pyramid network for real-time small object detection on embedded devices. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (pp. 402-403). [3] Quan, Y., Zhang, D., Zhang, L., & Tang, J. (2023). Centralized feature pyramid for object detection. IEEE Transactions on Image Processing. [4]Yang, G., Lei, J., Zhu, Z., Cheng, S., Feng, Z., & Liang, R. (2023). AFPN: Asymptotic Feature Pyramid Network for Object Detection. arXiv preprint arXiv:2306.15988. [5] Jin, Z., Yu, D., Song, L., Yuan, Z., & Yu, L. (2022, October). You should look at all objects. In European Conference on Computer Vision (pp. 332-349). Cham: Springer Nature Switzerland. [6] Chen, Q., Wang, Y., Yang, T., Zhang, X., Cheng, J., & Sun, J. (2021). You only look one-level feature. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 13039-13048). [7] Jin, Z., Liu, B., Chu, Q., & Yu, N. (2020). SAFNet: A semi-anchor-free network with enhanced feature pyramid for object detection. IEEE Transactions on Image Processing, 29, 9445-9457. [8] Chen, P. Y., Chang, M. C., Hsieh, J. W., & Chen, Y. S. (2021). Parallel residual bi-fusion feature pyramid network for accurate single-shot object detection. IEEE Transactions on Image Processing, 30, 9099-9111.

Questions

1. Could you elaborate on why YOLOv7-E6E has been chosen as the "L" model for comparison, given that its size is 1280? It seems that YOLOv7-X would be a more suitable choice for a fair comparison. 2. The relevance of the section "Masked image modeling pre-training" isn't clear, as it doesn't appear to directly relate to multi-scale features and FPN-based methods. Additionally, the results from this section don't seem to add significant value. Would it be more beneficial to include additional comparisons or ablation studies for multi-scale features in lieu of this section?

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.

Soundness

3 good

Presentation

4 excellent

Contribution

2 fair

Limitations

I want to highlight the author's commendable acknowledgment of the potential military applications of their model. They clearly state their commitment to prevent such uses. This level of social impact consideration is laudable and sets a positive precedent for responsible research conduct.

lets followup author response

Hi all, Thanks for serving as the reviewers for this submission. As the authors have already provided their responses. Now let's start further discussion. Here is a to-do list: (1) Please acknowledge the authors when you finish reading their responses. (2) Please indicate whether you have any further questions for the authors such that they can continue to response. (3) Please indicate whether you are willing to change the ratings. best, The AC