Summary
This paper introduces Coreset Selection for Object Detection (CSOD), a method designed to select a compact yet representative subset of the complete dataset for object detection. CSOD generates image-wise and class-wise representative feature vectors for multiple objects of the same class within each image. To ensure both representativeness and diversity, this paper proposes submodular optimization, utilizing the representative vectors in this optimization process to select a subset. Experimental results reveal that CSOD surpasses random search by 6.8% on the PASCAL VOC benchmark, even when selecting only 200 images.
Strengths
1. The idea is simple, and the paper is clear and easy to follow.
2. I truly appreciate the author's inclusion of Appendix Figure 7, as it significantly aids the reader in better comprehending the concept.
3. The core formula, identified as formula (4), is logical. The initial term's objective is to select the most representative from those that have not been previously chosen, whereas the latter term aims to select an option distinct from those that have already been selected.
Weaknesses
1. The author appears to be the first to propose and investigate this problem, given the absence of comparisons to other methods within this context. However, the methodology lacks rigorousness. The stated objective is to select the most representative subset from a complete dataset. Yet, the paper seems to primarily focus on selecting a subset to enhance the mAP of Faster R-CNN, which is not synonymous. Defining representativity is challenging, as it can vary across different datasets and models.
2. The results presented are not entirely persuasive. Firstly, the author does not compare the work to any other studies, but merely to a few basic counterparts such as random sampling and classwise random sampling. Even if there are no other methods available for object detection, it would be beneficial for the author to compare some classification methods and apply them to object detection.
Secondly, the paper's main experiments are based on PASCAL VOC, an older and relatively small object detection dataset. This is not particularly convincing as the resources required to train the model on the entire PASCAL VOC are minimal, rendering the selection of a small representative subset from it somewhat redundant.
Additionally, as demonstrated in Table 5, the improvement in performance on the COCO dataset is marginal when compared to the random sampling counterpart.
3. The proposed method is succinct yet lacks effectiveness. The paper's primary contribution appears to be exclusively formula (4). As demonstrated in Table 5, its performance modestly surpasses that of the random sampling counter.
Questions
I have put some of my questions and doubts in the weakness sections, here are some more questions:
1. The paper only tests Faster R-CNN on COCO, how about other models?
2. How about selecting more images on COCO dataset, for example, 4000/8000? 400/800 seems meaningless given today's computation resource
Rating
3: reject, not good enough
Confidence
4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.