Library

Amnesia as a Catalyst for Enhancing Black Box Pixel Attacks in Image Classification and Object Detection

It is well known that query-based attacks tend to have relatively higher success rates in adversarial black-box attacks. While research on black-box attacks is actively being conducted, relatively few studies have focused on pixel attacks that target only a limited number of pixels. In image classification, query-based pixel attacks often rely on patches, which heavily depend on randomness and neglect the fact that scattered pixels are more suitable for adversarial attacks. Moreover, to the best of our knowledge, query-based pixel attacks have not been explored in the field of object detection. To address these issues, we propose a novel pixel-based black-box attack called Remember and Forget Pixel Attack using Reinforcement Learning(RFPAR), consisting of two main components: the Remember and Forget processes. RFPAR mitigates randomness and avoids patch dependency by leveraging rewards generated through a one-step RL algorithm to perturb pixels. RFPAR effectively creates perturbed images that minimize the confidence scores while adhering to limited pixel constraints. Furthermore, we advance our proposed attack beyond image classification to object detection, where RFPAR reduces the confidence scores of detected objects to avoid detection. Experiments on the ImageNet-1K dataset for classification show that RFPAR outperformed state-of-the-art query-based pixel attacks. For object detection, using the MSCOCO dataset with YOLOv8 and DDQ, RFPAR demonstrates comparable mAP reduction to state-of-the-art query-based attack while requiring fewer query. Further experiments on the Argoverse dataset using YOLOv8 confirm that RFPAR effectively removed objects on a larger scale dataset. Our code is available at https://github.com/KAU-QuantumAILab/RFPAR.

cs.CVcs.AIComputer Science
View graphSource

Paper

References (44)

01Reinforcement Learning Based Black-Box Adversarial Attack for Robustness ImprovementS. Sarkar, Ashwin Ramesh Babu, Sajad Mousavi et al.2023 · 2023 IEEE 19th International Conference on Automation Science and Engineering (CASE) · 13 citations
02A Comprehensive Review of YOLO Architectures in Computer Vision: From YOLOv1 to YOLOv8 and YOLO-NASJuan R. Terven, Diana-Margarita Córdova-Esparza, J. Romero-González2023 · Machine Learning and Knowledge Extraction · 2.8k citations In Library
03Dense Distinct Query for End-to-End Object DetectionShilong Zhang, Wang xinjiang, Jiaqi Wang et al.2023 · Computer Vision and Pattern Recognition · 299 citations In Library
04When Adversarial Training Meets Vision Transformers: Recipes from Training to ArchitectureYi Mo, Dongxian Wu, Yifei Wang et al.2022 · Neural Information Processing Systems · 103 citations In Library
05A Large-scale Multiple-objective Method for Black-box Attack against Object DetectionSiyuan Liang, Longkang Li, Yanbo Fan et al.2022 · European Conference on Computer Vision · 63 citations In Library
06Boosting Black-Box Attack with Partially Transferred Conditional Adversarial DistributionYan Feng, Baoyuan Wu, Yanbo Fan et al.2022 · Computer Vision and Pattern Recognition · 51 citations
07Phantom Sponges: Exploiting Non-Maximum Suppression to Attack Deep Object DetectorsAvishag Shapira, Alon Zolfi, Luca Demetrio et al.2022 · IEEE Workshop/Winter Conference on Applications of Computer Vision · 60 citations In Library
08Swin Transformer V2: Scaling Up Capacity and ResolutionZe Liu, Han Hu, Yutong Lin et al.2021 · Computer Vision and Pattern Recognition · 2.9k citations In Library
09Parallel Rectangle Flip Attack: A Query-based Black-box Attack against Object DetectionSiyuan Liang, Baoyuan Wu, Yanbo Fan et al.2021 · IEEE International Conference on Computer Vision · 105 citations In Library
10Universal adversarial perturbations against object detectionDebang Li, Junge Zhang, Kaiqi Huang2021 · Pattern Recognition · 74 citations
11Daedalus: Breaking Nonmaximum Suppression in Object Detection via Adversarial ExamplesDerui Wang, Chaoran Li, Sheng Wen et al.2021 · IEEE Transactions on Cybernetics · 83 citations
12Training data-efficient image transformers & distillation through attentionHugo Touvron, M. Cord, Matthijs Douze et al.2020 · International Conference on Machine Learning · 9.5k citations In Library

Scroll for more · 32 remaining

Similar papers

Parallel Rectangle Flip Attack: A Query-based Black-box Attack against Object DetectionSiyuan Liang, Baoyuan Wu, Yanbo Fan et al.2021 · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) In Library
Superpixel Attack: Enhancing Black-box Adversarial Attack with Image-driven Division AreasIssa Oe, Keiichiro Yamamura, Hiroki Ishikura et al.2025 · Applied Informatics · 1 citation In Library
Parallel Rectangle Flip Attack: A Query-based Black-box Attack against Object DetectionSiyuan Liang, Baoyuan Wu, Yanbo Fan et al.2022 · arXiv (Cornell University) · 104 citations In Library
Towards Query Efficient Black-box AttacksYali Du, Meng Fang, Jinfeng Yi et al.2018 In Library
Pixle: a fast and effective black-box attack based on rearranging pixelsJary Pomponi, Simone Scardapane, Aurelio Uncini2022 · arXiv (Cornell University) In Library
Pixle: a fast and effective black-box attack based on rearranging pixelsJary Pomponi, Simone Scardapane, Aurelio Uncini2022 · 2022 International Joint Conference on Neural Networks (IJCNN) In Library
PICA: A Pixel Correlation-based Attentional Black-box Adversarial AttackJie Wang, Zhaoxia Yin, Jin Tang et al.2021 · arXiv (Cornell University) · 2 citations In Library
You Only Query Once: Effective Black Box Adversarial Attacks with Minimal Repeated QueriesDevin Willmott, Anit Kumar Sahu, Fatemeh Sheikholeslami et al.2021 · arXiv (Cornell University) · 3 citations In Library
GreedyPixel: Fine-Grained Black-Box Adversarial Attack Via Greedy AlgorithmHanrui Wang, Ching-Chun Chang, Chun-Shien Lu et al.2025 · IEEE Transactions on Information Forensics and Security · 5 citations In Library
GreedyPixel: Fine-Grained Black-Box Adversarial Attack via Greedy AlgorithmHanrui Wang, Ching‐Chun Chang, Chun-Shien Lu et al.2025 · IEEE Transactions on Information Forensics and Security In Library

Peer review

Accept (poster)NeurIPS 2024 · via OpenReview
Reviewer R24Y5/10 · confidence 4/52024-07-04

Summary

The authors present a few-pixel querying blackbox attack for image classification and object detection models, called Remember and Forget Pixel Attack using Reinforcement Learning (RFPAR). As this is a blackbox attack, the setting assumes the attacker does not have access to the target model weights, though as a querying attack, they assume they can query many test inputs to the victim model to optimize the attack. The resulting attacks edit only a small number of pixels and are effective at changing the classification target or at suppressing all objects in detection models. The authors demonstrate their attack on several models and dataset settings. They additionally present ablations of their attack optimization method.

Strengths

The authors present an effective attack that achieves high ASR with few pixel edits. Moreover, the authors demonstrate their attack not only on image classification but on object detection as well. The authors also demonstrate their attack on a diverse collection of models for both the classification and detection settings. Compared to the baseline methods, the proposed RFPAR achieves high ASR and lower queries. In terms of the image edit distance (L0), theirs is second only to OnePixel, though their ASR is significantly higher than OnePixel.

Weaknesses

In terms of presentation, it seems like the “Forget Process” is very simple, as it is only resetting the RL agent and passing the output of the previous optimization round. I recognize that the authors have demonstrated the importance of this step in their ablations. I just feel like they present the Remember Process and Forget Process as if both are equally major steps, when Forget is only a minor step compared to Remember. While the percentage of pixels edited is small, the attacks can still be noticeable to the human eye, especially when you have dark pixels on a bright background region like the sky. This is less noticeable when not zoomed in on the images. The author’s method optimizes pixels with an all-or-nothing approach, setting each pixel channel to either 0 or 255, which makes the edits more visible. The results on Argoverse (Section 3.4) are somewhat limited. If I understand correctly, the analysis is only applied to one image that was attacked. The paper lacks a dedicated section to discuss related works, though some parts of the introduction discuss related works.

Questions

Details of the Ablations (Section 3.5) could be presented more clearly. If M is ablated, how does the optimization change? Is it equivalent to only running the inner loop once? If I is ablated, does this mean the RL model is not reset between outer iterations? Is the optimization still run for a similar number of steps?

Rating

5

Confidence

4

Soundness

4

Presentation

3

Contribution

3

Limitations

The authors do not extensively discuss the limitations of the work. In Section 4 they briefly discuss the potential negative impact that adversarial attacks can have. The authors do not present any defense testing for their attack. It would be good to see some defense testing, to know if this attack is easily defended against or not. As this is a work focused on presenting an adversarial attack, there is a potential risk of negative impact.

Reviewer RX8p5/10 · confidence 4/52024-07-13

Summary

This paper proposes a new adversarial attack method called Remember and Forget Pixel Attack using Reinforcement Learning (RFPAR) for image classification and object detection models. The key contributions are: - A novel pixel-based black-box attack using reinforcement learning with "Remember" and "Forget" processes - Extension of query-based pixel attacks from image classification to object detection - Experimental results showing improved performance over existing methods on ImageNet and MS-COCO datasets

Strengths

The paper's approach is original in using a dual-stage reinforcement learning process to manage adversarial attacks, which is a unique combination in the context of pixel-based black-box attacks. The paper is well-written with a clear structure.

Weaknesses

Lack of Theoretical Foundation: The paper primarily focuses on empirical results without providing theoretical insights or proofs to back the efficacy of the Remember and Forget processes in adversarial contexts. Limited Evaluation: The robustness of the method to various defenses is not evaluated. It's unclear how RFPAR would perform against adversarially trained models or other defense techniques.

Questions

1. Can you provide more theoretical insight into why the Remember and Forget processes lead to improved performance? Perhaps some analysis of the optimization landscape? 1. How does the computational cost of RFPAR compare to other methods, especially for larger images? Are there ways to improve efficiency further? 1. Have you tested RFPAR against any adversarial defenses? How do you expect it to perform against adversarially trained models?

Rating

5

Confidence

4

Soundness

2

Presentation

2

Contribution

2

Limitations

The ethics and potential negative impacts of this work are not sufficiently discussed. Developing more effective adversarial attacks could have serious consequences. This paper should discuss potential negative impacts and how they might be mitigated.

Area Chair iCnt2024-08-12

Dear Reviewer RX8p, The authors have provided a rebuttal. Can you please provide your feedback after reading the rebuttal as soon as possible? The deadline is approaching fast. Thanks, AC

Reviewer RX8p2024-08-13

Thank the authors for the detailed feedback, which addressed my main concerns. While the proposed attack is interesting, can the authors elaborate more on its real world implications?

Authorsrebuttal2024-08-13

Reviewer RX8p, We thank the reviewer for responding and raising important questions. We would like to explain the real-world implications of the proposed attack through specific examples. --- **1. Reproducing a physical issue with the camera.** Physical defects in camera sensors, such as hot pixels(255) or dead pixels(0), can impact image quality and degrade the performance of neural network models. our approach is similar to a physical defect in the camera. In this paper, RFPAR simulates these physical issues by replacing specific pixels with values of either 0 or 255, which induces incorrect predictions by the neural network. This perturbation could occur in real-world scenarios, which is why neural networks must be robust against it. However, research on pixel($L_0$) attacks is limited compared to other types of attacks. This approach allows us to analyze the vulnerabilities of the model with respect to physical defects. By addressing these vulnerabilities, we contribute to the development of more robust neural networks that can withstand such defects in real-world scenarios **2. Impact on Defective Product Detection AI System[R1].** If the proposed attack is applied to a defective product detection AI system, it could prevent the system from accurately detecting defective products, leading to significant issues in product quality control. This increases the likelihood of defective products reaching end consumers, negatively impacting the company's credibility and brand image. Additionally, it could result in large-scale recalls or legal issues, causing substantial financial losses for the company. Therefore, by raising awareness about the potential for such attacks and emphasizing the need for defensive strategies at the corporate level, we can contribute to enhancing security. **3. Impact on Disease Prediction Systems[R2].** If the proposed attack is applied to disease prediction systems, there is a possibility of incorrect predictions. This could lead to fatal consequences, especially in the medical field, where misdiagnosis may result in inappropriate treatment plans. For example, if errors occur in predicting infectious diseases like COVID-19, patients might not receive timely treatment or could undergo unnecessary treatments, posing a significant threat to their lives. Given the potential severe impact on public health, implementing safety measures and defensive mechanisms against such attacks can contribute to preventing these risks. **4. Impact on Object Detection Systems.** When object detection technology is applied in autonomous vehicles or commercial AI robots, the proposed attack could disrupt the system, preventing it from correctly recognizing people or other objects. This could lead to traffic accidents where an autonomous vehicle fails to detect pedestrians, or safety incidents in workplaces where commercial robots are deployed. Such scenarios are not merely technical issues but are directly related to human life, making them critical considerations. The supplementary material we provided, "Argoverse Attack Video.pptx," demonstrates a practical example of such an attack.By addressing these vulnerabilities, we contribute to the development of stronger security measures that enhance the safety and reliability of AI-driven systems, ultimately protecting human lives. In conclusion, our proposed attack is not merely a theoretical construct; it addresses practical issues that could realistically occur in camera-based vision systems. While there are challenges in applying this method in practice, which limits its immediate real-world applicability, it effectively simulates camera defects that closely align with real-world problems. Therefore, it is crucial for camera-based vision systems to proactively investigate and defend against these potential vulnerabilities. --- We would be happy to continue this discussion if further clarification is needed. If there are no additional concerns, we kindly request the reviewer to consider raising the rating. Thank you. [R1] T. Czimmermann et al, "Visual-Based Defect Detection and Classification Approaches for Industrial Applications - A SURVEY," in Sensors, 2020. [R2] L. Wang et al, "Covid-net: A tailored deep convolutional neural network design for detection of covid-19 cases from chest x-ray images," in Scientific reports, 2020.

Reviewer p4SQ6/10 · confidence 4/52024-07-29

Summary

The paper proposes a pixel-based black box attack called RFPAR that uses RL to perturb pixels. The paper has a remember and forgetting mechanism. In the remember step, they memorize the perturb images to reduce the number of queries and in forgetting, they try to remove them from the memory. Although their main target are object detection algorithms, they are showing good accuracy drop, aka attack effectiveness on image classification task such as ImageNet-1K.

Strengths

The experiments of the paper are thorough and comprehensive. They are proving RFPAR's effectiveness. The numbers and the effectiveness of the method is perfectly proven.

Weaknesses

In my opinion, paper doesn't have that much of flaw. However, most of the models that are used in the paper are CNN-based model. The only transformer model used is the VIT which paper didn't specified which variant. Given the numbers, assuming the authors used VIT-Based, I'd like to know the effect of the attack on larger models like ViT-Huge, ViT-large, DeIT, etc. Besides, unlike the claim of the paper that is main target is object detection, most of the experiments are targeting image classification methods.

Questions

I like the idea and the methodology of the paper. However, the drop of accuracy in ViT is the lowest and the other object detection/image classification methods are all CNN-Based. Therefore, the effectiveness of the attack on transformer based model is vague in my opinion. It would be great if the authors provide more experiments on transformers or limit the scope of their method to to convolutional networks. Alternatively, an experiment that shows the effect of higher number of queries can also be helpful. Despite its success, the rate of the attack succession on ViT is significantly lower than CNN based methods. Perhaps increasing the number of queries could fix this ?

Rating

6

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

Please refer to weaknesses and questions.

Area Chair iCnt2024-08-12

Dear Reviewer p4SQ, The authors have provided a rebuttal. Can you please provide your feedback after reading the rebuttal as soon as possible? The deadline is approaching fast. Thanks, AC

Authorsrebuttal2024-08-13

Dear Reviewer p4SQ, Thank you very much for your continued engagement with our work and for your valuable feedback. I apologize for any confusion regarding the results mentioned in my previous response. These results are detailed in the (PDF) provided in the global rebuttal, which was due to space constraints. Specifically: - Table 1 presents the results of additional experiments conducted on ViT-large, Swin-V2-t, and Deit-B, where we compared the performance of OnePixel, Pixle, and our proposed RFPAR method. - Table 2 includes the outcomes of further experiments on object detection models, specifically Atss and Deformable DETR. - Table 3 shows the results of RFPAR on transformer-based models with varying query counts. I hope this helps in locating the information you are looking for, and I would be more than happy to provide any further details or clarification if needed. Thank you once again for your careful consideration of our work.

Reviewer R24Y2024-08-12

I thank the authors for their responses and discussion. I believe the authors should add these ablation clarifications to main work. While I still believe this paper has some weaknesses, I think the contributions warrant an accept rating, and I will maintain my initial rating.

Authorsrebuttal2024-08-12

Dear Reviewer fwtn, Thank you for reading our paper and providing comments to help us improve it. Below, we will address your concerns. --- **C1) Include a more comprehensive discussion of potential risks and misuse scenarios.** We will explain the potential risks that could arise in various application domains. For example, if this attack is applied to a defective product detection AI [R1], it could lead to a decline in product quality and negatively impact the company's credibility. Additionally, attacking a disease prediction system [R2] could result in incorrect diagnoses, which may have serious or even fatal consequences. In the case of object detection, if autonomous vehicles or commercial AI robots fail to detect people due to our attack, it could result in serious safety accidents. [R1] T. Czimmermann et al, "Visual-Based Defect Detection and Classification Approaches for Industrial Applications - A SURVEY," in Sensors, 2020. [R2] L. Wang et al, "Covid-net: A tailored deep convolutional neural network design for detection of covid-19 cases from chest x-ray images," in Scientific reports, 2020. --- **C2) Propose possible mitigation strategies or defensive measures against their attack method.** Our method requires an average of over 1000 queries to successfully deceive an object detection model. Similarly, as shown in Table 3 in the (PDF), transformer-based models also require an average of over 1000 queries to achieve a high success rate. If we limit the number of queries to around 1000 in a short period of time, our method can easily defend the model. For CNN-based models, since fewer queries are needed, limiting the queries to 400 can effectively defend the model. Additionally, according to the attack results on adversarially trained models shown in Table 5 in the (PDF), adversarial training effectively reduces the attack success rate and increases the number of queries needed. Therefore, by adversarially training the models and appropriately limiting the queries, this attack can be defended against. --- **C3) Discuss how this research could ultimately contribute to developing more robust and secure AI systems.** In real-world applications of AI models, not only performance but also the ability to defend against malicious attacks is of critical importance. Since actual attackers often attempt to compromise models without knowing their internal informations, research on various black-box attacks and the development of defense strategies are essential. This research contributes significantly to these areas in the following ways. **Highlighting the Need for Defense Against Pixel-Level Attacks and Understanding Neural Network Limitations** This research introduces a new method that enhances the success rate of black-box query-based $L_0$ (pixel) attacks while minimizing both the $L_0$ and the number of queries, an area that has been relatively underexplored. This work highlights the need for other defense research to also consider pixel-level attacks, which will play a crucial role in improving the overall robustness of AI systems. Additionally, the phenomenon where the prediction changes with only a small number of pixel modifications that do not alter the overall meaning can be considered an anomaly in artificial neural networks. This type of attack provides important insights into understanding the limitations of neural networks. **Proposing a Stronger Robustness Benchmark for $L_0$ (pixel) Attacks** The results of our study provide a new robustness benchmark for powerful $L_0$ (pixel) attacks, characterized by high attack success rates and minimal pixel modifications. This benchmark can be used to evaluate how robust AI systems are against such attacks and underscores the necessity for companies to develop and implement additional defensive mechanisms when commercializing AI models. **Application in Adversarial Training** The method we propose can be used in adversarial training by incorporating examples generated through pixel modifications into the training process. This approach can help develop neural networks that are more resilient to pixel-level attacks, thereby contributing to the construction of safer and more reliable AI systems in real-world environments. **Reproducing a physical issue with the camera.** Physical defects in camera sensors, such as hot pixels(255) or dead pixels(0), can impact image quality and degrade the performance of neural network models. In this paper, we employ an attack that simulates these physical issues by replacing specific pixels with values of either 0 or 255, which induces incorrect predictions by the neural network. This approach allows us to analyze the vulnerabilities of the model and contribute to the development of more robust neural networks that can withstand such defects in real-world scenarios. --- Thank you once again for your thoughtful comments. We will incorporate the discussed points into the final version.

Authorsrebuttal2024-08-12

Dear Reviewer ppjz, Thank you for reading our paper and providing comments to help us improve it. Below, we will address your concerns. --- **C) the paper acknowledges the risks. but doesn't propose how their research could actually be used to make neural nets more robust to adversarial attacks** We would like to respond to the points regarding *'how to make neural networks more robust against our attack'* and *'the contribution of our research in enhancing the robustness of neural networks against adversarial attacks.'* > **1. How to make neural networks more robust against our attack.** Our research proposes a method that achieves a higher success rate with fewer queries compared to previous query-based pixel attacks. However, our method still requires a significant number of queries, which may pose challenges for real-world applications. Consequently, we did not include a discussion on defensive strategies. Nevertheless, the following strategies could be considered to defend against our specific attack. **Defense through Query Limitation** Our method requires an average of over 1,000 queries to successfully deceive object detection models. Similarly, as shown in Table 3 of the (PDF), transformer-based models also require an average of over 1,000 queries to achieve high success rates. Therefore, limiting the number of queries to around 1,000 within a short period can effectively defend against our attack. For CNN-based models, which can be attacked with fewer queries, limiting the number of queries to 400 can effectively provide protection. **Defense through Adversarial Training** As shown in Table 5 of the (PDF), adversarial training significantly reduces the success rate of our attack and increases the number of queries required. Therefore, by training models with adversarial learning and appropriately limiting the number of queries, our attack can be effectively mitigated. > **2. Contribution to enhancing the robustness of neural networks against adversarial attacks** In real-world applications of AI models, not only performance but also the ability to defend against malicious attacks is of critical importance. Since actual attackers often attempt to compromise models without knowing their internal informations, research on various black-box attacks and the development of defense strategies are essential. This research contributes significantly to these areas in the following ways. **Highlighting the Need for Defense Against Pixel-Level Attacks and Understanding Neural Network Limitations** This research introduces a new method that enhances the success rate of black-box query-based $L_0$ (pixel) attacks while minimizing both the $L_0$ and the number of queries, an area that has been relatively underexplored. This work highlights the need for other defense research to also consider pixel-level attacks, which will play a crucial role in improving the overall robustness of AI systems. Additionally, the phenomenon where the prediction changes with only a small number of pixel modifications that do not alter the overall meaning can be considered an anomaly in artificial neural networks. This type of attack provides important insights into understanding the limitations of neural networks. **Proposing a Stronger Robustness Benchmark for $L_0$ (pixel) Attacks** The results of our study provide a new robustness benchmark for powerful $L_0$ (pixel) attacks, characterized by high attack success rates and minimal pixel modifications. This benchmark can be used to evaluate how robust AI systems are against such attacks and underscores the necessity for companies to develop and implement additional defensive mechanisms when commercializing AI models. **Application in Adversarial Training** The method we propose can be used in adversarial training by incorporating examples generated through pixel modifications into the training process. This approach can help develop neural networks that are more resilient to pixel-level attacks, thereby contributing to the construction of safer and more reliable AI systems in real-world environments. **Reproducing a physical issue with the camera.** Physical defects in camera sensors, such as hot pixels(255) or dead pixels(0), can impact image quality and degrade the performance of neural network models. In this paper, we employ an attack that simulates these physical issues by replacing specific pixels with values of either 0 or 255, which induces incorrect predictions by the neural network. This approach allows us to analyze the vulnerabilities of the model and contribute to the development of more robust neural networks that can withstand such defects in real-world scenarios. --- Thank you once again for your thoughtful and valuable comments. We will incorporate the discussed points into the final version.

Reviewer p4SQ2024-08-12

Unfortunately, I couldn't find the results that you are mentioning in the PDF.

Authorsrebuttal2024-08-13

Dear Reviewer p4SQ, It appears that there may have been an issue that made it difficult to find the results in the (PDF) attached to the global rebuttal. To help clarify the results for you, I have prepared the tables below, which consolidate the results for easy reference. --- **Table1:The results of transformer-based classifiers.** This table presents the experimental results on various transformer-based models, including VIT-L[1], Swin-V2-T[2], and Deit-B[3], compared to other attacks. The success rate is more effective when higher, and both $L_0$ and the query are lower. | Model | Attack | Succes Rate | $L_0$ | Query | |------------|----------|-----------|------|-------| | ViT-L [1] | OnePixel | 8.9% | 15 | 1654 | | | Pixle | 66.4% | 531 | 1396 | | | **RFPAR** | **78.0%** | 355 | 1042 | || | Swin-V2-T [2]| OnePixel | 5.0% | 15 | 1686 | | | Pixle | 66.8% | 1052 | 1509 | | | **RFPAR** | **69.4%** | 608 | 1096 | || | Deit -B[3] | OnePixel | 8.4% | 15 | 1137 | | | Pixle | 71.0% | 551 | 1473 | | | **RFPAR** | **84.3%** | 412 | 1161 | --- **Table 2 : The results of object detection models.** This table presents the experimental results on object detection models, ATSS[4] and Deformable DETR[5]. RM is more effective when higher, while mAP, $L_0$ and the query are lower. | Model | Attack | RM | mAP | L0 | Query | |---------------------|--------------|------|-------|------|-------| | Atss [4] | clean | - | 0.227 | - | - | | | RFPAR$_{0.01}$| 0.74 | 0.048 | 491 | 1530 | | | RFPAR$_{0.02}$| 0.88 | 0.026 | 1025 | 1633 | | | RFPAR$_{0.03}$| 0.90 | 0.026 | 1357 | 1504 | | | RFPAR$_{0.04}$| 0.91 | 0.008 | 1666 | 1243 | | | **RFPAR$_{0.05}$**| **0.92** | **0.006** | 2074 | **1288** | || | Deformable DETR [5] | clean | - | 0.339 | - | - | | | RFPAR$_{0.01}$| 0.61 | 0.170 | 333 | 1466 | | | RFPAR$_{0.02}$| 0.69 | 0.134 | 512 | 1502 | | | RFPAR$_{0.03}$| 0.72 | 0.135 | 869 | 1488 | | | RFPAR$_{0.04}$| 0.76 | 0.110 | 1200 | 1488 | | | **RFPAR$_{0.05}$**| **0.78** | **0.073** | 1274 | **1335** | --- **Table 3: The performance of RFPAR on transformer-based models with different iteration limits.** This table presents the experimental results on transformer-based models (ViT-B, L, H, Swin-V2-T, Deit-B) with iteration limits of 100 and 200. | Model | Maximum of Iteration | Success Rate | L0 | Query | |---------|-----------------|--------------|------|-------| | ViT-B | 100 | 64.1% | 211 | 613 | | | 200 | **83.4%** | 352 | 995 | || | ViT-L | 100 | 59.9% | 209 | 618 | | | 200 | **78.0%** | 355 | 1042 | || | ViT-H | 100 | 62.2% | 166 | 582 | | | 200 | **73.5%** | 229 | 917 | || | Swin-V2-T | 100 | 46.2% | 352 | 611 | | | 200 | **69.4%** | 608 | 1096 | || | Deit-B | 100 | 60.2% | 249 | 676 | | | 200 | **84.3%** | 412 | 1161 | --- I hope this helps clarify the results. Please feel free to reach out if there are any further questions or concerns. [1] A. Dosovitskiy el al., "An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale," in International Conference on Learning Representations, 2021. [2] Z. Liu et al., "Swin Transformer V2: Scaling Up Capacity and Resolution," in Conference on Computer Vision and Pattern Recognition, 2022. [3] H. Touvron el al., "Training data-efficient image transformers & distillation through attention" in International Conference on Machine Learning, 2021. [4] S. Zhang et al., "Bridging the Gap Between Anchor-Based and Anchor-Free Detection via Adaptive Training Sample Selection," in Conference on Computer Vision and Pattern Recognition, 2020. [5] X. Zhu et al., "Deformable DETR: Deformable Transformers for End-to-End Object Detection," in International Conference on Learning Representations, 2021.

Program Chairsdecision2024-09-25

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC