Boosting Semi-Supervised Scene Text Recognition via Viewing and Summarizing

Summary

This paper proposes a ViSu framework to enhance text recognition by viewing and summarizing. For the viewing process, the authors generate various text images to lead the model to focus on different text styles. For the summarizing process, they analyze the drawback in existing character consistency loss and propose a CUA loss to cluster the character under different styles. As a result, the proposed framework outperforms other existing methods on multiple datasets.

Strengths

1. The design of OGS can obtain promising improvements without additional human cost. 2. The authors point out that previous consistency loss mistakenly regards some positive samples as negative samples, which is an important theoretical drawback in existing metric learning methods. And the proposed CUA loss provides a solution for this issue. 3. The proposed method gets SOTA performance on both common and challenging datasets. 4. The proposed framework can be applied on multiple text recognition models seamlessly.

Weaknesses

1. To further verify the effectiveness of OGS. Authors should compare the performance of replacing OGS with normal synthetic data. 2. Tab.5 lacks the comparison with the baseline model. 3. In Tab.5, it seems that MAERec-B performs better on the Union-14m dataset.

Questions

1. As the semi-supervised framework can use large-scale unlabeled real data for training, why does ViSu require OGS to generate synthetic data? 2. To verify the fairness, do other semi-supervised methods in Tab. 1 and 5 have the same implementation details as ViSu?

Rating

7

Confidence

5

Soundness

3

Presentation

3

Contribution

3

Limitations

Please refer to Weaknesses.

Summary

This paper focuses on the character morphologies and proposes to boost the scene text recognizer through viewing and summarizing paradigm. In the viewing process, the Mean Teacher framework is used to train with unlabeled data. In the summarizing process, the proposed method theoretically proves the mistakes in the previous method and proposes a new loss function. I have read the author responses and the comments of other reviewers, I would recommend the 'accept' score.

Strengths

1. The proposed method achieves good results, especially on challenging benchmarks. 2. The paper is well written. It clearly points out the problem in the previous method and provides a detailed explanation. The proposed OGS and Mean Teacher framework effectively increase the diversity of character glyphs. The CUA loss corrects the previous error and better achieves clustering of identical characters. 3. There are lots of experiments to prove the effectiveness of the proposed method.

Weaknesses

1. There is a lack of clear description in the paper on how CRNN-ViSu and TRBA-ViSu in Table 1 are set up and trained. 2. Although the proposed method achieves high average accuracy on common benchmark and Union14B-Benchmark, it is not SOTA on certain benchmarks, such as SVT, SVTP, CUTE, Curve, Contextless, MultiWords. 3. When the proposed method recognizes text with vertical reading order, how to decide whether to rotate it 90 degrees clockwise or counterclockwise.

Questions

See Weaknesses.

Rating

7

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

The authors adequately addressed the limitations and potential negative societal impact

Summary

Existing scene text recognition (STR) methods struggle to recognize challenging texts, which originates from the insufficient exploration of character morphologies. To address the issues, the paper proposes to facilitate the contrastive learning-based STR framework in a self-motivated manner by leveraging synthetic and real unlabeled data without any human cost. An Online Generation Strategy is proposed to enrich the diversity of characters in training data. Besides, a new Character Unidirectional Alignment Loss is proposed for aligning the character features in the student model with the reference features in the teacher model. Extensive experiment results show the effectiveness of the proposed method.

Strengths

1) The paper proposes to improve the recognition for challenging texts, especially for artistic and severely distorted characters within a mean-teacher framework without the requirement for real labeled data. 2) An Online Generation Strategy is proposed to enrich the diversity of characters in training data without need of psuedo labeling real data explicitly. 3) A Character Unidirectional Alignment Loss is proposed to improve the existing Character Contrastive (CC) Los [45] in character representation learning. 4) Extensive experiments on Common Benchmarks, Union14M-Benchmark, and other challenging benchmarks demonstrate the effectiveness of the proposed method.

Weaknesses

1) The overall framework follows a mean teacher framework, which is not new for semi-supervised scene text recognition. 2) In Table 1, the baseline model without RU data has achieved a high performance than existing SOTA. Is there any special design in the baseline model? 3) Why not simply remove the second item in equation, i.e., exclude the positives in the denominator? 4) Absense of comparison between the proposed method and other methods in Sec. 2.3.

Questions

Please refer to the weakness part.

Rating

6

Confidence

4

Soundness

2

Presentation

3

Contribution

3

Limitations

Yes. The limiation is well descirbed in the paper.

Summary

This paper addresses the problem of insufficient exploration of character morphology in scene text recognition and proposes a new framework comprising an Online Generation Strategy (OGS) and Character Unidirectional Alignment (CUA) Loss to enable the model to learn from unlabeled real data. OGS mitigates the issue of data scarcity, while CUA aids in clustering characters. Comprehensive experiments demonstrate the effectiveness of the proposed method.

Strengths

1. The proposed method effectively addresses the issue of excessive representation methods for scene images through Unified Representation Forms. 2. The errors in the previous Character Contrastive Loss method are corrected. 3. The idea of utilizing character morphology is novel.

Weaknesses

1. The writing quality needs improvement. 2. There is a lack of detailed explanation regarding the Online Generation Strategy (OGS) and Unified Representation Forms (URF).

Questions

The proposed method struggles with handling images with extreme aspect ratios. The authors attempt to address this by splitting the image, recognizing each part, and then concatenating them. However, this splitting operation may interfere with character recognition and degrade the overall recognition quality.

Rating

6

Confidence

3

Soundness

3

Presentation

2

Contribution

3

Limitations

N.A

Official Comment by Reviewer Smtd

Thanks for the response by the authors. All my concerns have been properly addressed. After considering the rebuttals for all the reviews, I'd like to raise the score to WA.