Meta-review
This paper proposes Adaptive Self-Supervised Learning Strategies (ASLS) for the dynamic, on-device personalization of large language models (LLMs). The authors claim that ASLS leverages self-supervised learning for real-time adaptation through a user profiling layer and a neural adaptation layer. The goal is to personalize LLMs for individual users while minimizing resource demands. The paper evaluates its method across multiple datasets, reporting improvements in user engagement and satisfaction.
The primary strength of this submission lies in addressing a timely and practically significant challenge—on-device LLM personalization. This problem aligns with emerging demands for privacy-preserving and resource-efficient AI. The authors claim to use innovative self-supervised techniques for real-time fine-tuning, and they attempt to validate their approach with experiments.
However, the weaknesses of the paper outweigh its strengths. First, there are critical issues in the clarity and rigor of the methodology. The method section is poorly written, with inconsistent notation, a lack of illustrative diagrams, and vague explanations of the proposed layers. Several reviewers noted confusion regarding how the self-supervised learning aspect was implemented and how it supports real-time on-device applications. Furthermore, the core contribution appears to be incremental and closely resembles prior work, such as the HYDRA framework, with insufficient clarity on the novel aspects of the method.
The experimental evaluation also raises significant concerns. The comparisons are not rigorous, with no clear rationale for using different datasets across baselines and no inclusion of state-of-the-art benchmarks. Additionally, the evaluation metrics, such as "engagement score" and "satisfaction rate," are not well-defined, leaving the results open to interpretation. Reviewers also criticized the experimental presentation, noting that tables were difficult to interpret and lacked explanation.
While the problem itself is promising, the paper falls short in presenting a robust and innovative solution. It lacks the theoretical clarity, methodological rigor, and experimental depth required for acceptance.
The primary reasons for recommending rejection are the lack of clarity and novelty in the proposed method, the insufficient and poorly explained experimental results, and the weak comparisons with relevant baselines. The authors need to address these issues comprehensively in a future submission to make their work competitive for publication.
Additional comments on reviewer discussion
During the rebuttal period, the authors attempted to address concerns raised by reviewers but provided limited clarification. Reviewer PDiu pointed out the similarity to prior work and asked for a clearer explanation of the method's novelty, which was not sufficiently addressed. Reviewer vEb2 highlighted the lack of metric definitions and experimental clarity. While the authors responded with additional explanations, the fundamental issues of unclear methodology and insufficient experimental justification remained unresolved. Reviewer uQnh's concerns about the experimental setup and baseline selection were also not adequately addressed.
The rebuttal did not provide compelling evidence to change the reviewers' initial assessments. The lack of clarity and rigor persisted in both the methodology and experiments, and the authors' responses failed to resolve the major concerns. As Area Chair, I weighed the reviewers’ assessments heavily, as they were consistent and well-supported.
Given these considerations, I concur with the reviewers' overall ratings and recommend rejecting this submission. The authors are encouraged to substantially revise their paper by addressing the methodological and experimental weaknesses and resubmitting to a future venue.