Generative AI (GenAI) systems operate across diverse application domains where quality priorities shift dynamically in response to user expectations and contextual requirements. This variability calls for a comprehensive quality model that enables stakeholder-driven weight recalibration to support product evaluation and selection. However, existing approaches do not simultaneously account for GenAI-specific attributes, user-centric quality priorities, and domain-adaptive evaluation mechanisms. To bridge this gap, this study proposes the User-Centric Generative AI Quality Model (UC-GAIQM), a domain-adaptive framework in which Analytic Hierarchy Process (AHP) weights can be recalibrated to reflect quality priorities across different application scenarios and user profiles. The proposed model was developed through a mixed-methods, three-phase research design. In the first phase, a Systematic Literature Review (SLR) and Multivocal Literature Review (MLR) established the theoretical foundation. In the second phase, a quantitative survey of active GenAI users (<inline-formula> <tex-math notation="LaTeX">$n = 111$ </tex-math></inline-formula>) validated eight quality dimensions through exploratory and confirmatory factor analysis (<inline-formula> <tex-math notation="LaTeX">$\alpha = 0.94$ </tex-math></inline-formula>, KMO = 0.88, CFI = 0.943). In the third phase, a three-round expert-driven Delphi study confirmed the structural validity of the model (Kendall’s <inline-formula> <tex-math notation="LaTeX">$W = 0.84$ </tex-math></inline-formula>), and an AHP study demonstrated the weight recalibration mechanism. UC-GAIQM comprises eight quality dimensions and thirty sub-dimensions aligned with key ISO/IEC standards, the NIST AI Risk Management Framework, and the EU AI Act. The results demonstrate that the proposed model facilitates dynamic, context-sensitive evaluation of GenAI products by enabling quality priority adaptation across application domains.
Paper
Full text
A User-Centric Domain-Adaptive Quality Model for Benchmarking Generative AI Systems
Semantic Scholar · 2026
Abstract
Generative AI (GenAI) systems operate across diverse application domains where quality priorities shift dynamically in response to user expectations and contextual requirements. This variability calls for a comprehensive quality model that enables stakeholder-driven weight recalibration to support product evaluation and selection. However, existing approaches do not simultaneously account for GenAI-specific attributes, user-centric quality priorities, and domain-adaptive evaluation mechanisms. To bridge this gap, this study proposes the User-Centric Generative AI Quality Model (UC-GAIQM), a domain-adaptive framework in which Analytic Hierarchy Process (AHP) weights can be recalibrated to reflect quality priorities across different application scenarios and user profiles. The proposed model was developed through a mixed-methods, three-phase research design. In the first phase, a Systematic Literature Review (SLR) and Multivocal Literature Review (MLR) established the theoretical foundation. In the second phase, a quantitative survey of active GenAI users (<inline-formula> <tex-math notation="LaTeX">$n = 111$ </tex-math></inline-formula>) validated eight quality dimensions through exploratory and confirmatory factor analysis (<inline-formula> <tex-math notation="LaTeX">$\alpha = 0.94$ </tex-math></inline-formula>, KMO = 0.88, CFI = 0.943). In the third phase, a three-round expert-driven Delphi study confirmed the structural validity of the model (Kendall’s <inline-formula> <tex-math notation="LaTeX">$W = 0.84$ </tex-math></inline-formula>), and an AHP study demonstrated the weight recalibration mechanism. UC-GAIQM comprises eight quality dimensions and thirty sub-dimensions aligned with key ISO/IEC standards, the NIST AI Risk Management Framework, and the EU AI Act. The results demonstrate that the proposed model facilitates dynamic, context-sensitive evaluation of GenAI products by enabling quality priority adaptation across application domains.