A. Data Collection Data were collected through both qualitative and quantitative instruments aligned with each phase of the SAM model and the corresponding research questions: 1) Preparation Phase: Needs analysis through expert interviews and stakeholder focus group discussions (FGDs) to define goals, chatbot features, and alignment with SRL strategies in academic writing. 2) Iterative Design Phase: Design documentation, wireframes, and mockups were developed and reviewed. Feedback from instructional designers and writing educators was collected through design review sessions. 3) Iterative Development Phase: Refinement of the low-fidelity mockup was informed by design review sessions with a small group of users to gather students’ perceptions and by usability testing. The data were collected through structured usability testing, and an FGD was conducted to assess students' perceived ease of use and usefulness, in line with the Technology Acceptance Model (TAM). Data on students’ presage variables were assessed with: 1) Prior knowledge in academic writing: students’ prior knowledge was collected using multiple-choice questions. 2) Pre-SRL attribute: students’ SRL were collected using a Likert-scale self-report survey with 3 SRL indicators based on Schraw, Kauffman, and Lehman (2006): 1) Cognitive, 2) Motivation, 3) Metacognitive [8]. The effectiveness of Beementor was indicated by students’ academic writing skills and satisfaction with the user experience. Academic writing skills were assessed using the PTE Academic Essay Score rubric, which was validated by experts and lecturers. The user experience data were collected via a Likert-scale questionnaire within a three-layer framework comprising functional, emotional, and outcome aspects. B. Sample The study involved three categories of participants. Expert participants, including 4 experts, instructional designers, AI developers, and academic writing instructors, participated in interviews and design reviews to inform the chatbot development process. A small-group test was conducted with 7 third-year university students. Meanwhile, 62 undergraduate students participated in the chatbot's usability testing and pilot implementation in the Bahasa Indonesia and Academic Writing Courses. All participants provided informed consent, and ethical clearance was obtained before data collection. C. Study Context and Intervention The pilot implementation was conducted in the Bahasa Indonesia course. The Bahasa Indonesia course introduced students to academic writing as part of university-level language and communication studies. The intervention was conducted during the academic essay writing unit. Students used Beementor during three guided class meetings as they learned to write academic essays. During these meetings, the chatbot was introduced and used to support goal setting, essay planning, idea development, drafting, feedback interpretation, and revision. After the guided sessions, students were allowed to use Beementor independently until the end of the semester to support the completion of course-related academic writing tasks.
Paper
The full text of this publication is not hosted on 44B due to licensing.
Read it at OpenAlex