Examining the impact of augmented reality on students’ learning outcomes

Research design

This study used a mixed-methods research design combining quantitative and qualitative approaches. This approach aims to thoroughly understand how AR affects educational outcomes by measuring learning performance changes and gathering teachers’ and students’ subjective experiences and perspectives²⁹. The quantitative component involved conducting evaluations before and after the intervention to measure changes in learning outcomes. Conversely, the qualitative aspect included conducting interviews and focus groups to explore users’ AR experiences and gain more profound insights.

Utilizing a mixed-methods research design was deemed appropriate for this study because it employed data triangulation, augmenting the findings’ validity and dependability. By integrating quantitative and qualitative data, the study offered a more intricate comprehension of the interplay between implementing augmented reality, user perceptions, and learning outcomes.

Procedures

Before implementing the study, a pretest was given to both groups. This evaluation aimed to evaluate the students’ basic knowledge and abilities in the studied topic area. In order to conduct the study, the researcher selected a representative sample of students from the intended population. The participants were assigned to either the experimental or control groups randomly in order to reduce the potential for selection bias. The researcher created augmented reality (AR) learning modules and exercises specifically for the experimental group for seven weeks (refer to Appendix A).

For the first week, the goal was to improve vocabulary acquisition by utilizing interactive augmented reality (AR) experiences. The teacher develops an augmented reality (AR) application that uses graphics and precise explanations of vocabulary words in a real-life environment. The students were instructed to use the augmented reality (AR) application to familiarize themselves with vocabulary terms related to animals. The students engage with augmented reality features to access word definitions and sample phrases about animals.

For the second week, the students were taught grammar exercises, and the goal was to enhance comprehension of grammatical structures using augmented reality exercises on different animals. The teacher created augmented reality scenarios that showcase different animals, and students were required to respond to questions like “What is this (pointing to an animal). The students utilize augmented reality to engage with scenarios that emphasize grammar. The teacher supervises student improvements while they engage with augmented reality content, and offers assistance as necessary.

For the third week, reading was the focus, while the fourth week focused on writing. AR was used to design tasks and activities for these weeks. The fifth and sixth weeks focused on storytelling and pronunciation exercises, while the students were assessed in the seventh week. The curriculum integrated these elements to enrich the learning experience by incorporating interactive and immersive features.

The control group received instruction through traditional teaching techniques and materials, but did not incorporate Augmented Reality (AR) technology. While the experimental group learned grammar topics using an interactive AR module, the control group was taught the same content using traditional methods such as textbook exercises and instructor-led lectures. Following the intervention period, both groups were given a post-test. The purpose of this exam was to ensure that it had the same format and content as the pretest and to accurately assess any changes in learning outcomes. The researcher analyzed the pre-test and post-test scores of both the experimental and control groups to evaluate the influence of AR on learning outcomes.

Participants

The participants were selected from two universities that used augmented reality (AR) in their educational programs for three years. The participants were undergraduate students between the ages of 19 to 24 years. The study invitation was distributed through email and direct communication with the institution’s administration. The objective was to target teachers and students who have actively incorporated augmented reality (AR) into their educational activities for years to obtain pertinent and significant insights. On this premise, the study adopted the following inclusion and exclusion criteria:

• Only students currently enrolled in the participating universities participated in the study.

• Only students and teachers with at least one year of exposure to AR in educational activities were allowed to participate in the study.

• The teacher has used AR to design classroom tasks in their previous teaching.

• Participants with no prior AR exposure were not allowed to participate.

• Those with cognitive or visual impairments that would prevent interaction with AR tools were excluded from the study.

The study used a stratified purposive sampling strategy to recruit participants across the two universities with direct experience with augmented reality (AR) in their educational settings. Diversity in academic discipline and year of study was ensured to reflect a wide range of user experiences with AR. This non-probability sampling technique is suitable, as it permits the selection of participants with abundant information, hence enabling comprehensive insights into the study inquiries. The sample comprises four teachers and 84 students, guaranteeing various experiences and perspectives.

Before recruiting these participants, an a priori power analysis was conducted using GPower 3.1 to determine the appropriate sample size for comparing two groups with a medium effect size (d = 0.5), alpha = 0.05, and power = 0.80. The analysis suggested a minimum sample size of 64 participants, whereas we had a sample size of 88 participants, sufficient to conduct this study. For the interviews, 45 students were randomly recruited from the experimental group, and four teachers were to share their experiences while using AR for educational purposes. Randomization was achieved using a random number generator to select 45 students from the experimental group for interviews, ensuring equal distribution across departments and academic levels.

Prior to commencing data collection, all participants were provided with comprehensive information regarding the study’s objectives, methodologies, and advantages. The consent forms containing guarantees of confidentiality, the voluntary aspect of participation, and the entitlement to withdraw from the study without facing any repercussions were administered to the participants to obtain their consent before commencing the study.

Data collection

Standardized exams were used to acquire quantitative data. The students underwent pre- and post-intervention examinations to assess changes in learning outcomes following the integration of AR into their classes. The tests concentrate on the specific aspects where Augmented Reality (AR) was applied, evaluating the ability to retain knowledge on English language related topics (see Appendix B).

Qualitative data was collected using semi-structured interviews with teachers and focus group discussions with students (see Appendix C). For anonymity purposes, the students were coded as D1 to D45 and the teacher as S1 to S4. The teacher interviews delve into their encounters with augmented reality (AR), their obstacles, and perceived effects on their teaching methods and student learning. The focus group conversations with students aim to investigate their level of involvement with augmented reality (AR), their perception of the mental effort required, and the impact of AR on their collaborative learning experiences. The interviews and focus groups were recorded using audio equipment, and their comments were transcribed exactly as they were said.

Data analysis

The data was analyzed using statistical software, specifically SPSS. The quantitative data obtained from the pre- and post-tests were analyzed using paired sample t-tests to ascertain any statistically significant disparities in student learning outcomes before and after the augmented reality (AR) intervention. Efforts were made to control for potential confounders, such as prior exposure to similar technologies, gender distribution, and academic level. Baseline equivalence between the experimental and control groups was tested during pretests. Future studies should account for the instructor’s teaching style and technological proficiency.

The qualitative data obtained from interviews and focus groups were subjected to thematic analysis, employing the six-step process described by Braun and Clarke³⁰. Two coders were engaged in the coding process. The theme analysis concentrated on discerning shared characteristics and distinctions in the viewpoints of educators and learners on AR. Additionally, it explored any developing obstacles or advantages not reflected in the numerical data.

The transcripts were carefully examined and analyzed multiple times to fully comprehend the dataset. The key phrases and concepts were rigorously encoded throughout the dataset and recorded using a codebook. The codes were categorized into overarching themes representing recurring data themes. The themes were carefully examined and modified to reflect precisely the facts. The themes that did not fit were discarded after member checking. Precise definitions and specific names were established to represent each theme accurately. The emerging ideas and themes were integrated into a cohesive narrative addressing the study’s inquiries.

Ethical considerations

The research adhered to ethical standards to maintain the privacy and confidentiality of participants. Informed consent was secured from all participants before the study. The authors confirm that this work is original and has not been published elsewhere, nor is it currently under consideration for publication elsewhere. Before initiating the study, ethical approval was obtained from the ethics committee of Akdeniz Karpaz University (ref: AKUN/ETC/0322/2024). After reviewing the proposal, the ethical committee approved the proposal, as the methods and procedures align with ethical rules and regulations for scientific research. The ethical committee confirmed that the procedures used in this study adhere to the tenets of the Declaration of Helsinki of 1964.

Human ethics and consent to participate

Written informed consent was obtained from the participants by the corresponding author, where they declared their willingness to participate or withdraw from the study. The participants provided consent to the authors to publish the conclusions of the research in journals and conference proceedings. They provided consent for the data derived from this study to be used for research purposes and shared with the consent of the authors. The participants were informed that their identities would be protected by anonymizing their identities, as the research would not pose any harm to them.

Findings

RQ1: How does using Augmented Reality (AR) in the classroom impact students’ learning outcomes compared to traditional instructional methods?

Table 2 presents the pretest scores of the participants before commencing the study. The table shows that a total number of 95 students participated in the pretest. Furthermore, from the table, it can be deduced that 84 out of 95 students obtained scores between 0 and 49, indicating that 88% of the students obtained a fail grade. On the other hand, the table indicates that 11 out of 95 students, which represents 11.5% of the total number of students, scored between 50 and 100. The table further shows that the mean score of the 95 students was 41.6, while the standard deviation (SD) was 10.2, indicating moderate variability in performance. Only students who scored below 50 (n = 84) were included in the experimental/control group analysis for the intervention phase.

Table 3 presents the post-test results across groups. The post-test findings demonstrate a statistically significant disparity between the experimental and control groups. The t-test yielded a significant result (t (84) = 5.14, p < 0.001), indicating a statistically significant difference between the groups or circumstances being compared. The t-statistic is 5.14, which signifies the quotient of the discrepancy between the sample means and the standard error of the difference. The p-value is less than 0.001, indicating that the likelihood of finding a t-value as severe as 5.14 (or more extreme) under the null hypothesis is exceedingly small (less than 0.1%). Compelling evidence indicates a significant distinction between the groups or conditions under comparison. Considering the significant t-value and the extremely small p-value, it is probable that the effect or difference being assessed is substantial. These findings indicate that using augmented reality (AR) significantly improved learning outcomes, as evidenced by the experimental group outperforming the control group in test scores.

Table 4 presents the improvement in learning outcomes across the two treatment groups. The t-test yielded a result of t (84) = 4.73, p < 0.001, indicating a significant statistical difference between the groups or conditions being compared. The calculated t-statistic is 4.73. This value shows the standardized difference between the means of the sample. A higher t-value signifies a greater degree of significance than the variation in the data. The p-value is below 0.001, indicating a likelihood of less than 0.1% that the observed difference, or a more extreme one, is due to random chance if the null hypothesis is valid. The result is deemed statistically significant because the p-value is significantly lower than the conventional significance criterion, such as 0.05. The t-test result, t (84) = 4.73, p < 0.001, indicates a substantial dissimilarity between the compared groups or circumstances, suggesting a statistically significant distinction between them.

The average improvement scores demonstrate that students in the experimental group, who utilized augmented reality (AR), exhibited considerably larger advancements in learning outcomes in comparison to the control group. The significant increase in performance observed in the experimental group provides further evidence of the efficacy of augmented reality (AR) as a tool for promoting educational outcomes. This finding underscores the potential of AR to promote a more profound comprehension and long-term retention of academic material.

RQ2: How do students perceive the use of AR technology in their learning experience?

Table 5 is a thematic analysis of students’ perception of using AR technology. The table shows students’ perceptions of using AR technology. It further indicates that three themes emerged: engagement and enjoyment, cognitive load, and collaborative learning. The analysis is discussed in themes.

Theme 1

Engagement and Enjoyment.

This theme shows the student involvement and pleasure of using augmented reality for educational purposes. Students described the AR-enhanced sessions as highly dynamic and enjoyable, stimulating active class engagement. Here are some of the comments that reflect this theme.

It was engaging. D11.

I loved using the AR gadget because it was fun to use D8.

This made lessons enjoyable. D3.

This theme represents the students’ favorable emotional and motivational reactions to learning that are based on augmented reality (AR). The interactive and immersive aspect of AR enhanced the engagement and enjoyment of classes, hence fostering students’ active participation in the learning process. Here are some of their comments.

The lessons were engaging and lovely. I liked it, D17.

The increased level of involvement was notably evident in students who generally demonstrated lower levels of engagement in traditional lectures.

Theme 2

Cognitive Load.

The concept of cognitive load pertains to the first difficulty students ’encounter when engaging with augmented reality (AR). This theme describes students’ mental challenges when initially introduced to Augmented Reality (AR). Because of the intricate and interactive nature of AR, the students originally felt overwhelmed. Nevertheless, most individuals adjusted and eventually acknowledged that the technology ultimately facilitated their comprehension of challenging subjects. Below are some of their responses that reflect this theme:

Initially, it was too much, but as the fun level increased, I enjoyed it more. D22.

It wasn’t easy to do classwork and use the AR, but it was fun after all. D1.

Some students first felt overwhelmed by the intricate visual and interactive components of AR. Here is another comment:

It wasn’t easy to comprehend the numerous buttons and navigate the device. But slowly, it became easier. I think this has to do with technology. D20.

However, most participants adjusted and discovered that augmented reality (AR) enabled them to comprehend intricate ideas better once they became familiar with the technology.

Theme 3

Collaborative Learning.

This theme exemplifies the influence of augmented reality on promoting cooperative learning. According to students, using AR improved group discussions and teamwork by promoting collaborative problem-solving and peer learning. Here are some comments from the participants:

We worked in groups using the device. This was good for us. D8.

Using this gadget enabled us to work collaboratively, especially in executing the group work. D19.

The interactive aspect of augmented reality (AR) tools facilitated active engagement and meaningful contributions from students during group work. Their comments are stated below:

It was a good way to make learning meaningful and engaging. D27.

Normally, I will not talk during lessons, but this gadget made me interact more during lessons. D42.

The technology promoted engagement, collaborative learning, and collective problem-solving, thereby establishing a cooperative classroom atmosphere. Students valued the collaborative learning opportunities provided by AR, which frequently resulted in more in-depth conversations and improved group dynamics.

Using the device made us discuss more. The outcome is good. D39.

In general, the themes indicate that the students in the experimental group expressed a significantly captivating and pleasurable learning encounter using AR, which they perceived as more interactive than conventional approaches. While certain students initially faced a greater cognitive burden, this difficulty gradually decreased as they grew more accustomed to the augmented reality (AR) equipment. In addition, augmented reality (AR) promoted collaborative learning by improving group relationships and problem-solving skills, as students have reported. These qualitative observations enhance the quantitative data by explaining why the experimental group achieved better results than the control group in the post-test.

RQ3: What are teachers’ perceptions of incorporating AR technology into their teaching practices?

Table 6 is the thematic analysis of teachers’ perceptions of incorporating AR technology into their teaching practices. The table shows teachers’ perceptions of incorporating AR technology into their teaching practices. It indicates that three themes emerged from analyzing the teachers’ transcripts: perceived usefulness, perceived ease of use, and instructional impact. The themes are discussed below.

Theme 1

Perceived usefulness.

This theme emphasizes the claimed educational advantages of AR in terms of enhancing comprehension of intricate concepts and enhancing information retention. The teachers observed that AR’s capacity to visualize abstract concepts was especially advantageous, enhancing the relevance and comprehensibility of lessons. Here are a few comments from them that demonstrate this theme:

I think it is easier to understand some concepts using AR. This made lessons easier to deliver and understand for the students. S2.

Normally, the students would ask lots of questions, but given that they used AR, the questions were reduced. This suggests that they understood more easily than before S4.

This theme encapsulates the instructional significance that teachers ascribed to Augmented Reality (AR). The capacity of augmented reality (AR) to facilitate comprehension was a frequently mentioned aspect of commendation. Teachers saw that students who used augmented reality (AR) had improved understanding and memory, especially in areas that are complex.

Understanding key conversations in English is always a difficult aspect of teaching speaking. However, using AR made this easier. S1.

The students may ask questions when teaching some language skills, but it was easier while using AR.S3.

These were some of the comments of the teachers that reflect this theme.

Theme 2

Ease of Use.

This theme describes the practicality of augmented reality (AR) tools as perceived by teachers. Most participants reported that AR tools were easy to use once they overcame the initial learning curve. Following the initial training, the majority of participants deemed AR to be user-friendly. However, they encountered difficulties when using it at the beginning of the study. These are their comments:

It was not easy to use before, and I know I have used it before. This was difficult to figure out, but later, it was easy to use S1.

Using it initially was difficult, but the technical team helped a lot. S3.

Initially, it was not easy to use, but it became easier when I received some help from the technical team. S2.

Generally, it may be difficult to use initially, but it becomes easy afterwards. S4.

Some of the teachers’ comments reflected their opinions on using AR in their teaching practices.

Theme 3

Instructional impact.

The theme implies that the instructor believed integrating AR technology into their teaching methods affected instructional delivery and student learning outcomes. They were taught utilizing augmented reality, and this enhanced their performance. Here are some of their comments:

Of course, they understood more and answered difficult questions, so this must impact their grades. S3.

The student’s performance improved, and I think this is because of the use of AR S2.

Similarly, another added,

AR impacted their performance this semester. They performed better than the previous semester. S2.

Another narrated the experience that made her think that using AR impacted the student’s performance. Here is her narration:

Before, it was not easy to make the students understand the simple basics of language. I remember that the students struggled a lot during lessons. Then we decided to use this AR for lessons, and first, their moods changed. They started using it and loved it. Afterwards, they developed optimism about learning and attended the AR labs more. This also showed in their performance in the examinations. This is good. S1.

Some of the participants’ comments reflect these themes. The qualitative and quantitative data are discussed below.