Comparison of remote learning methods to on-site teaching -randomized, controlled trial | BMC Medical Education

The COVID-19 pandemic in 2020 permanently changed learning challenges and opportunities around the world by forcing schools and universities to fully transition to distance learning in just a few months [1,2,3,4].

Now, after the pandemic, distance learning methods seem to have become a permanent part of our education also in the fields of health care, medicine, and nursing [5, 6]. However, overall satisfaction with remote teaching has been low, especially among those who are not used to the methods and face challenges in technology competence [7]. In addition, there is considerable variation in the quality and accuracy of freely available online teaching materials, which reveals a need for developing high-quality materials (videos) with scientifically approved content for health care learning [8]. Similarly, there is a lack of controlled studies on the efficacy of distance teaching for producing sufficient learning, especially in clinical diagnostics.

Despite the at least partial shift to distance learning, the principles of learning have not changed. Motivation to learn remains vital and should be promoted, for example by the MUSIC model constructed by Professor Jones in 2016. The model is based on five valid dimensions underlying the student’s motivation: eMpowerment, Usefulness, Success, Interest and Caring [9,10,11]. Mnemonics have similarly been proven to serve as effective promotors of learning even though they are no longer widely used [12].

In this study, we wanted to promote the identification of breathing problems in children and to study the effectiveness of new distance teaching methods, vodcast and podcast, compared to live and streamed teaching. We chose the topic because learning the mechanism of breathing is multidimensional and complicated in practice and especially remotely [13, 14]. Breathing is affected by several physiological principles [13] with different pathophysiological etiologies causing similar breathing patterns. Tachypnea, for example, can be due to respiratory and metabolic reasons or hyperventilation. Failure to recognise breathing difficulties in time may lead to prolonged and unnecessary suffering, particularly among young children unable to express their symptoms verbally [14].

Vodcast

As early as the turn of the millennium, it has been shown that video recordings produced with high quality can be useful in learning [15]. Currently, this teaching method utilising video is referred to as a vodcast, which is a video recording that is watchable and listenable on a smart device and thus makes repetitive learning possible. Medical students seem to appreciate the use of videos as supplementary material, but not as a compensatory learning method [16, 17]. In this study, we used the term vodcast, by which we mean a video recording.

Several studies have suggested video recordings to be a higher quality pre-material for on-site learning in comparison with textbooks and conventional online learning methods [17, 18]. However, students have preferred more on-site teaching to video recordings when used as an exclusive teaching method, despite comparable immediate learning results [16].

Podcast

A podcast is a digitally implemented audio file that can be uploaded to a smart device by the listener or listened to directly from a playback service [19,20,21].

While podcasts are frequently used in medical education, there is sparse data on the efficacy of podcasts in medical learning. A recent Canadian study found no significant difference in learning by podcast or blog writing (blog) [22], whereas a study from North America argued that surgical nurse students attending traditional classroom lectures scored higher in exams than those who had only listened to prerecorded class lectures through an iPod™ pocket music player device [23]. However, the frequency of using the iPod™ improved the nurse students’ self-reported skills [23].

Promoting learning by new teaching strategies in classroom teaching: team, activating and multimedia teaching

In addition to new learning methods (vodcast and podcast), on-site team teaching has also received good feedback. In this method, two teachers give a lesson together in the same room. The method has been recently shown to be an efficient way to engage students [24, 25] by making lessons more interesting, stimulating, and informative compared to lessons given by one teacher [24]. Better contact between teachers and students in team teaching lessons results in active conversations on the lesson topic and makes teachers more receptive to critical review [24, 25]. Likewise, activating learners with short exercises and activities during the lesson seems to play a supporting role in learning [26].

Another way to promote long-term learning is to provide information in multiple ways during a single lesson. This sort of multimedia learning involves providing combinations of content in different formats such as text, audio, images, animations, or videos in a single interactive presentation. While this is an effective learning method, it can also overwhelm the learner with cognitive processing requirements. The cognitive theory of multimedia learning (CTML) relies on an understanding of how people learn from words and images, which is particularly important in the context of medical education [24, 25, 27].

Streamed teaching

Streamed teaching means that the learners follow live classroom teaching via the internet either simultaneously or later using a video recording. This method has gained popularity in recent years, although research data showed controversial results on the effect of the method on students’ academic performance already before the COVID-19 pandemic. An Australian review that included 71 articles (2016) on the learning of high school students showed that the impacts on learning outcomes were not positive. The results were evaluated within four result-efficient dimensions: academic, social, psychological and teacher’s mediation [28].

However, live-streamed learning methods may have some positive outcomes, as evidenced in two Canadian studies. In a study by Grafton-Clarke and colleagues (2022), live streaming was used in medical education in the context of a real-life patient interview situation. The students evaluated the method in terms of quality and found it effective (9.7/ 10 points) [29]. Another Canadian study found that an innovative (POV = point of view) live-streaming tool was more effective compared to video learning [30].

Objectives

The purpose of the present study was to compare the learning results between online and classroom teaching methods in teaching health care students to identify and treat breathing difficulties in children.

In this study, we wanted to combine team teaching with the new methods and create modern vodcasts, podcasts and activating lessons, and to compare their effectiveness. We chose a child’s respiratory distress lesson for our approach as it is a challenging task even for experienced physicians [14, 31]. Identifying a child’s breathing difficulties requires repeated learning that challenges educational resources.

As far as we know, there are no studies in the medical education field comparing vodcast, podcast and live-stream didactics as learning methods, at least in the context of learning to identify respiratory distress in child patients. Thus, we deemed it relevant to compare the efficacy and experience of such learning methods for the identification of acute respiratory distress in children.

Study design

The study consisted of eight separate teaching events organised during the academic year 2021–2022. Each event included 40 students randomly assigned to four small groups (n = 10) that received teaching by different methods: teaching in the classroom (live), streamed classroom teaching (live-stream), audio recording (podcast) and video recording (vodcast). Each student participated in the lesson only once.

The study design and participant flow are described in Fig. 1. Prior to the lesson, all of the participants took a baseline test measuring their knowledge of respiratory distress in children. After the test, teaching was performed simultaneously in the four teaching groups. At the end of the lesson, each group retook the same test (the final test). The total duration of the baseline test, teaching event and final test was 75 min. Five to seven weeks later, long-term learning was evaluated by repeating the test once more.

Participants and methods

Participants

A total of 325 students (175 medical students, 150 nurse students) were recruited to participate in the trial. All medical students were in their fifth year of studies at the University of Eastern Finland and participated in an eight-week paediatric course implemented between August 2021 and May 2022. Additionally, paramedic nurse (annual courses from one to four) students from the Savonia University of Applied Sciences in Kuopio (Finland) participated in the study during their paediatric course at the same time as the medical students. Both medical and nurse student groups were invited to participate in this randomized controlled trial. In this study, we refer to these professional groups as medical students and nurse students.

The participants were randomized into four groups: classroom teaching, streamed teaching, podcast and vodcast groups (Fig. 1.). A total of 24 students taking the baseline test did not give their permission to participate in this research or their data could not be combined at the analysis stage based on their identification codes. The participation rate was 93%.

All fifth-year medical students (n = 175) and all nurse students from each of the four classes (n = 150) were asked to participate in the study. Participation in the study was voluntary, but the examined teaching was a part of the medical students’ curriculum concerning respiratory distress in children.

Sample size calculation and randomization

A pilot study was carried out among 20 paramedic nurse students to estimate baseline mean values and standard deviations in the classroom teaching, podcast or vodcast groups. For sample size calculation ( the change in the test scores from baseline to final test was considered as the main outcome and a difference of 1.5 points between the groups was identified as an important difference. With this assumption and α-error of 0.05 and power (1- β) of 0.80, the result of the sample size calculation was 28 students per group.

The percentage of dropouts was estimated to be 30–40% after both the baseline and final tests, resulting in a need to recruit 40 students for each teaching group.

The students were randomized into four equally sized teaching method groups using a computer program (https://www.randomizer.org).

Intervention

The teaching took place simultaneously in four teaching groups. The students in the classroom teaching group (n = 72) participated in an on-site class facilitated by the teachers. The students in the remote teaching groups participated in the teaching intervention at their chosen locations using their own computers or smart devices. The podcast (n = 79) and vodcast (n = 73) groups participated in the teaching activity by listening to or watching recorded material. The stream teaching group (n = 77) viewed a live-taught course through Zoom webconferencing software, and had an opportunity to communicate with the teachers, either by speaking through their computer’s microphone or by texting their comments in Zoom’s chat feature.The the podcast and vodcast groups did not have such an opportunity.

The same script and protocol were used in all the teaching groups. The script was jointly prepared by three clinical teachers in paediatrics and evaluated before recording the material. For example, the vodcast was recorded and edited beforehand by adding subtitles. Subsequently, the audio file was isolated from the vodcast into a separate recording, which was subsequently made into a podcast. Therefore, the vodcast and the podcast were the exact same recording with the exception that there was no video content included in the podcast.

In all the teaching groups, the lesson was based on the MUSIC (empowerment, usefulness, success, interesting, caring) theory and followed the principles of multimedia teaching [9,10,11] to promote the students’ motivation and learning. The lesson content was designed to be simple, useful, and interesting by using several ways of teaching, such as sharing real-life experiences, producing typical wheezing sounds, showing videos of children with breathing difficulties, and solving patient cases together. The lesson was performed by two teachers (team teaching) who actively encouraged the students to discuss and complete practical exercises (activating and interactive teaching). The same real-life videos of children with breathing problems were shown in the live, live-stream and vodcast groups; meanwhile, the podcast group had access to only the teacher’s voice describing the findings obtained from the videos. In addition, we constructed a mnemonic (IREDO) on how to act if suspecting breathing difficulties in small children and taught it to all the students (Fig. 2.)

Content of teaching

The content of the lesson was similar in all the teaching groups and consisted of the identification and medical treatment of the most common respiratory problems in children (Table 1.).

Measure of learning

The primary outcome was how much the learning results improved in the identification of breathing problems in children. Learning was measured using the same Webropol e-Test that was sent to the participants via e-mail on three separate occasions (baseline, final and long-term memory tests). The test was time-constrained (15 min) and consisted of 10 multiple-choice questions: five theoretical questions and five video clips of children with respiratory distress. The number of correct response alternatives was one or more in each question: Every correct choice accounted for plus 2 points and every wrong choice for 1 minus point. The maximum score was plus 28 points, and the minimum was minus 26 points.

Learning efficacy was defined as the difference between the final and baseline test scores.

Statistical analyses

The IBM SPSS statistical software 27.0 was used to compare and analyze the results of the study.

As the test scores followed the normal distribution, the ANOVA multiple comparison test was used for assessing differences between teaching groups, and post hoc tests for paired comparison were performed using the Least Significant Difference (LSD) test. Multiple regression analysis was used to test the effect of background variables on the results. The level of statistical significance was set at p < 0.05.

Ethical considerations

All fifth-year medical students (UEF) and all paramedic nurse students (Savonia) were invited to participate in the study.

At the outset of the baseline test, students were requested to provide their consent for participation in the study. The students had the right to participate in each of the teaching interventions even if they opted out of participating in the tests of the present study. The data were collected with a pseudo-descriptive method. The data were collected and analyzed anonymously.

The research plan was evaluated by the Research Ethical Committee of UEF, although its authorization was not required under Finnish law.