Onscreen presence of instructors in video lectures affects learners' neural synchrony and visual attention during multimedia learning.

COVID-19 forced students to rely on online learning using multimedia tools, and multimedia learning continues to impact education beyond the pandemic. In this study, we combined behavioral, eye-tracking, and neuroimaging paradigms to identify multimedia learning processes and outcomes. College students viewed four video lectures including slides with either an onscreen human instructor, an animated instructor, or no onscreen instructor. Brain activity was recorded via fMRI, visual attention was recorded via eye-tracking, and learning outcome was assessed via post-tests. Onscreen presence of instructor, compared with no instructor presence, resulted in superior post-test performance, less visual attention on the slide, more synchronized eye movements during learning, and higher neural synchronization in cortical networks associated with socio-emotional processing and working memory. Individual variation in cognitive and socio-emotional abilities and intersubject neural synchronization revealed different levels of cognitive and socio-emotional processing in different learning conditions. The instructor-present condition evoked increased synchronization, likely reflecting extra processing demands in attentional control, working memory engagement, and socio-emotional processing. Although human instructors and animated instructors led to comparable learning outcomes, the effects were due to the dynamic interplay of information processing vs. attentional distraction. These findings reflect a benefit-cost trade-off where multimedia learning outcome is enhanced only when the cognitive benefits motivated by the social presence of onscreen instructor outweigh the cognitive costs brought about by concurrent attentional distraction unrelated to learning.

Knowledge revision processes during reading: How pictures influence the activation of outdated information.

Outdated information (i.e., information that is not or no longer accurate) continues to be automatically activated during reading and can hinder learning processes. Thus, it is important to understand which factors influence the activation of outdated information and, therefore, knowledge revision processes. In three online experiments, we investigated how illustrating updated or outdated information via pictures influences the activation of outdated information. In Experiments 1 (N = 421) and 2 (N = 422), we varied whether participants read texts containing outdated information that was later updated (outdated text) or texts containing only updated information (consistent text). In addition, the updated information was or was not illustrated by a picture. In Experiment 3 (N = 441), participants read outdated texts, and we varied whether the outdated, the updated, or no information was illustrated. In all experiments, we measured reading times for a target sentence referring to the updated information and the sentence following the target sentence. Results showed that target sentences' reading times were faster for illustrated than for non-illustrated texts (Experiments 1 and 2). Moreover, reading times were longer when the outdated information was illustrated than when the updated information was illustrated (Experiment 3). These results suggest that pictures overall facilitate cognitive processes during reading, but their content matters: Pictures showing the updated information had a greater impact on reading times than pictures showing the outdated information. The results extend existing theories on knowledge revision but also reading comprehension, by demonstrating how pictures might influence cognitive processes during reading.

Watching videos of a drawing hand improves students' understanding of the normal probability distribution.

Understanding normal probability distributions is a crucial objective in mathematics and statistics education. Drawing upon cognitive psychology research, this study explores the use of drawings and visualizations as effective scaffolds to enhance students' comprehension. Although much research has documented the helpfulness of drawing as a research tool to reveal students' knowledge states, its direct utility in advancing higher-order cognitive processes remains understudied. In Study 1, qualitative methods were utilized to identify common misunderstandings among students regarding canonical depictions of the normal probability distribution. Building on these insights, Study 2 experimentally compared three instructional videos (static slides, dynamic drawing, and dynamic drawings done by a visible hand). The hand drawing video led to better learning than the other versions. Study 3 examined whether the benefits from observing a hand drawing could be reproduced by a dynamic cursor moving around otherwise static slides (without the presence of a hand). Results showed no significant learning difference between observing a hand drawing and a moving cursor, both outperforming a control. This research links the cognitive process of drawing with its educational role and provides insights into its potential to enhance memory, cognition, and inform instructional methods.

Multimedia design for learner interest and achievement: a visual guide to pharmacology.

BACKGROUND: Medical education increasingly relies on digital learning materials. Despite recognition by the Association of American Medical Colleges Institute for Improving Medical Education, medical education design often fails to consider quality multimedia design principles. Further, the AAMC-IIME issued a call to study the role of design principles in medical education. Thus, the current study investigated the cognitive and affective effects of redesigning PowerPoint slides used to support pharmacology content during the preclinical years of medical school training. METHODS: Researchers implemented a quasi-experimental design, using traditionally-designed (original) slides with a Traditional group (n = 100) and slides redesigned to follow principles from the Cognitive Theory of Multimedia Learning with an Experimental group (n = 40). Participants in the Experimental group completed a post-survey incorporating the Situational Interest Survey for Multimedia to indicate motivational engagement with the media. Students in the Experimental group also responded to additional preference questions. Researchers analyzed survey responses and students' scores in pharmacology-related coursework across the preclinical Foundations Phase of training to determine the impact on achievement and motivation. RESULTS: Findings related to learner achievement showed a modest but significant increase in the Experimental group compared to the Traditional group in the Cardiac, Pulmonary, and Renal (CPR) educational block (105%, normalized to Traditional group, p = 0.013) and cumulative pharmacology grades (101%, normalized to Traditional group, p = 0.043). Additionally, participants in the Experimental group indicated a significantly higher average triggered situational interest in redesigned slides (M = 4.85, SD = 0.25) than the original slides (M = 3.23, SD = 1.40, t=-6.33, p < 0.001). Similarly, the interest rating of the redesigned slides (M = 4.87, SD = 0.24) was significantly greater than that of the original slides (M = 3.89, SD = 0.86, t=-6.824, p < 0.001). Moreover, results further indicated significant differences in the maintained-feeling and maintained-value constructs, and all participants in the Experimental group indicated a preference for the redesigned slides. CONCLUSIONS: The findings provide strong evidence in support of using the Cognitive Theory of Multimedia Learning design principles to intentionally design media in medical education. In addition to increased achievement scores, students in the Experimental group demonstrated significantly higher levels of situational interest in the redesigned slides, especially triggered interest and maintained-feeling interest. Medical education learning designers should seriously consider redesigning media to achieve similar results.

Facilitating active learning of sectional anatomy with technology-enhanced small-group tasks: Assessment of knowledge gains, technology usability, and students' perceptions.

Learning 2D sectional anatomy facilitates the comprehension of 3D anatomical structures, anatomical relationships, and radiological anatomy. However, the efficacy of technology-enhanced collaborative instructional activities in sectional anatomy remains unclear, especially if theoretical frameworks, namely the Cognitive Theory of Multimedia Learning (CTML), are applied in instructional design. Thus, this study compared the educational impact of distinct 45-min-long technology-enhanced collaborative learning tasks in sectional anatomy. A sample of 115 first-year medical students was randomly divided into three experimental groups that used different supporting technologies to learn the sectional anatomy of the chest: IMAIOS e-learning platform and Microsoft Surface Hub (n = 37); anatomage table (n = 38); anatomage table with CTML-based presets (n = 40). Prelearning and postlearning tests revealed that significant knowledge gains in sectional anatomy were obtained by all groups even though no inter-group differences were found. Moreover, a five-point Likert scale questionnaire showed that the learning session was highly valued by all participants and that users of the anatomage with CTML-based presets reported higher enjoyment than users of the IMAIOS system (mean difference = 0.400; p = 0.037). In addition, students using the IMAIOS system and the anatomage with CTML-based presets provided System Usability Scale (SUS) scores of 67.64 and 67.69, respectively, reaching the benchmark of usability. By contrast, students using the anatomage table without presets awarded a SUS score of 64.14. These results suggest that the integration of multimedia technologies in anatomy teaching and learning should be grounded on CTML principles of instructional design. Otherwise, students' perceptions of ed-tech usability are potentially hindered.

Multimedia Instructions for Motor Control Exercises in Patients With Chronic Nonspecific Low Back Pain.

BACKGROUND: Low back pain (LBP) is one of the top 3 diseases that may lead to disability. Current treatment guidelines define exercise as a first-line treatment for nonspecific LBP (NSLBP). There are various evidence-based exercise approaches for treating NSLBP, and many of them include motor control principles. Motor control exercises (MCEs) are better than general exercises that do not include motor control principles. Many patients find learning these exercises complex and challenging, in that MCE exercises have no standard teaching method. The researchers of this study developed multimedia instructions for an MCE program to make teaching MCE easier; thus, more effective. METHODS: The participants were randomized into multimedia or standard (face-to-face) instruction groups. We applied the same treatments to both groups at the same dosage. The only differences between groups were the exercise instruction methods. The multimedia group learned MCE from multimedia videos; the control group learned MCE from a physiotherapist with face-to-face instructions. Treatment lasted 8 weeks. We evaluated patients' exercise adherence with Exercise Adherence Rating Scale (EARS), pain with the Visual Analog Scale, and disability with Oswestry Disability Index. Evaluations were made before and after treatment. Follow-up evaluations were carried out 4 weeks after the end of treatment. RESULTS: There was no statistically significant interaction between the group and time on pain, F2,56 = 0.068, P = .935, partial η2 = .002 and Oswestry Disability Index scores, F2,56 = 0.951, P = .393, partial η2 = .033. Also, there was no statistically significant interaction between the group and time on Exercise Adherence Rating Scale total scores F1,20 = 2.343, P = .142, partial η2 = .105. CONCLUSIONS: This study showed that multimedia instructions for MCE have similar effects to standard (face-to-face) instructions on pain, disability, and exercise adherence in patients with NSLBP. To our knowledge, with these results, the developed multimedia instructions became the first free, evidence-based instructions that have objective progression criteria and a Creative Commons license.

Seductive details hamper learning even when they do not disrupt.

Previous research often revealed detrimental effects of seductive details on learning with multimedia instruction, but there are mixed findings regarding how to best explain these detrimental effects. We investigated whether the detrimental effects of seductive details are mainly mediated by the cognitive processes of diversion (deeper processing of seductive details rather than pertinent content) or disruption (unsuccessful attempts to integrate seductive details with pertinent content) by assessing the effects of instructional prompts. In an online learning experiment, participants (N = 247) learned either without seductive details (control condition) or with seductive details in one of three conditions: Participants received either a prompt informing them about the irrelevance of seductive details (irrelevance-prompt), a prompt to process seductive details and pertinent content separately (separation-prompt), or no prompt within their task instruction. We assessed recall and transfer of knowledge as dependent variables. Supporting the diversion hypothesis, participants in the no-prompt condition regarded seductive details as more relevant and consequently spent more time processing them compared to participants in the irrelevance-prompt condition, which negatively influenced their recall performance. Against the disruption hypothesis, participants in the no-prompt condition reported lower integration avoidance between seductive details and pertinent content compared to participants in the separation-prompt condition, but this led to better rather than worse transfer performance. Our results thus suggest diversion, and not disruption, to be the main process driving the seductive details effect. Reducing the details' diverting potential seems a good way to deal with seductive details in instruction.

A review of eye tracking research on video-based learning.

Eye tracking technology is increasingly used to understand individuals' non-conscious, moment-to-moment processes during video-based learning. This review evaluated 44 eye tracking studies on video-based learning conducted between 2010 and 2021. Specifically, the review sought to uncover how the utilisation of eye tracking technology has advanced understandings of the mechanisms underlying effective video-based learning and what type of caution should be exercised when interpreting the findings of these studies. Four important findings emerged from the analysis: (1) not all the studies explained the mechanisms underlying effective video-based learning through employing eye tracking technology, and few studies disentangled the complex relationship between eye tracking metrics and cognitive activities these metrics represent; (2) emotional factors potentially serve to explain the processes that facilitate video-based learning, but few studies captured learners' emotional processes or evaluated their affective gains; (3) ecological validity should be improved for eye tracking research on video-based learning through methods such as using eye tracking systems that have high tolerance for head movements, allowing learners to take control of the pacing of the video, and communicating the learning objectives of the video to participants; and (4) boundary conditions, including personal (e.g. age, prior knowledge) and environmental factors (e.g. the topic of videos, type of knowledge), must be considered when interpreting research findings. The findings of this review inspire a number of propositions for designing and interpreting eye tracking research on video-based learning.

I am Alexa, your virtual tutor!: The effects of Amazon Alexa's text-to-speech voice enthusiasm in a multimedia learning environment.

Modern text-to-speech voices can convey social cues ideal for narrating multimedia learning materials. Amazon Alexa has a unique feature among modern text-to-speech vocalizers as she can infuse enthusiasm cues into her synthetic voice. In this first study examining modern text-to-speech voice enthusiasm effects in a multimedia learning environment, a between-subjects online experiment was conducted where learners from a large Asian university (n = 244) listened to either Alexa's: (1) neutral voice, (2) low-enthusiastic voice, (3) medium-enthusiastic voice, or (4) high-enthusiastic voice, narrating a multimedia lesson on distributed denial-of-service attack. While Alexa's enthusiastic voices did not enhance persona ratings compared to Alexa's neutral voice, learners could infer more enthusiasm expressed by Alexa's medium-and high-enthusiastic voices than Alexa's neutral voice. Regarding cognitive load, Alexa's low-and high-enthusiastic voices decreased intrinsic and extraneous cognitive load ratings compared to Alexa's neutral voice. While Alexa's enthusiastic voices did not impact affective-motivational ratings differently from Alexa's neutral voice, learners reported a significant increase of positive emotions from their baseline positive emotions after listening to Alexa's medium-enthusiastic voice. Finally, Alexa's enthusiastic voices did not enhance the learning performance on immediate retention and transfer tests compared to Alexa's neutral voice. This study demonstrates that a modern text-to-speech voice enthusiasm can positively affect learners' emotions and cognitive load during multimedia learning. Theoretical and practical implications are discussed through the lens of the Cognitive Affective Model of E-learning, Integrated-Cognitive Affective Model of Learning with Multimedia, and Cognitive Load Theory. We further outline this study's limitations and recommendations for extending and widening the text-to-speech voice emotions research.

The role of live transcripts in synchronous online L2 classrooms: Learning outcomes and learner perceptions.

This study explored the role of live transcripts in online synchronous academic English classrooms by focusing on how automatically generated live transcripts influence the learning outcomes of lower-proficiency and higher-proficiency learners and on their perceptions towards live transcripts. The study adop ted a 2 × 2 factorial design, with the two factors being learner proficiency (high vs. low) and availability of live transcription (presence and absence). The participants were 129 second-year Japanese university students from four synchronous classes taught on Zoom by the same teacher under an academic English reading course. Learning outcomes in this study were evaluated according to the course syllabus through grades and participation in class activities. A questionnaire consisting of nine Likert-scale questions and a comment box was administered to explore participants' perceived usefulness of, perceived ease of use of, and perceived reliance on live transcripts. Results showed that contrary to previous studies reporting the effectiveness of captioned audiovisual materials in L2 learning, live transcripts as a special type of captions were not effective in promoting the grades of learners of either proficiency. However, it significantly improved the activity participation of lower-proficiency learners, but not that of higher-proficiency learners. Questionnaire results showed that there were no significant differences between learners of two proficiencies in their perceptions towards live transcription, which contradicts previous findings that lower-proficiency learners tend to rely more on captions. Besides enhancement of lecture comprehension, participants reported innovative uses of live transcripts such as screenshots with transcripts for notetaking purposes and transcripts downloaded for later review.

Bilingual children's visual attention while reading digital picture books and story retelling.

This study examined Mandarin-English bilingual children's visual attention over repetitive readings of Mandarin enhanced digital books and static books as well as the effects visual attention has on story retelling. We assigned 89 4- and 5-year-old preschoolers in Singapore to one of three reading conditions: (a) digital books with visual and auditory enhancements, (b) digital books with only auditory enhancements, and (c) static digital books with neither visual nor auditory enhancements. We presented three stories to the children in four sessions over 2 weeks, traced their visual attention with an eye tracker, and examined their story retelling after the first and fourth readings. The results demonstrated that the digital books with visual and auditory enhancements maintained greater visual attention from children compared with that from children in the other two conditions across the four repetitive readings. Moreover, children's bilingual language proficiency significantly modulates the conditional effects of attention. Children with higher bilingual proficiency in the visual and auditory enhancements condition outperformed their peers in the other two conditions in terms of visual attention across most readings. However, for the children with lower bilingual proficiency, the digital books with auditory and visual enhancements only outperformed the static condition but not the auditory enhanced condition. Children with lower language proficiency maintained their attention at a relatively high level across the repetitive readings in the enhanced digital book conditions but demonstrated significantly decreased visual attention in the static digital book condition. Because children with better visual attention and higher bilingual proficiency retold the stories significantly better, the results indicate that influencing visual attention helps to improve story comprehension.

Using commonly-available technologies to create online multimedia lessons through the application of the Cognitive Theory of Multimedia Learning.

Principles derived from the Cognitive Theory of Multimedia Learning (CTML; Mayer in: Multimedia learning, Cambridge University Press, Cambridge, 2021) provide valuable guidance for enlisting commonly-available technologies to create effective online multimedia lessons. Specifically, CTML can guide instructional designers on the use of slide-sharing programs to create concise, narrated animation segments; the use of survey programs to interpolate questions and prompts between these segments to facilitate generative learning activities; and the use of video-sharing sites to provide learners with control over relatively superficial aspects of instruction. The application of CTML to the design of online multimedia lessons raises a number of theoretical and practical questions, including the need to better understand the relationship between working memory capacity and working memory duration, the importance of retrieval as a learning process, and the relative impact of selection and organization processes on learning.

Strategies for facilitating processing of transient information in instructional videos by using learner control mechanisms.

Learner control of video presentations by using pause buttons or timeline scrollbars was suggested as helpful for learning from sources of transient information such as dynamic visualizations and spoken words. However, effective learner control could be difficult to attain without sufficient instructional support. This study developed strategies for facilitating processing and integration of transient information based on cognitive load theory by providing learners with explicit guidance in when and how to use pausing and timeline scrollbars while watching instructional videos. A single-factor between-subjects experiment was conducted to examine the effects of the proposed strategies. Ninety undergraduates were randomly assigned to one of three groups - strategy guidance group (learners were provided with guidance in strategies), learner control group (learners were allowed to control the video but without any guidance in strategies), and continuous presentation group (without any learner control mechanism). The results revealed that compared to the learner control group, the strategy guidance group had a greater number of pauses and scrollbacks on the timeline, demonstrated significantly better performance in the immediate comprehension test and higher performance efficiency in the immediate recall and comprehension tests. Compared to the continuous presentation group, the strategy guidance group demonstrated significantly better performance in the immediate recall and comprehension tests and higher performance efficiency in both these tests, as well as better performance in the delayed recall test and higher performance efficiency in the delayed recall test.

Evaluating the Efficacy and Optimisation of the Peer-Led Flipped Model Using TEL Resources Within Neuroanatomy.

The flipped classroom (where students prepare before and then develop understanding during class) and technology-enhanced learning (audio-visual learning tools) are increasingly used to supplement anatomy teaching. However, the supporting literature lacks robust methodology and is conflicting in demonstrating efficacy outcomes. Contrastingly, near-peer teaching (where senior students teach juniors on the same academic programme) is well researched and reported to be both effective and versatile. This provides an ideal vehicle in which to investigate and potentially optimise these approaches.This study aims to assess educational impact of the peer-led flipped model and student engagement and perceptions regarding traditional and TEL resources.A quasi-randomised, cross-sectional study was conducted with 281 second-year University of Southampton medical students. Students were randomly allocated to 3 groups: traditional lecture (control), flipped text resource, or flipped video resource. The first group received no pre-teaching material, but the flipped groups received a text or video pre-teaching resource. Objective outcomes measured were: Knowledge gain and retention via multiple-choice questionnaires and formative exams Student perceptions and engagement using questionnaires and 2 focus groups All groups demonstrated significant knowledge gain post-teaching (p < 0.0001). However, regardless of engagement with pre-teaching material, no significant difference was found in knowledge gain or retention between the groups. Students engaged 21.1% more with the text rather than video resource (p = 0.0019), but spent equal time using both (p = 0.0948). All resources and teaching approaches were perceived 'very useful' with no significant differences found between groups. A qualitative approach utilising thematic analysis of focus groups identified 4 themes, including 'Attitudes towards flipped classroom', which revealed mixed reviews and perceptions from participants.This study has found the peer-led flipped model is of no detriment to educational impact compared to peer-led traditional approaches in a well-established peer teaching programme in undergraduate medicine at the University of Southampton. Students value traditional and video resources but engage with them differently. Additionally, it was reported that in this experiment, NPT did not seem well suited to the flipped classroom, suggesting a rare limitation of the utility of NPT application within an anatomy curriculum.

TEL Methods Used for the Learning of Clinical Neuroanatomy.

Ubiquity of information technology is undoubtedly the most substantial change to society in the twentieth and twenty-first centuries and has resulted in a paradigm shift in how business and social interactions are conducted universally. Information dissemination and acquisition is now effortless, and the way we visualise information is constantly evolving. The face of anatomy education has been altered by the advent of such innovation with Technology-Enhanced Learning (TEL) now commonplace in modern curricula.With the constant development of new computing systems, the temptation is to push the boundaries of what can be achieved rather than addressing what should be achieved. As with clinical practice, education in healthcare should be evidence driven. Learning theory has supplied educators with a wealth of information on how to design teaching tools, and this should form the bedrock of technology-enhanced educational platforms. When analysing resources and assessing if they are fit for purpose, the application of pedagogical theory should be explored and the degree to which it has been applied should be considered.

Is a Picture Worth a Thousand Words? Evaluating the Design of Instructional Animations in Veterinary Education.

Empirical evidence demonstrates that student learning outcomes improve when animations are developed in alignment with the design principles of the cognitive theory of multimedia learning (CTML). The extent to which these principles are used in the design of veterinary instructional animations is unknown. In this study, we reviewed the veterinary education literature for articles that discussed specific veterinary medical animations as learning resources. The 30 referenced animations accessed through this search were analyzed to determine whether they used the CTML's 11 major design principles. Analysis revealed that the animations most commonly adhered to only 4 principles: coherence, redundancy, modality, and spatial contiguity. The majority of the 11 CTML principles were used in fewer than 40% of the animations. We also examined the alignment between raters' perceptions of the effectiveness and enjoyment of the animations and adherence to the design principles. Analyses revealed that the animations deemed by raters as most enjoyable and effective did not utilize more design principles than animations they viewed as least enjoyable and effective. The results of this study indicate many missed opportunities to increase learning by developing animated learning resources according to empirically based design principles. Decisions to include specific animations in instruction should be based on whether the resources include elements that have been shown to increase learning rather than subjective perceptions of effectiveness and enjoyment.

A randomized controlled trial to test efficacy of digital enhancements of storybooks in support of narrative comprehension and word learning.

Experiments with film-like story presentations have been found to be beneficial in supporting children's story comprehension and word learning. The main goal of the current study was to disentangle the effects of visual and auditory enhancements in digital books. Participants were 99 typically developing children (41 boys and 58 girls) aged 4-6 years from two public kindergartens in Bursa, Turkey. A randomized controlled trial was conducted with a control group and four experimental conditions that included all possible combinations: static illustrations with and without music/sounds and animated illustrations with and without music/sounds. In each experimental condition, children read two different storybooks twice, each time in small group sessions of 2 or 3 children. The posttest included, apart from story comprehension, expressive and receptive vocabulary tests of book-based words. Story comprehension, not word learning, benefited from visual enhancements in digital books. Music and background sounds did not stimulate story comprehension and even had a negative effect on receptive vocabulary. To explain the findings, we refer to multimedia learning principles such as temporal contiguity. Consequences for a digital storybook format are discussed.

Assessing the Role of Screencasting and Video Use in Anatomy Education.

The subject of anatomy, commonly taught with applied clinical focus on medical programmes, is frequently brought to life alongside art, imagery and visualization. Yet, despite being continually hailed as the cornerstone of medicine, the cyclic revalidation of its curricula has often found its educators in the unenviable position of maintaining knowledge standards in the face of reduced contact time. However, the gravity of such challenges has created an opportunity for creative and innovative solutions to these problems. The ease by which educational technology can now be used by non-experts is constantly increasing and the use of technology enhanced learning has now become universal within Higher Education. Many anatomical science educators have turned to building bespoke interactive and engaging online supplementary material which can be blended with face to face delivery as a way to circumvent the time pressure issues. Today's students appear to have a growing preference for visualising moving images and audio explanations as opposed to older traditional static resources, underpinned by vast pages of unattractive dense text and pictures. One such technique being used to provide flexible and student-centred learning is screencast videos. These digital recordings of screen captured drawings, with accompanying narration are overwhelmingly popular with students and on the ascendance. However, as new tools emerge, it becomes increasingly important to determine their impact on both the student experience and knowledge gain. It is also valuable for educators to share their classroom experiences or instructional techniques to optimise their use for learning. This chapter explores the rise of this application in anatomy education and discusses the evidence available investigating student engagement and learning outcomes in the context of well-established learning theories.

The Influence of Presentation Format of Story on Narrative Production in Chinese Children Learning English-as-a-Second-Language: A Comparison Between Graphic Novel, Illustration Book and Text.

Past studies have shown that multimodal presentation of story can improve story-retelling performance in the first language. The purpose of the present study was to investigate whether similar multimedia effects can be observed in second language learning and graphic novel reading. A total of 51 Chinese elementary school children, aged 7-8, who were learning English as a second language were recruited. They were randomly assigned to one of the three experimental conditions that differed in the format of story presentation: English text, English text with pictorial illustrations or graphic novel. After reading the same story, the children retold the story in English. The narratives produced were then rated by two independent raters. The results of group comparison showed that children from the three experimental groups had similar performance, indicating that multimedia presentation may not always facilitate narrative production in English as a second language. Within-subject comparison further showed that the children were relatively strong in language skills and capturing the main ideas of the story, while showing weakness in story structure awareness, elaboration, as well as local and global cohesion. Suggestions for the application of multimodal presentation of narrative texts are discussed.

How pictures in picture storybooks support young children's story comprehension: An eye-tracking experiment.

In a within-participant design, 41 children (mean age = 64 months, range = 50-81) listened to brief stories in four conditions. Written text was present on the screen in all conditions (similar to the typical storybook experience) but combined with other sources of information: (a) only oral narration, (b) oral narration and a picture that was congruent with the narration, (c) oral narration and an incongruent picture, and (d) only a picture but no oral narration. Children's eye movements while looking at the screen were recorded with an eye-tracker. An important finding was that a congruent picture contributed substantially to children's story retellings, more so than a picture that was incongruent with the narration. The eye-tracking data showed that children explored pictures in a way that they could maximally integrate the narration and the picture. Consequences for interactive reading and picture storybook format are discussed.