One size definitely does not fit all – but choices help
To what extent is the proliferation of podcasts helping students learn? With the growing popularity of lecture recordings, which may be just audio and hence podcasts, or voice over audio with slides (slidecasts?) or the Picture-n-picture fanciness of Mediasite or Echo360, the catalog of digital media capture of lectures is rapidly growing. Indeed, iTunes U provides a nice and convenient distribution system, aided by the fact that you actually disseminate the object rather than just stream it. Even though the majority of iTunes U users are watching or listening to content iTunes podcasts on their laptops rather than iPhones or iPod Touches, they are or at least can be disconnected from the network while they are doing it. But connected or not are they learning anything?
Do students perform differently on when supplemental materials are presented as simply plain text vs. media-enabled text and images (podcasts with graphics plus audio)?
They used a “Pre-post test non-equivalent groups switching replications design” – which in common language means something relatively simple. They gave one group of students in a class supplemental material that was text only. The other group of students in the same class got supplemental material as podcasts. The were tested before and after the particular topic in the class was discussed. Then, on the next class topic the groups switched. The text-only group got the podcast supplemental material and the podcast group got the text only material. It’s not using subjects as their own controls like some clinical trial designs but it’s a nice and relatively powerful method.
What did they find?
The tests administered involved remembering, understanding and evaluating the material that was presented. They also had students respond to a series of surveys (self-reporting). Independent variables included age, gender, student level, learning styles (I won’t rant here about “learning styles” but simply say it’s time this was dropped as a meaningful variable – it’s just bunk… but to continue), motivation and learning strategies. They employed an analysis of covariance (ANCOVA) on the pre-post test differences using the pre-test scores as the covariate. They also did some simple regressions where appropriate.
The core finding was that overall among the some 262 students in the class, a lecture that met twice a week for 75 minutes plus 50 min tutorial or recitation section between the lectures there was no statistical difference between the text only vs. the podcast supplemental material. The dreaded NSD reared its ugly head. But before the ‘I told you so’ choruses build among the technophobic or those otherwise clinging to cherished values of broadcast lecture with academic text, there’s more.
If you look at the data more carefully, Vovides and Garrett found a more nuanced outcome. When they looked at the different age, gender, motivations and learning strategies among the students there were differences in learning, statistically significant differences, primarily along the lines of the “understanding dimension of the course material. Interestingly, it is precisely in the unmotivated students, those who showed poor levels of intrinsic motivation, that the podcasts seemed to make a significant difference. Where perceived interest, motivation or utility of the course material were low, students receiving the podcasts scored higher than students getting text-only supplemental material.
That shouldn’t be that surprising. Podcasts have shown to be more engaging than just text, especially for students with low intrinsic motivation.
Decomposing the graphic the overall dependent variable here is the pre-post difference score for understanding the content presented. Recall that the treatment type, as it’s referred to here, is whether or not the students got the text version or the podcast version of the supplemental materials. Students rated “strong” found the text material worked well but average students found the podcasts gave them a better understanding. Weak students were, well, weak students who only slightly benefited by the podcasts. There is a significant difference just from the student level regardless of the treatment. But there is clearly an interaction effect by student level (strong vs. average, primarily) shown by “X” marking the spot!
As Vovides and Garrett note in their conclusion:
It seems that podcasting, instead of text, can be more beneficial for certain types of students and that learner differences and that learner differences as well as well as levels of learning should be carefully considered when designing instructional materials.
Nice study. Podcasts aren’t that helpful to the motivated and high performing student. But for that big group in the middle, it seems it can make a real, substantive difference.
This was presented at Educause 2010 at their poster session, specifically as Poster #13, in Anaheim last month (October). The poster itself is up on ScribD