Tuesday, November 19, 2013

MOOCs: Less image management, more design principles, please

I begin with the news and then offer a perspective of an educational researcher. First, the news--The MOOC experiment at San Jose State University continues, but one of the pioneers of the experiment is expressing reservations and its would-be faculty guinea pigs are using the machinery of campus politics to slow down the effort.

The SJSU faculty have requested a review of the campus chancellor's leadership after last spring and sunmer's online education experiment showed weak results: http://www.insidehighered.com/news/2013/11/19/facultyi -members-san-jose-state-u-urge-outside-review-institutions-governance

In the meantime, the leader of the SJSU experiment, Sebastian Thrun of the MOOC provider Udacity, recently told a writer for the magazine Fast Company that he is downshifting his MOOC ambitions from the loftier climes of higher education learning to the supposedly much more straightforward and achievable workd of workforce training (????), specifically for computer scientists: http://www.fastcompany.com/3021473/udacity-sebastian-thrun-uphill-climb.

Although Udacity continues its work with SJSU after a very poor success rate (25% passed) compared to face-to-face classes (53% passed) last spring, the (entirely predictable!) lack of evidence of actual course completion and learning in many MOOCs has been (finally!) grabbing media  attention lately:

"And yet, all of these efforts have been hampered by the same basic problem: Very few people seem to complete courses when they're not sitting in a lecture hall. Udacity employs state-of-the-art technology and sophisticated pedagogical strategies to keep their users engaged, peppering students with quizzes and gamifying their education with progress meters and badges. But a recent study found that only 7% of students in this type of class actually make it to the end. (This is even worse than for-profit colleges such as the University of Phoenix, which graduates 17% of its full-time online students, according to the Department of Education.)"

"Among those pupils who took remedial math during the pilot program, just 25% passed. And when the online class was compared with the in-person variety, the numbers were even more discouraging. A student taking college algebra in person was 52% more likely to pass than one taking a Udacity class, making the $150 price tag--roughly one-third the normal in-state tuition--seem like something less than a bargain. The one bright spot: Completion rates shot through the roof; 86% of students made it all the way through the classes, better than eight times Udacity's old rate. (The program is supposed to resume this January; for more on the pilot, see "Mission Impossible.")

I have been reporting these media reports rather dutifully for several months now, and at last, I have an actual opinion and perspective to express. Let's start here: Note that the experience with the SJSU developmental math MOOC was somewhat different from other MOOCs.  The low completion rates that have plagued most online learning for the past 15 years were not replicated. In this case, students finished; they just didn't pass. This is an interesting finding that no one seems to be discussing much. 

One wonders why these students didn't do what most savvy students do when they are gaming the traditional higher Ed system: Drop the class you're clearly not passing to "maintain" your GPA--and the illusion that you are a traditional "good student."  Perhaps they're not savvy students. There is good circumstantial evidence to support this claim. For example, perhaps not being savvy students is why they are in dev Ed. Somewhere along the line, they didn't get the memo about playing the school game. They didn't cram before the placement test to avoid the dev Ed class when they came to the admissions office. But make no mistake--the savvy students do that. Further, we are at SJSU, a tier 2 college. These students likely didn't take lots of SAT prep classes and GPA-goosing AP classes in high school either. After all, that is what the savvy students do. So these students were able "only" to get into SJSU rather than a supposedly "better" institution of higher Ed.

And think about that last statement. What is it with the brand names in higher Ed? I have worked with some very terrific SJSU grads and some pretty lame Stanford grads. Enough on the brand name nonsense! Is anyone else noticing how perverse things have become? What does any of the business of being a savvy student and embroidering your resume with the right schools have to do with actual learning? It really boils down to a ridiculous level of image management. Do we really want, as a society, our youngest, most idealistic citizens focused on image management, which is really what being a savvy student has become? Where does that lead us, as a society? 

Now, that's my opinion. Here is the substantive perspective part as an educational researcher: What is the difference between finishing and passing, and what does this tell us about learning? Could it be that the 75% of folks who did not pass may tell us something about the failures of thinking you can simply put typical course material online (video lectures, slides, readings) and then make magic happen? What needs to change in the design of learning content to permit the other 75% to learn online? As it happens, there is plenty of research into online learning that has clarified that point, and yet no one discusses that research in these various reports.

Back to opinion: Instead, the reporters seek the answer from entrepreneurs with pedigrees from brand name institutions who have not spent not more than a moment studying pedagogy. Hey, why would they? Studying pedagogy--let's face it, studying "education"--isn't good for their "image." Education is not a prestige field. Everybody knows that, right? Why do it then?

Back to perspective: But, for the record, here are some design principles we know... You need social support to learn, mainly to provide what is called learner "self-regulation," which means keeping the learner focused consistently to finish. (In the old days when mostly women taught in schools, we called it "helping." But I digress...) Anyway, you also need multiple representations of knowledge, frequent feedback, and -- get this -- multiple trials to reach mastery. Yes, it turns out that the notion that you learn "fast and efficiently" is simply goofy. You learn slow and painstakingly, through lots of hard work and effort. Yes, the sad truth is that we must be "grinds" to learn. But again, what would saying that do for our "image?" Oh by the way, that reminds me, we in the Ed field are calling hard work "grit" now. 

Could it also be that there is an implicit identification of the central problem of the design of traditional higher Ed learning in these data? Perhaps those 75% non-completers represent the kinds of students who do not learn very well under the old model of "time pressured, image-managed, competitive grade learning" the first time around, but ultimately could learn if permitted to study with some form of support on their own time and less pressure around image management? The MOOC experiment does not answer these questions, but these results raise such questions. 

Instead, in the MOOC coverage, we see the predictable narrow focus on one data point or two (pass rates! completion rates!), and on to the next personality piece about the next entrepreneur with something to sell us. There is very little thought about what the evidence of pass and completion rates tells us about the design of education. Instead, we see the predictable political positioning to protect the status quo without any overt evidence of a good faith effort to try to improve the design of the status quo (e.g., SJSU and UCSC faculty "protests"). We see the sort of half-baked questioning about the "social justice" of such proposed innovations without really looking at the details of what is actually happening in the cases of the oft-quoted stereotypes of the "underrepresented, needy student." So, we use these stereotypes as a prop to block change and we learn nothing about how the design of learning could be improved to actually help them (and, hey, if we are honest, everyone else too)-- (for the PC perspective, see this: http://www.hackeducation.com/2013/11/14/thrun-as-saint/?utm_source=feedburner&utm_medium=email&utm_campaign=Feed%3A+HackEducation+%28Hack+Education%29 ). 

None of this is constructive or focused on solving the education problem in this country. That takes real work and a laser focus on the problem and designing a solution -- and not a focus on the personalities, political strategy, or political correctness. 

Be advised, that kind of focus on the design of learning IS happening, but coverage of that work is in short supply in the media of all forms and all channels. 

Why not? Because it isn't a sexy story, and that isn't good for the image of the reporter or the media outlet publishing it. 

2 comments:

learner said...

First, a quick note on the data from the experiment: the populations in the in person vs online versions of the math course were very different. Comparing pass rates directly is dangerous.

Second: I had a chance to interact with some of the students who took the Mooc and failed. In many of the cases, their experience was good. They would say things like "this is the first time I'm getting math." These students failed, even though they were learning because the schedule of the course moved too fast for them. I believe that unyielding schedules are an artifact of traditional higher ed, and we will start to see solutions to this problem soon.

Michael Freed said...

It's great to see you (someone!) trying to understand what the MOOC data really means. Peter Norvig suggested that people might be coming into these courses with very different intentions than is typical with traditional classes, e.g. by sampling based on interests and not even trying to complete the whole thing. Like your post, that seemed at least plausible and thoughtful. By contrast, I just read a terrible Slate rant on the topic that focused on bashing Sebastian Thrun for arrogance in thinking this could work -- not an insight to be found in the piece.