Wednesday, June 11, 2014

Calculating the value of college

President Obama has proposed rating colleges as a way to channel more students to colleges that are worthwhile, but the idea has yet to gel and already is facing criticism by academia. Academics dismissed the presidents' proposal overwhelmingly in comments to an article in Inside Higher Education covering an interview by Tumblr with the president about high student debt. The president's broad goal is to permit consumers to determine the cost-benefits of higher education expenses. The actual Tumblr interview was much more wide-ranging--addressing government efforts to attract more students to STEM fields and to help those going into low-paying, public service fields. Currently, the most popularly-known college ranking system is conducted by the U.S. News and World Report, which is informed by factors other than student outcomes, such as how much research funding a university amasses. There have been efforts by a group using regional employment data and other research organizations using college loan data to provide such data to students and their families too.

Friday, April 4, 2014

Facing changes in funding game, CCs define new rules

Community college systems, which have increasingly seen their state funding shift away from the number of students they enroll to how many they graduate, seek to influence the way their performance is gauged by redefining the rules of accountability. Two states have adopted the Voluntary Accountability Framework, an initiative of the American Association of Community Colleges, as a system to report how how their colleges are performing, and several other states are considering adoption. The framework seeks to broaden accountability metrics beyond a narrow focus on degree completion and to consider the performance of part-time, not just full-time, students. Community colleges have long argued that federal accountability systems place too much emphasis on the types of degree completion metrics that favor the 4-year university model. By contrast, they argue they serve a more diverse set of students who have a more complex set of goals and educational pathways.

Tuesday, April 1, 2014

CCs and Udacity focus on Corporate Training

The vast Maricopa Community College system in Arizona has responded to reduced state support by expanding its contract training business with local industry. This article mentions Global Corporate College (GCC), a small Ohio woman-owned startup to help colleges capture this business. I met with GCC founder, Denise Reading, and her team, last year as part of some exploratory technology work with my employer. Additionally, this article refers to Udacity founder, Sebastian Thrun's, announcement some months back about his interest in seeing MOOCs support corporate training. There are some technological hurdles to clear to design and develop the content for online learning materials, particularly to ensure the quality of the learning experience.

Friday, January 24, 2014

Few CC Students Prep for Math Placement Exam

According to data from the American Association of Community Colleges, only a third of entering students study for the mathematics placement exam. In most colleges, a poor score on this test channels students into remedial math classes that cost them and taxpayers money without earning college credit. Research indicates as many as 40% of these students never complete college as they fail to pass these classes. These classes cover high school algebra, a form of mathematics required in only 5% of jobs that require "some college."

Wednesday, January 22, 2014

Measuring Knowledge that Matters in the Workplace

Interesting report on the PBS Newshour about how employers find that A's in school fail to predict solid performance on the job. The misalignment is particularly acute in school mathematics. Most jobs require mathematics taught through middle school. About 5% of jobs require higher mathematics, and the job growth is not in those jobs. Employers requiring middle skills are reporting more predictive success based on ACT's Workkeys Assessment than diplomas and GPAs.

Thursday, December 12, 2013

Competency vs. Credit Hours

Like MOOCs, the call for "competency-based" higher education is an effort to provide more flexibility around seat-time, which has been a mainstay of the nation's systems for adult learning. The notion is, if you know your stuff and you can pass the test quickly, why should you be required to put in a certain amount of hours? The idea is receiving increasing scrutiny and experimentation from both the U.S. Department of Education and foundations. Pioneers in this concept include non-traditional institutions such as Western Governor's University, Excelsior, and Capella, and Southern Hampshire University has become the first traditional higher education institution to decouple seat time ("the credit hour") from degree attainment. Assessment becomes the central indicator of competency, but not all assessments are created equal. Some tests focus on measuring how much terminology and facts someone knows. Some focus on the testing the skill of communicating and explaining connections among ideas. Others focus on applying concepts while solving problems. Some focus on hands-on skill and situational decision making. Many of the more complex skills are not easily or accurately captured by test formats that are easiest to administer and auto score; they require complex judgments based on multiple factors. Few people like to think about tests, but getting the metrics right offers a way out of proxies such as "seat time." The only way to do this well is to think about what few like to think about: assessment.

Tuesday, November 19, 2013

MOOCs: Less image management, more design principles, please

I begin with the news and then offer a perspective of an educational researcher. First, the news--The MOOC experiment at San Jose State University continues, but one of the pioneers of the experiment is expressing reservations and its would-be faculty guinea pigs are using the machinery of campus politics to slow down the effort.

The SJSU faculty have requested a review of the campus chancellor's leadership after last spring and sunmer's online education experiment showed weak results: -members-san-jose-state-u-urge-outside-review-institutions-governance

In the meantime, the leader of the SJSU experiment, Sebastian Thrun of the MOOC provider Udacity, recently told a writer for the magazine Fast Company that he is downshifting his MOOC ambitions from the loftier climes of higher education learning to the supposedly much more straightforward and achievable workd of workforce training (????), specifically for computer scientists:

Although Udacity continues its work with SJSU after a very poor success rate (25% passed) compared to face-to-face classes (53% passed) last spring, the (entirely predictable!) lack of evidence of actual course completion and learning in many MOOCs has been (finally!) grabbing media  attention lately:

"And yet, all of these efforts have been hampered by the same basic problem: Very few people seem to complete courses when they're not sitting in a lecture hall. Udacity employs state-of-the-art technology and sophisticated pedagogical strategies to keep their users engaged, peppering students with quizzes and gamifying their education with progress meters and badges. But a recent study found that only 7% of students in this type of class actually make it to the end. (This is even worse than for-profit colleges such as the University of Phoenix, which graduates 17% of its full-time online students, according to the Department of Education.)"

"Among those pupils who took remedial math during the pilot program, just 25% passed. And when the online class was compared with the in-person variety, the numbers were even more discouraging. A student taking college algebra in person was 52% more likely to pass than one taking a Udacity class, making the $150 price tag--roughly one-third the normal in-state tuition--seem like something less than a bargain. The one bright spot: Completion rates shot through the roof; 86% of students made it all the way through the classes, better than eight times Udacity's old rate. (The program is supposed to resume this January; for more on the pilot, see "Mission Impossible.")

I have been reporting these media reports rather dutifully for several months now, and at last, I have an actual opinion and perspective to express. Let's start here: Note that the experience with the SJSU developmental math MOOC was somewhat different from other MOOCs.  The low completion rates that have plagued most online learning for the past 15 years were not replicated. In this case, students finished; they just didn't pass. This is an interesting finding that no one seems to be discussing much. 

One wonders why these students didn't do what most savvy students do when they are gaming the traditional higher Ed system: Drop the class you're clearly not passing to "maintain" your GPA--and the illusion that you are a traditional "good student."  Perhaps they're not savvy students. There is good circumstantial evidence to support this claim. For example, perhaps not being savvy students is why they are in dev Ed. Somewhere along the line, they didn't get the memo about playing the school game. They didn't cram before the placement test to avoid the dev Ed class when they came to the admissions office. But make no mistake--the savvy students do that. Further, we are at SJSU, a tier 2 college. These students likely didn't take lots of SAT prep classes and GPA-goosing AP classes in high school either. After all, that is what the savvy students do. So these students were able "only" to get into SJSU rather than a supposedly "better" institution of higher Ed.

And think about that last statement. What is it with the brand names in higher Ed? I have worked with some very terrific SJSU grads and some pretty lame Stanford grads. Enough on the brand name nonsense! Is anyone else noticing how perverse things have become? What does any of the business of being a savvy student and embroidering your resume with the right schools have to do with actual learning? It really boils down to a ridiculous level of image management. Do we really want, as a society, our youngest, most idealistic citizens focused on image management, which is really what being a savvy student has become? Where does that lead us, as a society? 

Now, that's my opinion. Here is the substantive perspective part as an educational researcher: What is the difference between finishing and passing, and what does this tell us about learning? Could it be that the 75% of folks who did not pass may tell us something about the failures of thinking you can simply put typical course material online (video lectures, slides, readings) and then make magic happen? What needs to change in the design of learning content to permit the other 75% to learn online? As it happens, there is plenty of research into online learning that has clarified that point, and yet no one discusses that research in these various reports.

Back to opinion: Instead, the reporters seek the answer from entrepreneurs with pedigrees from brand name institutions who have not spent not more than a moment studying pedagogy. Hey, why would they? Studying pedagogy--let's face it, studying "education"--isn't good for their "image." Education is not a prestige field. Everybody knows that, right? Why do it then?

Back to perspective: But, for the record, here are some design principles we know... You need social support to learn, mainly to provide what is called learner "self-regulation," which means keeping the learner focused consistently to finish. (In the old days when mostly women taught in schools, we called it "helping." But I digress...) Anyway, you also need multiple representations of knowledge, frequent feedback, and -- get this -- multiple trials to reach mastery. Yes, it turns out that the notion that you learn "fast and efficiently" is simply goofy. You learn slow and painstakingly, through lots of hard work and effort. Yes, the sad truth is that we must be "grinds" to learn. But again, what would saying that do for our "image?" Oh by the way, that reminds me, we in the Ed field are calling hard work "grit" now. 

Could it also be that there is an implicit identification of the central problem of the design of traditional higher Ed learning in these data? Perhaps those 75% non-completers represent the kinds of students who do not learn very well under the old model of "time pressured, image-managed, competitive grade learning" the first time around, but ultimately could learn if permitted to study with some form of support on their own time and less pressure around image management? The MOOC experiment does not answer these questions, but these results raise such questions. 

Instead, in the MOOC coverage, we see the predictable narrow focus on one data point or two (pass rates! completion rates!), and on to the next personality piece about the next entrepreneur with something to sell us. There is very little thought about what the evidence of pass and completion rates tells us about the design of education. Instead, we see the predictable political positioning to protect the status quo without any overt evidence of a good faith effort to try to improve the design of the status quo (e.g., SJSU and UCSC faculty "protests"). We see the sort of half-baked questioning about the "social justice" of such proposed innovations without really looking at the details of what is actually happening in the cases of the oft-quoted stereotypes of the "underrepresented, needy student." So, we use these stereotypes as a prop to block change and we learn nothing about how the design of learning could be improved to actually help them (and, hey, if we are honest, everyone else too)-- (for the PC perspective, see this: ). 

None of this is constructive or focused on solving the education problem in this country. That takes real work and a laser focus on the problem and designing a solution -- and not a focus on the personalities, political strategy, or political correctness. 

Be advised, that kind of focus on the design of learning IS happening, but coverage of that work is in short supply in the media of all forms and all channels. 

Why not? Because it isn't a sexy story, and that isn't good for the image of the reporter or the media outlet publishing it.