The Missing Middle of Student Success Part 1

Was this forwarded to you by a friend? Sign up for the On Student Success newsletter and get your own copy of the news that matters sent to your inbox every week.

This is the first part of an article version of a presentation I recently gave to the Canadian Association of Instructional Designers (ACCP-CAID). For the truly keen among you I have included a link to the whole presentation at the end.

When a student starts struggling at university, where do we look first?

There is a fairly predictable list of explanations. Perhaps the student lacks motivation or isn't managing their time effectively. Maybe they're facing financial or personal difficulties, arrived insufficiently prepared for academic work, or haven't taken advantage of the support services the institution provides.

These are all reasonable possibilities. But how far down the list do we get before asking whether something about the courses the student is taking might be contributing to the problem? We have become very good at identifying students who are struggling and developing services to help them. We are considerably less practiced at examining the educational experiences we have designed for them.

Where do we look for student success?

I find it useful to distinguish three areas in which institutions can influence student success:

1. Student support & services: Advising, financial aid, tutoring, mental health services, belonging, and the other supports that help students navigate the institution.

2. The curriculum and pathways: The structure and sequencing of programs, course availability, scheduling, prerequisites, and pathways to completion.

3. Inside the course: How learning is organized, how students interact with material and with one another, how assessment and feedback work, and how the course is designed to support learning.

All three matter. Yet institutional student success strategies tend to give far more explicit attention to student support and curricular pathways than to what happens inside courses.

One indication of how we think about student success comes from this Chronicle of Higher Education survey, which asked university leaders, faculty, and students which institutional offices or units needed improvement to help retain and graduate more students. Academic advising, financial aid, career services, and other support functions feature prominently.

The framing of the question itself is indicative of how little attention we pay to courses. Respondents are asked to think about student success in terms of institutional offices and units. Courses, teaching practices, assessment, and course design don't really fit into that framework. Faculty appear as a category, but the design of the educational experience does not. This is only one survey, of course, but it illustrates how easily our organizational structures become the framework through which we think about student success.

This tendency to look outside the course isn't confined to how we organize our student success efforts. It is also evident in how we respond when a particular course has persistently high rates of D, F, and withdrawal grades. Here, the problem has been identified at the course level, yet many of our familiar interventions still focus on helping students get through the course rather than examining how it is designed.

These responses can be valuable, and some have strong evidence of effectiveness. But how often do we begin by examining the course itself, including its workload, assessment structure, feedback, and learning activities?

It is entirely possible to improve the support available to students while leaving unchanged aspects of the course that are contributing to their difficulties. We may be helping students navigate problems that we could also be trying to reduce through better course design.

There is another curious aspect to this imbalance. Students spend far more time in courses than they do interacting with institutional support services.

Consider Jane Student, whose typical week includes lectures, labs, assignments, readings, and assessments. She might meet with an academic advisor once a semester, visit the tutoring center occasionally, and hopefully never need emergency financial assistance.

There is also the question of whether students use the services available to them. Research from Tyton Partners has repeatedly documented gaps between the availability of support services, student awareness of them, and actual use. Courses, by contrast, are an unavoidable part of the educational experience.

The frequency of these encounters matters. A student might see an advisor once or twice a semester, but they experience the design of their courses every week. A confusing assignment, poorly organized learning materials, an unrealistic workload, or feedback that arrives too late to be useful can create difficulties that recur throughout the semester. Conversely, well-designed courses can provide structure, clarity, meaningful interaction, and opportunities to recover from setbacks as part of the student's everyday experience.

This doesn't diminish the importance of advising and other services. It does suggest that we may be overlooking one of the most frequent and consequential points of contact between students and their institutions.

The missing middle

One reason we overlook courses is that our approaches to measuring student success tend to skip over them. We collect enormous amounts of information about students: their demographic characteristics, prior academic performance, socioeconomic circumstances, first-generation status, and much else besides. We also collect extensive information about outcomes: course completion, credit accumulation, persistence, retention, graduation, and increasingly employment and earnings.

But between the student and the outcome lies something rather important. Students take courses. Those courses have been designed in particular ways. Students encounter learning activities, assessments, feedback, expectations, and interactions that shape their experiences and what they learn.

Yet this middle part of the process is much less systematically measured and understood than either the characteristics students bring with them or the outcomes they eventually achieve.

This matters because it limits our understanding of why students succeed or struggle. If students fail a course, we can often identify characteristics associated with their likelihood of failing. We may even be able to predict who is most at risk. But that doesn't necessarily tell us what happened in the course, why it happened, or what we could change to improve the outcome.

Consider two students with similar academic backgrounds and circumstances taking different sections of the same introductory course. One encounters clearly sequenced assignments, regular formative feedback, and opportunities to recover from early difficulties. The other encounters a large assessment late in the semester, limited feedback, and little indication of how well they are progressing until it is almost too late. A model based primarily on student characteristics might assign these students similar probabilities of success, even though their experiences of the course could be very different. This is a hypothetical example, but it illustrates what we lose when we treat the course as a largely neutral environment through which students pass. The course itself is a variable that may help explain student success or failure.

We have built increasingly sophisticated systems for identifying students who may fail while devoting much less attention to understanding whether the design of their courses is contributing to that failure.

We know course design matters, but the evidence is fragmented

The research approaches this question from several directions, examining teaching practices, students' reasons for withdrawing, assessment structures, and characteristics of courses associated with student outcomes.

One example comes from Freeman and colleagues' meta-analysis of 225 studies of undergraduate STEM education. Across the 67 comparisons reporting failure rates, the average was 33.8% in traditional lecture courses and 21.8% in courses using active learning. This doesn't tell us which overall course designs are most effective, but it is powerful evidence that the choices we make about teaching and learning can affect whether students pass or fail.

A study by Hachey, Wladis, and Conway offers another perspective. Examining students' reported reasons for withdrawing from online and face-to-face sections of a large course at the City University of New York, the researchers found that course-related factors featured prominently, particularly workload and course characteristics. The findings don't establish that course design caused students to withdraw, but they suggest that students' experiences of courses deserve much more attention when we investigate withdrawal.

Course characteristics and lack of time feature prominently in students' reasons for withdrawal.

A study based on interviews with professors at the Universitat Oberta de Catalunya examines how assessment design may contribute to withdrawal. In courses organized around continuous assessment, the first assignment could become a decisive hurdle. Students who missed or failed it had limited opportunities to recover, and withdrawing from the assessment process also meant losing access to subsequent opportunities for feedback and learning.

This is a useful reminder that flexibility isn't simply a property of online delivery. It is also a consequence of decisions about assessment timing, workload, deadlines, and opportunities to recover from setbacks.

Other researchers are beginning to investigate the course itself as a unit of analysis. Freels and Sankaranarayanan examined 925 undergraduate course sections to explore associations between LMS course-site characteristics and D/F/W rates. Their measures of course design were limited, but the study is interesting because it shifts attention from what students do in the LMS to characteristics of the learning environment they encounter.

These four studies are neither an exhaustive review of the research nor definitive evidence about which approaches to course design work best. They examine different aspects of teaching and course structure, use different methods, and measure different outcomes. I have chosen them to illustrate that there is research connecting what happens inside courses to student outcomes, including course failure and withdrawal.

There are still important gaps in what we know. Much of the research examines individual teaching practices or components of courses rather than course design as an integrated whole. And we know considerably less about how improvements in individual courses translate into longer-term outcomes such as persistence and graduation. But there is enough evidence to make the course itself a serious object of investigation in our student success efforts.

Why do we keep overlooking the course?

None of this means that course design is the answer to every student success problem. Financial insecurity, family responsibilities, inadequate academic preparation, and other barriers remain enormously important.

But we have enough evidence to establish that what happens inside courses matters, and good reason to believe that course design can either reduce or compound the difficulties students face.

What I find difficult to understand is why this has had so little influence on how we organize our student success efforts. We have invested heavily in identifying students who are struggling, building services to help them, and improving their pathways through programs. Yet even when we identify a particular course as a persistent source of failure, our instinct is often to look for ways to help students get through it rather than asking whether something about the course should change.

Next week in Part 2, I'll explore how our definitions of student success, our research traditions, and the way institutions organize responsibility for improving outcomes have contributed to this blind spot, and why it is becoming increasingly important to address it.

As promised, here is the original presentation.

Know someone who works in student success? Forward this to them. Know someone who designs courses? Forward it to them too. With any luck, we can get the two groups talking to each other.

Thanks for being a subscriber.