Tag Archives: student learning outcomes

Inputs and outcomes – how we wound up with two systems for grading students in english language programs

row-students-doing-examAssigning final grades to students has been done in various ways over the years. In some contexts, everything rested on a final exam – this was the case with the O-level and A-level exams I took in a British high school ‘back in the day.’ Then ‘continuous assessment’ became popular, making the final grade a composite of grades for assignments completed during the course, either with our without a final exam.  This approach became popular in U.S. intensive English programs, where the final grade might be made up of homework assignments, projects, tests and quizzes, and the usually ill-defined ‘participation’ by the student. 

But English language programs, like all other schools in the U.S., became caught up in larger forces that had an enormous impact on how students were evaluated. Following the successful launch of the Russian satellite Sputnik in the late 1950s, there was much nail-biting over the quality of American education, culminating in the ‘A Nation at Risk’ report in 1983, which painted an anxiety-inducing picture of failing U.S. public schools. 

In the years following the publication of the report, the means of defining quality in education were questioned. The focus of quality tended to be on inputs – number of hours in class (this is where the ‘credit-hours’ system came from), teacher qualifications, teaching methods, and so on. Many schools in competitive environments still make such inputs the basis of their quality claims – “highly qualified teachers!” “innovative teaching methods!” 

But those raising the red flag about school quality were less concerned with inputs and more concerned with what the students came out of their education with – that is the outcomes, or as they have come to be known, student learning outcomes, or SLOs. No matter how great the inputs, if students were not learning useful knowledge and skills for the job market, the education they were receiving was not valuable. The solution was to turn the traditional curriculum planning process around and start at the end by first defining the desired outcomes, and having course design lead to student achievement of outcomes. 

This enabled education bureaucracies to hold schools and teachers accountable: school and teacher quality could be judged not by the quality of the teachers or the hours spent in class, but by the extent to which students were meeting the defined learning outcomes. In the public schools, those outcomes were assessed by standardized tests, and schools were judged and ranked by how well students scored on those tests. (The downside of all this was that quality aspects of school such as adequate breaks between classes and time for the arts, music and sports suffered as schools honed in on efforts to increase standardized test scores in math, science, and English.)

Back to grading. ESL teachers have for many years been used to giving final grades based on a combination of test and quizzes, homework assignments, projects, participation, and final exams. But with a shift toward accreditation of English language programs – mandatory in many cases, voluntary in others – teachers in those programs are now required to fall in with the requirement to define learning outcomes at the outset, and assess and evaluate students with sole reference to the students’ achievement of the outcomes. This has to be done at the school level, and it results in a greater standardization of curricula, syllabi, and assessments in schools. Schools are required to record and analyze the data arising from the assessment of SLO achievement. Decisions about whether a student may progress to the next level of study or complete the program successfully must be made solely on the basis of whether the student achieved the learning outcomes. 

The result is that schools have to take a mixed approach to grading students. Schools may still assign a traditional grade based on continuous assessment and participation, but they must also maintain a system that isolates achievement of the student learning outcomes and makes promotion and completion decisions based on that. What’s certain is that choosing one or the other of these two systems is not possible – both are needed. Yes, we can agree that it’s important for students and their sponsors to understand what the expected outcome of a course or program was and whether the student achieved it. This kind of accountability is needed when many are questioning the dollar value of their education. But as educators we also want to know whether students engaged with the educational process – collaborated with peers, challenged themselves on difficult projects or assignments, sought help and advice and gave them to others in turn. How the students got there is important to us, and still largely defines the benefit of studying at one school rather than another. 

And so we ended up with two types of student assessment and evaluation, one based on inputs into the process and ongoing or continuous assessment, the other on outcomes. Both systems are here to stay, and educators need to be familiar with the rationale and procedures for each of them. 

Background photo created by pressfoto – www.freepik.com

Why you should be teaching to the test

Back in the day, if you were ‘teaching to the test,’ you weren’t really doing your job as a teacher. You isolated the pieces of knowledge and the skills that you knew would come up on the test and taught them to the exclusion of broader educational activities that might have enriched the students’ experience. You might have done this to ensure a high pass rate, which reflected well on you as a teacher if the higher-ups were judging you on your students’ test scores. But teaching to the test was frowned upon as a kind of shortcut for both teacher and students.

Since the advent of the accountability movement, teaching to the test is exactly what you should be doing. In the currently popular paradigm, schools and teachers are accountable for students’ achievement of defined learning outcomes, expressed in behavioral terms: “The student will be able to…” Examples in language programs are:

  • give a five-minute presentation on a topic of personal interest
  • write a five-paragraph narrative essay
  • summarize, in writing, a radio news story
  • re-tell orally the plot of a short story

If the learning outcomes are well-conceived, they should be a guide to what the test – let’s call it an assessment – should be. How do you assess students’ ability to give a five-minute presentation? Have them give a five-minute presentation at the end of the course. How do you assess their ability to write a five-paragraph essay? Have them write a five-paragraph essay. And so on. (The specifics of the assessment will need to be made clear, and rubrics provide a means of determining the students’ level of performance.)

And so what is the best way to prepare students to give a presentation? Teach to the test and have them practice giving presentations. To write an essay? Teach to the test and have them write essays. This is what I mean when I say ‘you should be teaching to the test.’

Some summative tests – including many of those provided in published textbooks – are not good tests to teach to. A grammar gap-fill isn’t much use in giving information about a student’s final level of achievement, unless your learning outcome is ‘be able to provide the appropriate grammatical forms in a gap-fill test.’ That’s not a very useful outcome to anyone, though this activity might help promote student learning along the way. A well-defined learning outcome is a behavior that you can describe to a future employer or school indicating the student’s ability to do useful things with language.

So let’s embrace teaching to the test – as long as you have good learning outcomes and a corresponding test that assesses them appropriately. (If you don’t, maybe it’s time for an overhaul.) And while we’re doing that, let’s not forget that games, songs, poetry, sharing experiences, and laughter create a positive, human environment that leads to unanticipated learning and ideal conditions for students to learn.

 

 

The baby and the bathwater

If you read anything about curriculum design these days, or attend a presentation or workshop, you will learn only one model. Backward design starts at the end, defining student learning outcomes, then working backward through assessment, teaching and learning objectives, content and sequencing, and finally teaching and learning. This approach to curriculum design is so pervasive that anyone new to education might think there is no other way.

Thanks to the recently published second edition of Jack Richards’ book Curriculum Development in Language Teaching, we can learn about, or be reminded of, the alternative. Although it was never recognized with the name, forward design took the opposite approach: decide on your content and sequence it, teach, assess, and evaluate with a grade. This used to be the standard way to design  and teach a course in higher education. And the fact is, many teachers who used this approach continue to do so, or try to do so, in tension with an institutional backward design ethos promoted by accrediting organizations.

Backward design has the benefit of identifying (in theory anyway) student needs, developing measurable learning outcomes, and demonstrating program quality through analyzing and publishing student success rates against those outcomes. There is accountability for student success, which is important in an increasingly competitive environment in which customers (students and their sponsors) demand transparency in results. Yet as we are reminded in Richards’ book:

“The experience of language teachers today is often one of diminished classroom autonomy and of being managed by business-savvy administrators.” (Hadley, 2014, cited in Richards, 2017, p. 228)

This is because teachers are increasingly told to work with standardized outcomes and learning objectives, demonstrate that their assessments address the student learning outcomes, and use textbooks that deskill teachers by driving many of their instructional decisions (Richards, p. 247).

If backward design has introduced an obsessive focus on outcomes, or product, forward design was always much more about the educational process. A process curriculum:

“…is person-centered, considers users’ needs, identifies problems rather than rushing to solutions, and does not rely on top-down mechanistic models but is a process that works towards interaction between participants at all levels.” (Kennedy, 2013, cited in Richards, 2017, p. 227)

The emphasis is more on what teachers might call the art of teaching, making meaningful experiences for students, and letting the teaching and learning follow, to an extent, students’ needs and interests as they arise during the course. In a process approach, students and teachers form a learning community that explores together, often with an unclear destination. While backward design requires assessment of students’ achievement of learning outcomes exclusively, forward design grades participation, homework, and attendance, because these are indicators of students’ engagement in the process of learning.

While I don’t think we can or should return to a purely process-focused approach, we should consider what is lost when we throw out the process approach baby with the forward design bathwater, and embrace a product-based approach too strongly. Now that backward design is established as the accepted way to design curriculum, I hope that we can start to talk about how to maintain art in the process of teaching, one that recognizes teacher creativity, responds to the students in the class and the needs of the particular group, and provides a unique and unrepeatable experience for learners.

Thanks to Richards’ book, we may be able to start having that conversation.

Jack Richards’ book Curriculum Development in Language Teaching is published by Cambridge University Press.

What makes a school?

We are used to talking a lot about quality in education. It used to be normal to describe quality in terms of inputs: faculty to student ratios, faculty degrees, school facilities, test scores of the incoming class, and so on. More recently, we have been pressured by government departments, funding agencies, and accreditors to prove our quality in terms of outcomes: can-do statements, demonstrable skills gained, behavioral changes in our students at the end of their course or program.

The input-outcome paradigm for determining quality is adequate enough if we are in a production mindset. In this mindset, education is analogous to the production of goods or services. “Our shoes are made of the finest Italian leather” is an input-based quality claim. “Kills 99.9% of bacteria” is an outcome-based quality claim. Similarly, “highly qualified and friendly teachers” is an input-based quality claim. “Our Academic English program will prepare you to succeed at a university” is an outcome-based quality claim.

I wonder if this paradigm tells the whole story about the quality of an education? This is important to consider for intensive English programs (IEPs), because they are increasingly competing against other models of English language education and training, such as in-country classes, online tutoring and courses, even apps on devices. The producers of these alternatives can point to their inputs and outcomes and on that basis apparently offer a viable alternative to an intensive English program.

But I want IEPs to revive the notion of the school, a concept that is far broader than mere production or educational delivery. If you think about your own educational experiences which were the most memorable? Which shaped you most as a human being? Were you most influenced by a program that had clearly defined student learning outcomes? Did you learn the most from the teacher who was the most highly qualified? Unlikely.

A school is a place where community is formed. Diverse (however you choose to define diverse) students, teachers, and staff, come together in the shared enterprise of teaching and learning. There is social interaction, friction, the challenging of dearly-held beliefs. There is laughter, disappointment, joy, and frustration. Students encounter teachers with idiosyncrasies that they will never forget. A story heard sticks in the mind forever. A kindness is extended and remembered.  All of this is the quality of an education that is not recognized by the production paradigm of educational quality. It cannot be measured. You cannot really put a price on it.

Think about this kind of quality when  you go to your school in the morning. You still have to hire qualified teachers and measure student learning. But you have the opportunity to create unique, precious, and lasting experiences for your students, staff, and faculty. This is what really makes a school.