All posts by Alan Broomhead

Challenge and change in intensive English programs

From left: Bill Hellriegel, Carol Swett, Michelle Bell, Amy Fenning, Alan Broomhead

Challenges over the past few years have deeply impacted intensive English programs, forcing irreversible changes in their organizational cultures that result in anxiety and tension, but also innovation and adaptation. That was the theme of a panel session, “Organizational Culture in University and Proprietary IEPs: Challenges and Changes,” presented by Michelle Bell (University of Southern California), Amy Fenning (University of Tennessee at Martin), Bill Hellriegel (Southern Illinois University), Carol Swett (ELS Language Centers at Benedictine University, Illinois) and myself at the TESOL International Convention on March 28. Recognizing the cultural types of IEPs and how they are affected by changes is the first step in adapting and surviving in an increasingly competitive field.

IEP cultures can roughly be divided into collegial and managerial types, following Bergquist and Pawlak’s (2007) typology of academic cultures. A collegial culture, more likely to be found in a university-governed IEP, is faculty-focused, with faculty scholarship and teaching, academic autonomy and freedom, and faculty ownership of the curriculum as the organizing principle. A managerial culture is administration-driven, motivated by considerations of fiscal responsibility and effective supervision, and organized by systems, processes, and standards.

The massive shift to accreditation in IEPs has moved collegially-oriented programs in a managerial direction. Faculty are required to plan, teach, and assess in compliance with program-wide student learning outcomes; policies and procedures have to be written and followed; and program success is measured by data, which has to be systematically collected, analyzed, and evaluated. Proprietary IEPs are seeing a a shift in the other direction: faculty standards require minimum levels of certification, experience, and ongoing professional development, and these are affecting faculty hiring and employment practices in many proprietary programs.

The severe enrollment challenge of the past two years has also affected both types of program. University IEPs are becoming more revenue-driven and entrepreneurial, actively seeking new recruitment partnerships and designing new programs – such as short-term high school programs – to respond to changing demand. Faculty may have little say in these initiatives. Meanwhile, proprietary IEPs are increasingly developing conditional-admit and TOEFL-waiver agreements with partner universities, requiring them to make programs more academically-focused and hire masters-level teachers who are qualified to teach English for academic purposes.

These are ground-shifting developments, and program leaders who recognize the need to address profound cultural change in their organizations – and not just surface-level adjustments – will be in the strongest position to navigate these challenging times.

Reference
Bergquist, W.H. & Pawlak, K., Engaging the Six Cultures of the Academy, Jossey-Bass 2007

Is language proficiency measurable?

For the past few years, ESL teachers have been pushed to focus their efforts on helping their students achieve ‘measurable objectives.’ Am I the only person who finds this a strange idea? Measuring something is a matter of determining how much of something there is. To measure, we need a unit of measurement: inches and feet, centimeters and meters, pounds and ounces, grams and kilograms. By agreeing on standardized units of measurement, we can determine, objectively, the quantity of something.

Could we use this approach to evaluate a work of art or a piece of music? What would Van Gogh’s Starry Night or Mozart’s Eine Kleine Nachtmusik measure? What unit of measurement would you use? Impossible, because we are not dealing with quantity (the amount of paint or size of canvas, or the number of notes) but with quality. There is no unit of measurement for quality.

When we want to assess and evaluate language are we dealing with something more like a distance or weight, or like a work of art? Is it the quantity of language or the quality of language we want to know about?

If you believe it is quantity, then you might say we can ‘measure’ the number of words a student has learned, or the number of grammar points. These don’t work as units of measurement, though, because defining exactly what is meant by ‘learning a word’ is complicated, as language teachers know. A student may be able to say it but not spell it, may use it but in an inappropriate context, may not recognize it when written down but may hear it in another person’s speech, may forget it on one occasion but recall it on another.

The psychometric testing tradition has given us tests which appear to measure learners’ language ability by assigning a score and thus appear to be objective. Classroom assessments by teachers are often regarded as a subjective second-best.

We should move away from the notion that language proficiency is measurable, and that test scores give us an ‘objective measure’ of a learner’s ability.  Language should be evaluated qualitatively, by people, using rich description rather than fantasy units of measurement that give the false impression of objectivity.

The Inadequacy of “ESL” for International Student Preparation

Wrapped up in the term ESL (English as a Second Language) is an assumption that language, above all, is what students need to succeed in an English-speaking environment. The same kind of assumption can be found in the name of the most popular standardized U.S. admissions test for international students, the TOEFL (Test of English as a Foreign Language). The CEFR (Common European Framework of Reference for Languages) lists levels of language proficiency by skill, and many ESL programs continue to organize their curricula on the basis of Listening, Speaking, Reading, and Writing skills. The field of SLA (Second Language Acquisition) is a major feeder discipline in ESL teacher preparation programs.

A focus on the acquisition of language skills gets us only so far if we are preparing an international student for academic work in an English-speaking setting.  One thing among very many that this student needs to do is to read a text critically and offer an original, well-thought-out, supported, and argued response. The student may need to argue that response in class, and defend it against other points of view, in an assertive yet diplomatic manner. To be taken seriously, the student will need to behave in what is recognized as a normal and appropriate manner in that environment – and know when and how to revert to a more informal style when class ends. All of this goes far beyond language skills.

What this student needs to learn is what James Paul Gee in Social Linguistics and Literacies refers to as Discourse (with a capital D). Discourse “is composed of distinctive ways of listening/speaking and often, too, writing/reading coupled with distinctive ways of acting, interacting, valuing, feeling, dressing, thinking, believing with other people and with various objects, tools, and technologies, so as to enact specific socially recognizable identities engaged in specific socially recognizable activities” (p. 152). These are less language skills than “social practices into which people are apprenticed as part of a social group” (p. 76). As we move in different Discourse communities, we need to know how to play our part and be recognized as a legitimate member of each community. Discourses are mastered by “enculturation…into social practices through scaffolded and supported interaction with people who have already mastered the Discourse” (p. 168).

This helps us understand why any program of learning that reduces preparation to language skills is inadequate. Students need to learn the ways of interacting, believing, valuing, and effectively being in the academic Discourse community. University IEPs (intensive English programs) teach English for academic purposes, but they still largely identify as English language programs with language-based missions, their faculty members have degrees in teaching English, and classes are often language skill-specific. They are often isolated from the rest of the campus, and therefore don’t allow for the kind of apprenticeship into the social practices of the campus that would make international students full members of the Discourse community.

In order to address this wider understanding of international student preparation:

  • Intensive English programs should ensure their missions, their curricula and teaching, and their names, encapsulate the full meaning of international student preparation – not simply ESL.
  • University administrations should make international student preparation a task for the whole university, supported by, but not the sole responsibility of, an intensive English program. The IEP’s efforts should be integrated into a campus-wide strategy for international student preparation.
  • Universities should not expect that simply raising the required TOEFL scores will improve international student outcomes – students need induction into the Discourse community, not just a higher TOEFL score.
  • ESL teacher preparation programs need to include coursework on social literacy and in preparing students to enter and successfully navigate their target Discourse communities.

Some of this has already been achieved. Many IEPs recognize their wider mission of orienting students into academic culture, and more recently,  pathway programs have been structured to provide ESL support alongside credit-bearing classes that, in theory at least, offers an apprenticeship into the academic community. But there is a long way to go before the notion of Discourse communities drives international student preparation beyond the inadequacy of “ESL.”

Reference
Gee, J.P., Social Linguistics and Literacies, 5th Ed., Routledge 2015

Working with the tension between language test validity and reliability

The combination of validity and reliability is the holy grail when it comes to language assessment, yet these two qualities are always in direct tension with each other. This can create a challenge when English language programs try to put in place effective measures of language learning, and especially when they have to convince their accreditors that they’ve done so. Student achievement standards are frequently not met in accreditation reviews for precisely this reason.

An assessment is valid if it measures what it is supposed to measure. So, a multiple choice test is generally not a very valid means of testing speaking ability; nor is a gap-fill test a very valid way to determine whether a student has learned to use a grammar structure in communication. On the other hand, a student presentation might serve as a useful basis for a valid assessment of speaking ability, and a speaking or writing test that elicits a target grammar structure would bring to light a student’s ability to use grammar.

An assessment is reliable if would yield the same results for that student if administered by a different person or in a different location. An in-class presentation or role-play assessed by the class teacher is vulnerable to having a low level of reliability, since the test conditions would be difficult to reproduce in another class. The TOEFL iBT is probably the gold standard for test reliability, with extremely detailed protocols for ensuring the uniformity of the test-taking experience for all students, and two-rater grading of written and spoken assignments.

You can probably see the tension: the greater the validity, the harder it is to attain reliability; the greater the reliability, the harder it is to make the test valid (in the three-hour iBT, the test taker is not required to interact with a single human being).

To increase the reliability of valid assessments, programs can:

  1. use a common set of learning objectives across the program and hold teachers accountable for teaching to them
  2. use standard assessment rubrics across the program
  3. calibrate grading through teacher training
  4. have more than one person assess each student’s performance.

These measures might generate pushback among faculty in some university programs.

Unfortunately, there aren’t any great ways to increase the validity of highly reliable of achievement tests. Doing so would require standardizing the teaching – teaching directly to the test – which nobody in an IEP wants, except in a course specifically for test preparation. Programs that use external standardized tests for level promotion are not using a valid means of assessing what was taught (since the test makers don’t know what was taught).

Instead of seeking the absolute standard of ‘assessments that are valid and reliable,’ we need to

  1. start by creating assessments that are valid – that measure precisely what was taught and was supposed to be learned; and then
  2. design and implement measures to reach as high a level of reliability of those assessments as is possible and practical.

Using this approach is a recognition that you can’t have it all, but you can work within the tension of validity and reliability to reach a satisfactory compromise.

Teacher Leadership for Program Improvement and Development

“The function of leadership is to produce more leaders, not more followers.” – Ralph Nader

If you’ve read anything from the popular leadership literature, you’re probably familiar with the prescriptions for strong leadership: confidence, vision, integrity, charisma, and so on. Analyses like these are premised on positional leadership, or what James Spillane called ‘the heroics of leadership’ – the notion of the strong leader standing at the head of an organization, leading the way. Think Jack Welch, Jeff Bezos, Larry Page.

There is an alternative to this view of leadership. Distributed leadership begins with the idea of leadership tasks and asks how these tasks can be accomplished by individuals across an organization. In an intensive English program (IEP), leadership tasks can include curriculum development, teacher professional development, and materials writing, among others. In many management-driven IEPs, these tasks are still carried out by non-teaching staff. But there are advantages to bringing teachers into leadership tasks and distributing them more widely.

Bringing teachers into leadership roles – whether formally or informally – is great professional development for teachers in a profession that otherwise may have no upward career path. It can give teachers a sense of fulfillment from helping other teachers or the program as a whole, and can serve to retain talented teachers by more thoroughly networking them into the program. And it serves the program by drawing on the enormous pool of talent that teachers bring, and which is otherwise confined to their classrooms.

There are challenges to introducing teacher leadership. The role of leader is not a part of the ‘role schema’ of a teacher – teachers are socialized to be followers and may not see themselves as leaders. Teachers need to reframe their self-identity to include leadership. External constraints include how teachers are perceived by other teachers when they take on leadership roles – there is often a culture of egalitarianism among teachers, and teacher leaders may draw disapproval from their teacher peers. Finally, structuring  teacher jobs to include leadership can be a challenge: teachers may lack the time or energy to take on leadership roles, and the program may be limited in its ability to compensate them or structure their jobs appropriately to accommodate leadership tasks.

For teacher leadership to succeed, therefore, it requires strong support from the program director or academic director. Teachers need to be empowered to take on leadership roles, and they need to be given the time and resources to succeed. They may also need coaching on how to manage their identity as teacher leaders, and they may need to be protected from resentment that may occur among peers as they take on leadership roles.

IEPs serve their students best when everyone has an opportunity to contribute to decision-making and program improvements. Here are some examples of teacher leadership I’ve seen over the past few years:

  • A teacher took the initiative to start a reading corner to encourage extensive reading among students. With management support, she built up a collection of books for ESL learners at all levels, and now runs the reading corner as part of her job.
  • A teacher started a peer observation group to encourage teachers to visit each other’s classes and give feedback.
  • Teachers took on program coordinator roles for which they were given release time from their teaching.
  • Teachers played a role in the selection and hire of new teachers.
  • Teachers delivered professional development workshops for their peers, or organized a professional development program.
  • Teachers self-organized into ‘level’ teams to collaborate on in-class projects and assessments.
  • Experienced teachers mentored newer teachers.
  • Teachers became subject or skill experts in the program.

Think about how teacher leadership can be extended in your program.

Can an intensive English program go virtual?


Business image created by Jcomp – Freepik.com

Along with continuing enrollment challenges for university and proprietary intensive English programs (IEPs) comes a demand for fresh ideas, re-thinking the model, and new types of programs that meet the needs of today’s learner. Given the rise and ubiquity of online learning, many IEP leaders are asking whether and how they might take their programs online.

Online ESL is already big business, with many startup companies connecting students and teachers in different parts of the world through synchronous online lessons. The first challenge for IEPs thinking about breaking into this market is how to devote the resources to develop and market an online program while not diverting resources from their current on-ground operations. But the greater challenge is how to take a model that has developed and established its value in one format (on-ground) over many years and adapting that model to a new, online format.

In their book on academic cultures, Bergquist and Pawlak identify the ‘tangible’ culture and the ‘virtual’ culture as two cultural types that may be in tension with each other. IEPs have developed around a tangible culture that emphasizes location, student life, interaction with local people, institutional facilities – the whole student experience. Additionally, as a result of visa regulations, they have built curricula and weekly schedules that prioritize compliance over the needs of students (example: there is no strictly educational reason why students should spend 18 hours in class in order to learn a language). This model has been valuable to the many thousands of students who have attended IEPs. But how much of this on-ground value can an IEP retain when it puts its programs online? And with many providers in the online market, most of which are specialized, agile, not tied to an on-ground model, and highly entrepreneurial, how feasible is it for established IEPs to make significant inroads into this market?

My prediction is that most intensive English programs will not play a significant role in the online ESL market, nor will they want to break from the on-ground model they have spent years nurturing. To survive, they will need to continue adapting to the needs of current students who want to travel for an education. Right now this means offering short, specialized programs, and pathways into universities. While the demand for intensive English programs is currently in a slump and may never bounce back to the numbers of recent years, the tangible academic culture is not going away, and there will always be value in traveling for a global, intercultural, and language education. IEPs need to continue working to demonstrate that value to tomorrow’s students.

Reference
Bergquist, W.H. & Pawlak, K., Engaging the Six Cultures of the Academy, Jossey-Bass 2008.

Keeping your Intensive English Program Relevant on Campus


Created by Ijeab – Freepik.com

These are trying times for many on-campus intensive English programs. Enrollment and revenue are down, and there is increased pressure from senior administration for many IEPs to demonstrate their continuing relevance and usefulness to the wider institution.

At the same time, many universities have enrolled international students who can benefit from language, cultural, and social support. IEPs have faculty and staff who are highly qualified to provide programming in these areas (and who may currently have less work to do), yet because IEPs are typically viewed as profit centers rather than service units, they are not called on to offer such support. This is short-sighted, as increased support for degree-seeking international students will improve their retention and completion rates – which is good for the students,  the university’s bottom line, and the institution’s reputation.

IEP directors can sell this idea to university administrators. Here are some activities the IEP can offer to improve the international student experience on campus:

Workshops for faculty: Offer strategies to encourage international students to participate in class discussions, or give advice on assessing written work of students using English as a second language.

Resource webpage for English language support:  Like this one at Hunter College. Include online dictionaries, grammar resources, and writing advice for international students across campus.

Tutoring: Many universities have a writing center, but few have a place specifically to help with second language issues. The IEP can provide this.

English language workshops: Students who have gained a high score on the TOEFL or IELTS may still be lacking essential English skills. Offer workshops in pronunciation, pragmatics, or giving presentations.

Career preparation workshops: Many international students may seek on-campus employment, co-op or internship positions, or CPT/OPT opportunities. Help them write an effective application and interview effectively.

Pre-arrival language preparation: Develop a short online course to give incoming international students confidence with English. Prepare them for the various situations they will encounter and provide strategies to continue working on their English once they arrive.

These ideas will likely require building relationships with other offices on campus, and IEP directors may run into territory issues. Getting buy-in from a senior administrator who can support these efforts may be essential. This person may also be needed in making the case that the costs incurred in these activities will be more than recouped in international student performance, retention, and completion.

On-campus IEPs are home to enormous expertise on international student success. It’s time to put that expertise to work across the campus.

 

Job number one for education managers and leaders

Startup Stock Photos

I used to see one former boss only when he stopped by my cubicle with a question or an urgent demand. Another gave me the shrill and unhelpful advice, “You should be panicking!” in response to enrollment declines. This is not what you would call effective supervision.

It’s a problem in higher education that individuals are promoted into positions in which they have oversight over others, without having undergone training to prepare them for that role. For university IEP directors, the problem can be compounded by the fact that they report to managers who have little or no knowledge of the workings of an IEP. While there are many good people working in higher education administration, these circumstances can lead to strained relationships, loss of motivation, and diminished performance.

The solution is for managers to understand their primary role, their job number one.  It isn’t revenue generation, test score improvements, or student retention, which are indicators of great performance but not activities in themselves. No, the manager’s first job is to support his or her people. They are the ones on the front line of providing service to faculty, other staff, or students, and who are best placed to deliver quality through their work. The manager’s most frequent question to those employees should be, “What can I do to support you?” This is followed by careful listening and the  acquisition and direction of resources to provide that support, so that employees can do the high quality work expected of them.

On the basis of a well-supported faculty and staff delivering quality, managers can have confidence in focusing on goals of increased enrollment and revenue, improved student outcomes, and program development. There are no short cuts to these goals. Don’t forget job number one: support your people. 

Performance evaluations – a few polite reminders

http://www.charlesstone.com/images/uploads/performance_review.jpg

Performance evaluation season will be upon us before we know it, so here are a few gentle reminders and tips to make this a good experience for both parties.

For those doing the evaluating:

  1. This isn’t about you. You are evaluating the employee on behalf of the institution, and your evaluation should be a fair reflection of how the employee has served the institution. Put aside your personal feelings and base your evaluation on facts.
  2. Gather your information along the way. Don’t try to remember how the employee performed just a few days before the evaluation. Make it a habit, every month, to take notes on each employee’s performance, so you are not puzzling over the narrative at the last minute. Put it on your calendar.
  3. Make it an ongoing process. Have a deliberate check-in meeting with each employee every three months. Before the meeting, ask the employee to take notes on what has been going well, what the challenges are, what s/he has been working on, and what support s/he needs. Use these meetings to initiate course corrections, and let the notes from the meetings feed into the year-end performance evaluation. Put these meetings on your calendar.
  4. Get your employee’s input. Before the formal evaluation meeting, ask the employee to give you written notes on what has gone well this year and what the challenges were. BUT please don’t ask the employee to write his/her own performance evaluation, and don’t simply reproduce the employee’s notes as the performance evaluation. This is disrespectful to the employee and shows you don’t care. By the same token, don’t ask for the employee’s input and then ignore it in the evaluation.
  5. Rate the employee fairly and realistically. You’ll likely have to check some boxes indicating whether the employee ‘exceeds expectations,’ ‘meets expectations,’ or ‘needs improvement.’ Ensure that you take the entirety of the employee’s record into account when you check these boxes. Don’t mark ‘needs improvement’ to express a gripe about a single incident. And don’t mark ‘exceeds expectations’ across the board – doing so may make you feel generous, but it indicates that you have low expectations, or that the employee should be in a more challenging position.
  6. Discuss goals for the next year. Do this with the employee, and get his or her buy-in. Don’t simply impose goals  on the employee. And be sure to check in regularly with the employee in the following months on the progress toward those goals – don’t wait till the next annual performance evaluation.
  7. And of course, the cardinal rule: No surprises. If you follow the advice above, the performance evaluation will hold no surprises for the employee, which is how it should be. If your evaluation is negative and this is a surprise to the employee, you have failed as a manager, since you should have been working to correct any negative behaviors along the way, and you should have discussed them already with the employee. Your goal should be to have the employee leave your office feeling valued and appreciated, even if there are areas for improvement.

For those being evaluated:

  1. Keep an ongoing record. Make it a habit each month to recall your achievements and write them down. Otherwise, you will forget half of what you achieved and will undersell your accomplishments at year’s end. Put a reminder on your calendar.
  2. Ask to meet with your supervisor once every three months specifically to check in on how things are going with your work. Be prepared to ask what your supervisor is happy or dissatisfied with, and what s/he would like to see from you in the coming months. Report on progress toward your annual goals. Don’t expect your supervisor to initiate this. Put it on your calendar.
  3. If you get a negative surprise in your evaluation, be sure to raise this with your supervisor. It was his or her job to alert you to any performance issues along the way.
  4. Agree on goals for the coming year. You or your supervisor (or both) may develop goals for the year, but you must be sure they are reasonably achievable. Don’t set yourself up for failure with unrealistic goals.
  5. If you’re not satisfied with your evaluation, bring it up with HR. They will be able to advise you on how to effectively raise the problem with your supervisor.

Performance evaluations should be a helpful process for both the evaluators and the evaluated. Too often, they are a source of worry, stress, and disappointment. The advice above (based on years of hard-won experience on both sides of the process) should contribute to a peformance evaluation that is helpful to the employee, the supervisor, and the institution.

Do you have any other ideas for effective performance evaluations?

Keep Calm and Dance

Jerome Murphy of Harvard’s Graduate School of Education has made a career of educational leadership, and has first-hand experience of the stress, burnout, and even despair that can come with a lifetime of trying to satisfy the needs and demands of faculty, staff, students, and a host of other stakeholders. “Honking and hissing like geese,” goes Murphy’s goose theory of leadership, “faculty and staff members will cruise into the boss’s office, ruffle their feathers, poop on the rug, and leave” (p. 44), expecting a solution to whatever problem they brought in. The unskillful response of many leaders under these conditions is to obsessively ruminate, resist the discomfort and try to escape it, and rebuke themselves for not measuring up. The more they try to escape their discomfort, the more entangled they become in it. Can anyone relate yet?

Murphy’s answer in Dancing in the Rain: Leading with Compassion, Vitality, and Mindfulness in Education, is learn to live with the emotional discomfort and get it to work for you. When it rains, don’t run for cover; learn to dance in it. His formula for doing this, developed over a career, is summed up by the acronym MYDANCE:

Mind your values: Take action inspired by what matters most to you
Yield to now: Slow down and focus on the present moment
Disentangle from upsets: Mentally step back, observing and making room for upsets
Allow unease: Open up to upsets even if you dislike them
Nourish yourself: Engage in activities that replenish your energy and restore your perspective
Cherish self-compassion: Give yourself the kindness you need and deserve
Express feelings wisely: Carefully reveal your human side so that you can build trusting relationships (p. 41)

Murphy takes the reader through these Buddhist-inspired precepts chapter by chapter, and includes many easy-to-do exercises. For example in the Mind Your Values chapter, we are invited to call to mind a favorite leader, reflecting on the person’s values and how the person makes us feel. In Yield to Now, a 5-minute exercise suggests focusing in turn on the five senses, bringing attention back gently each time the mind wanders.

This definitely isn’t your typical educational leadership book. It’s more of a handbook on surviving and thriving amid the slings and arrows of academic administration. If your professional life seems to be a constant struggle, this may be the therapy you need.

(This review was also posted on Amazon)

Why language is best assessed by real people


“Classroom decoration 18” by Cal America is licensed under CC BY 2.0

What is the most effective way to assess English learners’ proficiency?

It has become accepted in the field to rely on psychometric tests such as the iBT (Internet-Based TOEFL) and the IELTS for college and university admissions. Yet these and most other language tests are an artifice, a device that is placed between the student’s actual proficiency and direct observation of that proficiency by a real human being. Students complete the limited set of tasks on the test, and based on the results, an algorithm makes an extrapolation as to their broader language abilities.

When you look at a TOEFL score report, it does not tell you that student’s English language ability; what it tells you is what a learner with that set of scores can typically do. And in the case of the TOEFL, this description is an evaluation that is based largely on multiple choice answers and involved not one single encounter with an actual human being. Based on this, university admissions officers are expected to make an assumption about the student’s ability to handle the demands of extensive academic reading and writing, classroom participation, social interaction, written and spoken communications with university faculty and staff, SEVIS regulations, and multiple other demands of the U.S. college environment. (Although the IELTS includes interaction with the examiner and another student, these interactions are highly structured and not very natural. TOEFL writing and speaking tasks are limited, artificial, and assessed by a grader who has only a text or sound or text file to work with.)

Contrast that with regular, direct observation of students’ language proficiency by a trained and experienced instructor, over a period of time. The instructor can set up a variety of language situations involving variation in interlocutors, contexts, vocabulary, levels of formality, and communication goals. In an ACCET or CEA accredited intensive English program, such tasks are linked to documented learning objectives. By directly observing students’ performance, instructors are able to obtain a rich picture of each student’s proficiency, and are able to comment specifically on each student’s strengths and weaknesses.

Consider this a call, then, for colleges and universities to enter into agreements with accredited intensive English programs to waive the need for a standardized test such as the TOEFL. Just as those colleges and universities don’t use a standardized test to measure the learning of their graduates, they should be open to accepting the good judgment of teachers in intensive English programs – judgment based on direct observation of individual learners rather than the proxy scores obtained by impersonal, artificial tests.

Goodhart’s Law and the Measurement of English Proficiency

Goodhart’s Law was first proposed by the British economist Charles Goodhart. In essence it states that, “When a measure becomes the goal, it ceases to be a good measure.” Measurements are often used as a proxy for performance. For example, it’s sometimes reported that in Soviet Russia, when the success of nail production was measured by quantity of nails, many tiny nails were produced. When it was measured using weight of nails, smaller numbers of large nails were produced. The measure became the target, and gaming the system created the illusion of success.

In U.S. university admissions, the TOEFL is the most common measure of the English proficiency of international applicants. It’s easy to understand why the complexity of language proficiency needs to be reduced to a small set of numbers when large quantities of applications have to be evaluated. Unfortunately, TOEFL preparation is very often a great example of Goodhart’s Law in action: many students focus on attaining the necessary score rather than comprehensively working on the cognitive-academic language skills and cultural skills they need to succeed in the U.S. university, and this can result in serious challenges for those students. Once matriculated, as those students seek to earn good grades – a proxy measure for learning – they may wind up trying to game the system by plagiarizing, using online essay services, cramming at the last minute, or begging the instructor for a better grade.

Although it would present practical difficulties, it would serve everyone better – schools and students – if the schools used a broader set of mechanisms to determine English proficiency. These might include evidence of English (not just test prep) study, Skype, phone, and in-person interviews, recorded presentations by applicants, synchronous online discussion groups, and reports from instructors in intensive English programs who have first-hand – not proxy – knowledge of the students’ English.

Hourly teaching rates in IEPs – reflection

In my last post, I questioned the hourly rates for ESL teachers in intensive English programs. I looked at the rates themselves, which can be very low, and the practice of counting class-hours as the basis for the hourly rate, which neglects the time that teachers put in on preparation, grading, and other duties.

There is no simple solution to this, since institutions and programs vary in their expectations of teachers for out-of-class work, and teachers themselves spend very different amounts of time preparing and following up on their lessons. Early-career ESL teachers may burn themselves out with over-preparation (as I almost did), or impose time constraints on themselves (knowing that their salary doesn’t justify an enormous amount of preparation) – which can lead to greater spontaneity in the classroom and can therefore  be a useful discipline to learn. Offering an hourly rate that teachers must work within may be the fairest and most workable way to manage all this variation.

Keep in mind, too, that proprietary English language schools are often the first stepping stone into a teaching career for newly-minted ESL teachers, who may have completed only a one-month certificate in addition to their bachelor’s degree. This is a low bar for entry into a teaching job, yet student satisfaction surveys indicate that such teachers can perform well, and the school can be seen as a kind of apprenticeship and nurturer of teaching talent. One teacher I employed did great work before deciding to complete his master’s degree and going on to become a business English professor at a prestigious English language program in Tokyo.

In the end, schools employing hourly-paid teachers should do their best for their teachers, providing resources and programs to develop the skills of their teachers, who may well leave for greener pastures when the time comes.  Additionally, hourly-paid teachers should inform themselves about the ESL job market, understand what they are likely to be able to achieve career-wise, decide whether to earn further qualifications, and make good decisions for themselves.

 

Let’s talk about those hourly teaching rates

A school vice principal I once reported to told me of his younger days teaching 60-hour weeks at Berlitz school. It was exhausting, he said, but at least with the Berlitz method, all he had to do was walk into the classroom and follow the script. There was no lesson planning, no grading, no syllabus or evaluations to write.

The Berlitz method is gone, and good riddance, but how should the hours of part-time ESL instructors be calculated in an age when teaching involves so much more than walking in and following the script? Many ESL teachers are employed on an hours-per-week basis, which invariably refers to the number of hours spent in class teaching students. Many programs in their turn – and this includes university IEPs, non-profit organizations, and English language schools – advertise their rate on a per-hour basis, which also refers to hours spent in class. At the upper end, this could be $60 – $80 per hour in some university programs. At the lower end, it could be as low as $15 or $20 per hour, especially in some non-profits and private language schools. As all ESL teachers know, these are not the true hourly rates, because teaching requires preparation and grading, in addition to required meetings, office hours, evaluation writing, and other activities. According to the federal government, actual hours worked should be calculated as time in class x 2.25. That is, for each hour spent in class, it is reasonable to assume that instructors spend another 1.25 hours working outside the class. Additional required activities should be compensated additional to this.

Many ESL teachers will tell you that 2.25 hours is too conservative, and that they spend longer on lesson planning and grading. Others may find it over-generous. But for the sake of argument, let’s take this number as a reasonable expectation. It doesn’t take advanced math skills to work out that a “15-hours-per-week” (not untypical for part-time ESL teaching) assignment is almost a full work week. Add those additional duties and you easily have a full-time job. Further, that $20 “hourly wage” is well below minimum wage in most states. Even that higher paying $80 per hour teaching gig is actually $35.50.

Everyone in the field knows about this. Part-time teachers tend not to complain because they enjoy the work and it’s the best they can do. University administrators and business owners know that advertising actual pay rates might be embarrassing and paying more would reduce margins. Accreditation standards for intensive English programs don’t mandate salaries or impose specific workload requirements. Associations such as TESOL and EnglishUSA either have little will or power to address salaries and teaching loads for part-timers.

It is way past time for a conversation about this, to include teachers, administrators, business owners and managers, accreditors, and associations. A start could be made at the national TESOL convention or the NAFSA conference. Can the field overcome its inertia and finally address part-time teaching salaries?

 

What’s the real story on international student applications?

“Survey Finds College Applications from International Students Down,” cries the U.S. News and World Report headline from March 13. “Amid Trump Effect Fear, 40% of Colleges See Dip in Foreign Applicants,” reports the New York Times on March 16. “Will International Students Stay Away?” asks Inside Higher Ed on March 13, reporting a “nearly 40% decline” in international applications.

What’s the story here? The actual percentage reporting declines was 38% or 39% (depending on whether you read the original AACRO preliminary report or one of the news stories), not 40%. This is three or four percentage points higher than institutions reporting increases in international applications, not a whole lot. And the rest – 27% – reported no change. Which means that 62% of institutions are reporting the same or higher applications.

While most stories touch briefly on some of the institutions experiencing increases, the slant to all the stories is negative. Far from simply reporting anxiety about future enrollments, they appear to be creating it. Is there a real basis for anxiety here? Consider:

  1. We haven’t yet learned (though a full report is expected) which institutions took part, how they were selected (or if they self-selected) or if they are representative of U.S. institutions as a whole. It is possible that institutions experiencing declines disproportionately responded to the survey.
  2. Only 250 institutions responded to the survey, a small sample of the thousands of institutions accepting international students. Without more information, it is too early to conclude that “nearly 40% of U.S. colleges are seeing declines.”
  3. We are not given any information about how these results compare to previous years, or whether the survey was conducted in previous years. Is this year better or worse than before? The only clues we get from the stories are anecdotal and speculative.
  4. No information has been given about the margin of error – which could even out the institutions reporting increases and decreases.
  5. No actual numbers are given, meaning we don’t have any real idea of the extent of the increases and decreases reported.

None of this is meant to diminish the anxiety that some institutions are clearly experiencing over enrollment declines. It is also not a criticism of the original study. But stories like this, especially those with a negative and alarmist slant, can quickly take on the status of orthodoxy and shape the conversation over the entire higher education landscape. They need to be read and interpreted with caution.

Sources for this post:

http://www.aacrao.org/docs/default-source/TrendTopic/Immigration/intl-survey-results-released.pdf?sfvrsn=0 

https://www.insidehighered.com/news/2017/03/13/nearly-4-10-universities-report-drops-international-student-applications 

https://www.usnews.com/news/education-news/articles/2017-03-13/college-applications-from-international-students-down

 

University IEPs: get the business back

Right after September 11, 2001, when many IEP student left the country and new applications tailed off to near-zero, I received some advice from a good friend: “Be here when the business comes back.” What he meant was to increase the school’s visibility, especially online, because the downturn wasn’t going to last forever and prospective students would once again be looking for schools to study English in the U.S. So I worked like a fiend to develop the school’s website, turning it into not only an information source in multiple languages, but also a highly efficient sales tool. I learned about SEO and made sure the site was appearing at the top of the search results. In time, the business did come back and sales turned around and took off.

Many IEPs, especially those in universities, have not been very well oriented to promoting themselves. For a few years, it was more or less enough to hang out a sign and wait for the students to arrive. Not any more. In these times of reduced enrollments, university IEPs need to take active steps to seek new students. They should adopt a three-point strategy:

  1. Define and communicate the unique value of the IEP. If your IEP looks just like any other, then you have a commodity and a race to the bottom on price. Sit down with your faculty, staff, university colleagues, students, and external contacts, and really figure out what makes your program unique. In most cases, it won’t be a single feature but a combination of features that are unique to your program. Think about the benefits of your location, your faculty’s expertise, opportunities for community interaction, and university resources. Also, keep in mind that you are unlikely to be attractive to all students everywhere. What is your niche? What do you want to be known for?  Once you have defined it, communicate it – on your website, in your social media, in your brochure, and whenever you and your staff talk to others about your program. If you advertise, be sure to include your message in your advertising.
  2. Develop new programs and adapt existing ones. Accept that the market is changing, and that what was attractive ten years ago may be less so today. Be aware – through conference attendance, reading (ICEF Monitor, The PIE Weekly, etc.), and your conversations with your external contacts – of what new programs are likely to be attractive. Match these ideas with your faculty’s expertise, and get faculty on board with program development. Think about populations you haven’t served, such as short-term summer high school students, incoming degree students, or professionals, and develop new content and means of delivery to appeal to these markets. Once you have developed curricula, put these new programs on your website and in your materials to signal your program’s capacity to deliver them, and talk them up in your conversations.
  3. Actively recruit students. As the cliché says, waiting for the phone to ring is not a good sales strategy. Expensive student fairs are generally not very effective for individual IEPs. Go with recruitment methods that are likely to lead to repeat business over time. Identify target countries based on resources such as Open Doors, Dr. Education, and NAFSA. See if you can work with the Undergraduate Admissions Office or the graduate colleges on campus to piggy-back on their recruitment and outreach efforts.   If you are permitted to work with agents, increase your network through agent fairs or in-country visits, or work more closely with your existing partners. Work with EducationUSA offices and try the State Department’s Gold Key service to connect with institutions and corporations in your target countries. Connect with foreign institutions at NAFSA and develop those relationships.

These are trying times for most university IEPs. If they are to survive and thrive in a hyper-competitive environment and during uncertain political times, they need to be pro-active. It’s not enough to wait for the business to come back. The strategy outlined above will get an IEP moving in a positive direction and ready to take on future shifts in the markets.

Recognizing the university’s obligations to its international students

Many U.S. universities have increased their international student enrollments over the past few years, through building out their international student recruitment capacity, working with overseas study abroad agents, or partnering with a pathway provider. Financial motivations are at the heart of much of this recruitment activity, but many institutions have been keen to implement and communicate a campus internationalization plan that adds value to the students’ experience by offering a diverse student community. But such a plan requires more than merely a recruitment effort: institutions need to develop the capacity to support their growing international population of students, who may need help with cultural adjustment, language skills, and orientation to the demands of the U.S. higher education system. In many cases, institutions have been slow to recognize the need for and create appropriate support systems for their international students, and this has led to concerns or complaints among faculty that students are not prepared, heightened stress for international students, and academic disciplinary measures taken against students who may not be familiar with citation practices. Clearly, institutions that have recruited international students should take their responsibilities toward these students seriously by offering plentiful support through dedicated staff, offices, and programs.

But this is only part of the solution. Much talk about international students on campus focuses on getting them to integrate, getting them to learn the American way, improving their communication skills. In this discourse, which belongs to the longstanding assimilationist tradition in the U.S.,  international students are viewed as a problem to be fixed. If institutions really want to develop their global credentials though, they need to look in the mirror: how prepared are faculty and staff to work with students from other cultures and language backgrounds? To what extent does the institution prepare its U.S. students to break out of their familiar social groups to befriend and welcome those from other countries? At a time when many institutions are signaling that they welcome international students, how many are actually taking measures to build that welcoming environment?

We know that most international students leave the U.S. not having formed a single significant friendship with an American or having once stepped into an American home. To serve their international students well, university administrations and faculties should refrain from merely problematizing those students, and accept their share of the burden in creating a welcoming, globally oriented institution.

IEPs and the Rise of Pathway Programs

This week my colleague from the University of Kansas, Deborah Osborne and I led a discussion on the future of

Deborah Osborne and Alan at the TESOL Convention, Seattle

university intensive English programs at the TESOL Convention in Seattle. Many university IEPs reported enrollment declines of between 10% and 70% last fall, and some are really struggling at this point. Meanwhile, pathway programs resulting from partnerships between universities (such as American University, Oregon State University, and George Mason University) and corporate partners (such as INTO, Navitas, and Shorelight) continue to proliferate, suggesting that English language and academic preparation for international students is undergoing a major shift.

For prospective students, the major attraction of a pathway program is the word ‘guaranteed.’ If they complete the program successfully, they will matriculate into the partner university. Many university IEPs can offer college advising and assistance with applications, but are not able to offer that guarantee. For universities, pathway programs can offer a fast track to increasing international student enrollments with little upfront investment or the need to build their own international student recruiting capacity.

There are two main choices for university IEPs. First, they can try to compete on the pathway providers’ territory by setting up their own pathway programs. The challenge here is that they generally don’t have the recruitment network to make that happen quickly, and there are very few instances of a university recruiting significant numbers of students via a home-grown pathway. Second, they can adapt to provide programming that pathway programs can’t. Examples include shorter programs, custom programs, specialized programs, and programs to support admitted or matriculated international students on their campus. This will require flexibility and an innovative and entrepreneurial spirit.

These are challenging times for IEPs, and it remains to be seen whether they can adapt to the challenge of the pathway model.

Aligning assessment and IEP culture

Since the passage of the Accreditation Act of 2010, intensive English programs (IEPs) have been under pressure to justify their quality claims by recording and reporting on student achievement. This has meant devising program-wide systems for assessing and evaluating students, and has been a challenge for many IEPs.

The type of system a program develops is influenced by its culture. A more managerial (top-down, administratively driven) culture typical of proprietary English schools tends to favor standardization of assessment that includes program-wide level-end tests. Many university IEPs have more of a collegial (faculty-driven with a degree of shared governance) culture in which individual faculty decision-making and autonomy are valued. In the latter type, it can grate against the culture when there is an attempt to introduce or impose standard testing. It may be more agreeable to retain faculty autonomy in assessment but introduce checks to ensure that assessments are aligned with course objectives and outcomes.

Both approaches (and blends of the two) are used by CEA-accredited programs and are able to meet the CEA standards. There is no need to create standard assessments across a program if they do not fit the culture. On the other hand, the imperative to assess students in a more consistent way can be a catalyst for culture change. This will need leadership, persuasion, and buy-in from faculty.

I’ve designed and overseen assessment and evaluation systems in proprietary and university programs, and can support programs in determining and developing the right approach. Get in touch if I can help!

Have a great weekend!

(Learn more about academic cultures in Engaging the Six Cultures of the Academy by William Bergquist and Kenneth Pawlak. I highly recommend it.)

Where most IEPs trip up in CEA accreditation

The CEA Planning, Development, and Review standards have proven tricky for many intensive English programs to get right at the first attempt. The seriousness of failing to measure up has been mitigated in recent years by CEA’s decision to collapse what were four planning and review standards into two – meaning that a program will have fewer standards of concern if it is having difficulty in this area. Nonetheless, the challenge for many programs is that they have never had a plan for regular review of their academic and administrative areas in place. Even knowing what such a plan would look like can be an obstacle to developing one effectively.

I’ve created development and review plans for a rolling-intake proprietary IEP and a semester-based university IEP. They were very different plans, each one tailored to the needs of its program, but both worked well and met CEA standards. If this is an area your IEP is having difficulty with, why not contact me to find out how I can help?

Have a great week!