The illusion of success
What is sacrificed in education when the emphasis is on performing and testing?
What is sacrificed in education when the emphasis is on performing and testing?
If we consider creativity as a form of productive disobedience, we are presented with questions about the world in which we might grow it.
Traditional education has shied away from the instability necessary to develop deep creative thinking. Accordingly, where we encounter fertile environments for growing creativity in schools, they are generally the result of the persistence, ingenuity and strength of individuals (or sometimes small clusters of teachers). However, these people are forced to operate in a largely antithetical environment, heavily shaped by values based upon explicitness, data-driven assessment and accountability. Given this situation, perhaps it is useful to take a look at what we have come to call assessment. Let’s start with something historical. At Pukeatua Primary, where I went to school, the incinerator was over by the fence. It was an old iron contraption with a chimney, and a little door that opened for feeding in the rubbish. It was replicated in a hundred schools in a hundred districts all around this country. Back in 1962, at Pukeatua Primary there were also lions, cheetahs and rhinoceroses. They lived in Room 3 and sat at desks in lace-up shoes. The lions and cheetahs, who made up most of the class, didn’t have much to do with the incinerator, but Nigel Terpstra and I, who were rhinoceroses, did. As the only members of the bottom group, we were delegated the dubious responsibility of being rubbish monitors. We learned how to cook our apples on the hot plate, how to smoke the teachers’ cigarette butts and, in times of crisis, when Nigel Terpstra got an asthma attack from inhaling too much cigarette smoke, we learned that holding him under the macrocarpas to breathe in the excess oxygen didn’t work.
We were a result of the 60s, a decade in New Zealand schools that saw a clear push towards reassessing the highly streamed nature of education. J.C. Daniel’s essays on the effects of streaming in the primary school were widely cited in the argument for continued social promotion and mixed-ability grouping. Unfortunately, at Pukeatua Primary and in many other schools up and down the country, what happened was that groups one, two and three were simply re-
placed by lions, cheetahs and rhinoceroses (or their equivalent), and eventually most of the rhinoceroses woke up to the fact that a fancy name still equated with being at the bottom of the pile.
Being at the bottom of the pile isn’t much fun. It’s the place where many of us have found ourselves at some time in our schooling, and it’s the place where we are most unmotivated to learn. Nearly all of us have come through school systems that use comparative methods of assessment, and most of us have learned that they do one thing very well: they teach us that we are not as good as other people. They teach us that in the race for learning, there are winners and losers and, as in most races, the losers outweigh the winners. To illustrate this, one has only to stop for a moment and think: What did I learn at school that I was no good at? Was it mathematics, or English, or chemistry? And if it wasn’t, did a belief in my incompetence come from one of those competitive, peripheral obligations, the cross-country race or the school speech contest? These were the annual institutions that sent generation after generation staggering from the stage or the playing field, lowly ranked and firmly convinced that they were innately incompetent. Every year the few were lauded and the majority defeated.
But, you know, if we could go back to Room One, on any weekday morning, we would probably see ourselves in front of the class, giving morning talks or running frantically around the playground. It would never have entered our heads that by the other end of this education conveyor belt we would have come to believe that we were failures. Back then, we were all orators and endurance runners, all cheetahs in the wind, and nowhere was there a rhinoceros on the horizon.
So why do we create rhinoceroses? Well, there is a lot of pressure in teaching to mark people and divide them into categories. It comes from schools with their need to meet Ministry of Education requirements, from parents who believe that learning is competitive, and from kids who have become convinced that a B+ or a Merit means they are better mathematicians or writers than their peers. Every year, we fill up registers with class lists and test results. We set exams and design teaching plans around generating these marks, and in the end we hand them over to students and ourselves as an indication of their competence. By the same token (despite the rhetoric of politicians), these numbers are often used by parents as a method of assessing a teacher’s ability and a school’s potential.
The institution of marking is so ingrained in education that most New Zealanders believe that these grades are indications of their true ability. Despite the decades of educational studies and working alternatives, we still continue to place an inordinate amount of emphasis on comparative marking (either comparing students to other students or to pre-established criteria). We change by changing the language. Scaled percentages are replaced by raw marks, that are replaced by grades, that are replaced by levels of merit … but the fundamental premise remains the same: we preoccupy ourselves with measuring the performance of learning. We assume that what is demonstrated is what is known. As a consequence we elevate what can be made explicit and what can be narrated, and somewhere in there we miss the point that learning is not a performance. It is a process.
Reporting using marks actually tells us very little. We learn nothing about the nature of an individual’s learning, nothing about “how” they think and little about their approaches to prob- lem solving. Marks become a substitute for insightful description. They replace educational analysis with comparability, providing us with data that we can rank, equate and contrast.
Where the inherent value of learning becomes negligible, marks become a substitute motivation. Despite being an abstraction, they are experienced as concrete, because in the end they are what is rewarded. Students work for them. They want to know what content will be in a test. If what they study is directly related to the marking, they will understand its value and concentrate on it. They make assessments about what will be “worth” learning based not on its intrinsic value but on its capacity to earn them marks. Accordingly, they end up demonstrating selected pieces of learning and being rewarded for a dumbing-down of their educational potential. In this system, the value of what is learned doesn’t really matter anymore because reward is not related to what is intrinsic, only to what is performed.
Unfortunately, “getting good marks” has become a major preoccupation of politicians, of low-level educational reformers and, tragically, of students themselves. But if we think deeply we are left with a nagging question: with this emphasis on performing and testing, what is sacrificed? What at the heart of education pays the price for this tradition of comparison?
Well, it is learning. It is learning with its multi-layered, idiosyncratic relationships, with its messy edges and complex trajectories. It is learning that is not standardised, never experienced the same way by two individuals, never timed the same way, and is always most insightfully understood in the context of the learner. It is learning as a process, not a product.
The overuse and misuse of standardised comparative testing has many critics. In New Zealand, especially since the 1990s, much has been done to moderate its use (in contrast to countries such as the US, England and Wales). However, it still wields significant influence. Beyond its capacity to be mishandled in decisions of accountability and effectiveness, including ill-considered proposals by politicians that teacher quality in New Zealand should be measured by marks produced in testing, it can also lead to a narrowing of the curriculum and the breadth of thinking we provide for learners. This is because teachers, in an effort to be seen as effective, find themselves caught in ethical
and professional dilemmas. They encounter explicit or tacit instructions and incentives to “teach to the test” because that is what will be rewarded. Despite assertions that standardised testing can be used without it distorting curriculum and teaching, the truth is that often what is not tested is not taught. Moreover, training in the rituals and devices of testing becomes a silent but valued part of the curriculum.
Internationally, such narrowness is not a given. If one considers the 25 years of educational reform initiatives in Finland, we see a country that has produced an education system that achieves among the highest rankings for equality and excellence in the world. In this system there is no tracking of students during their common basic education. Children are not measured at all for the first six years, and the only mandatory standardised test is taken when they are 16. Classes are not streamed, and the difference between the weakest and strongest students is one of the smallest in the world.
Significantly, there is no merit pay for teachers’ performance. Finnish educators are not monitored or rated according to test results, and although they follow a basic national curriculum, they have a great deal of autonomy over how they develop methods for learning and evaluation.
If freedom from high levels of testing and reporting can have such a significant impact on an education system like Finland’s, we might be left asking why we continue to hold so rigidly to a belief in the merits of standardised examination in this country. We know that it narrows what is valued and taught, and we know that it has a profound impact on higher-order learning. [US education theorist] William Ayers notes that such tests “can’t measure initiative, creativity, imagination, conceptual thinking, curiosity, effort, irony, judgment, commitment, nuance, goodwill, ethical reflection, or a host of other valuable dispositions and attributes. What they can measure and count are isolated skills, specific facts and function, content knowledge, the least interesting and least significant aspects of learning.”
The difficulty is that standardisation assumes a high level of convergence. The divergent, deep-questioning and intellectually disobedient thinker either performs within the realm of the expected or pays a substantial price. The student is forced to trade inside the limitations of what has been pre-imagined and predetermined. Many gifted children pressured by the overuse of such testing get bored and angry. Some become cynical, some disheartened. More perniciously, some begin to conceive of themselves as flawed. A system predicated on predefined criteria, levels and exemplars lacks the flexibility for genius. The very innovation and excellence it purports to value, it also constrains and punishes. This limiting of disobedient thinking is toxic to any system that seeks to grow highly intelligent, creative beings.
However, standardised tests and the rankings they provide can make people feel very secure. When we do well (either as an individual, school or nation), we feel valued — and New Zealand does quite well internationally. We like marks when they suggest that we are better than other people. So long as we don’t ask awkward questions about whether these marks adequately describe learning, the rankings they generate are easy to understand, easy to report, and therefore easy to use politically.
An obsession with marks produces another, arguably deeper, problem. Because we place such high emphasis on rewarding outcomes over the process of thinking, most of us leave the school system ill-prepared to be life-long learners (despite aspirational rhetoric to the contrary). Instead, we are trained in the formulas and rituals of testing. We learn what is being asked for and what we need to do to display the required ability. Through an insistent emphasis on performing for marks we become dependent. Deeply dependent. We wait anxiously to hear how good we are because our sense of educational achievement has become reliant on outside verification.
This is a problem, because despite the fact that the average New Zealander spends 12.6 years in school, at the end of that time they actually know very little about themselves as learners. Think about this for a moment. It is a frightening thing. Ask yourself how much you actually know about the way you learn:
Do you know what causes you to retain certain information?
Do you know how you increase chances for discovery in what you do?
Do you know your most effective strategies for learning from failure?
Do you know how you push yourself beyond competence?
These questions probably feel like very unfamiliar territory. After all of your years inside the intimate experience of your own learning, in a system that assures you of your “learner centredness”, these questions are tough for you to answer. You probably don’t know the answers. What you probably do know, however, is how well you did when marking came around.
So herein lies a problem. If we don’t know how we learn, we remain reliant on somebody else to direct us. We depend on verification. We need direction and approval. Our intellect and its growth — arguably the greatest gifts we possess — are sold out to somebody else’s value system, and we are systematically massaged into compliance.
In considering the implications of an education system that has become overly focused on assessing performance we are faced with one other problem. This is the awkward issue of corruption. While most educators prefer to pretend that massaging marking doesn’t happen, in truth it does, and behaviour associated with it is widespread. Some of us have experienced it as students, and some of us encounter it in the schools in which we work. The reason such corruption occurs is perhaps best explained by Campbell’s Law. Donald Campbell was an American social psychologist, methodologist and philosopher who in 1975 developed a law. Put very simply, [it says] if you set up fixed measurements for a social agent such as education, people within the system will begin to corrupt it so that it works in their favour.
Public schooling in New Zealand was developed as a hierarchical structure. Salaries and privilege were locked into the design of this system, and value was measured by what could be made explicit. Within this structure people learned to compete for
While most educators prefer to pretend that massaging marking doesn’t happen, in truth it does.
attention and promotion. As schools morphed across subsequent decades, teachers were formally and informally assessed on the quality of student grades. Good teachers, it was assumed, taught students who performed well in tests. Because professionalism was associated with the attainment of good grades, when things became difficult, teachers and schools learned to “work” the system.
At its most transparent, this involved the deliberate training of students in examination strategy and the prioritising of material that teachers knew would be tested. These became explicit undertakings that troubled few people, even though we know that such initiatives skew the reliability of results against students who are not similarly engineered. These behaviours continue to be instituted as a way of distorting results inside standardised systems of measurement and reward. They become deeply embedded and eventually assimilated into the practice of education. As a consequence, teachers often feel obliged to adopt such practices because they care about the opportunities high performance will provide for students in their care. They also care about their own professional profiles. However, there are arguably more damaging instances of exclusion and marginalisation that teachers and schools use more covertly to advantage their performances.
Iwould like to give you an example. Before I do, though, I would like to make it clear that I worked with Mr Shiffol and he was an honourable man. In the small town where I taught in the 1980s, he was valued as a caring and responsible person. He was the head of a department in which some of my colleagues worked. Every year, his students gained the top grades in his subject in the school and ranked well above the national average. On a graph of performance his results were constantly tapping against the top line.
Mr Shiffol seemed to have found a magic formula — and perhaps he had. Because of his position he set the timetable and annually populated his classes with “high-performing” students selected from the preceding year (with one or two exceptions prominently distributed across the classes of other teachers in his department). Where he ended up with an unexpected or token underachiever, he talked them into the benevolent option of a non-examined, alternative pathway. This was always presented as a caring and pedagogically compassionate gesture, because he wanted them to get the most out of their education and he was worried about the stress that examination might place on them. In the rare case where a student or parents resisted his offer, he explained that their child’s performance was too low to qualify for sitting the endof-year examination. If that didn’t work, he eventually moved them to another class so they were not made to feel polarised by the high performance of their peers operating in his class. In one worst-case scenario, when even these strategies failed, he simply wore a defiant student down with expressions of exasperation and subtle ridicule. The girl packed up and left by mid-year.
Mr Shiffol hated to see people fail.
The interesting thing was that, in the community, parents thought Mr Shiffol was an excellent teacher, the school hierarchy admired him, and so did the students (even those he massaged away from his classroom). The marks were evidence of a great educator, and students scrambled to be selected for his class.
In an ethically conflicted way many of his colleagues also admired him, because his techniques for survival and distortion were so convincingly masked. He was very generous with advice. He gave workshops to other schools on “professional development” and he was invited to be a member of numerous curriculum working committees. Mr Shiffol modelled strategic self-promotion, and edited stories of classroom experience and current educational vision. But he wasn’t a bad man. He had simply learned to survive in a system that wasn’t prepared to look deeply into what constituted quality learning. In a position of power he was able to make important decisions that preserved his potential for high performance.
This may be why no one understood when one year, on the morning after Guy Fawkes celebrations, he awoke to find on his lawn a half-burned effigy. It had been retrieved from a fire the night before. The polyester jacket was charred and twisted into a melted memory of itself. In the painted mouth, in recognition of his fondness for cigars, there was an unlighted firecracker. But the most distressing detail was that somebody had scrawled in angry crayon on a sign around his neck, “Shiffol-Shithole”.
Fifteen years later, I returned to the high school for a reunion. Many students I had
known had done wonderful things with their lives, but what struck me was the significant shift in their thinking. Almost without exception they referred to Mr Shiffol in disparaging terms. The scrawled name on the card around the effigy’s neck had become his moniker. The respect of these young men and women now focused on other teachers: those who had spent time with them, who had supported them through failure and who cared about them as people. Across the inevitable shake-down that time provides, Mr Shiffol’s pedestal had collapsed.
The underperformers Mr Shiffol weeded out each year come in many guises, but the education system ensures they are identifiable with updated marks that designate their value. These students are the product of an under-critiqued regime of marking and they constitute a kind of educational “fall-out”. The phenomenon has been much documented and is often associated with stories of damage to students who early in the system are relegated to the bottom of the pile. But the destruction is more pervasive and subtle than this.
Of course, testing has a place (especially when it is used as a diagnostic tool), but great teaching is concerned with effectiveness, and its primary emphasis is on informal watching, questioning and assessing inside the practice of learning. Teachers know that learning is nuanced, cumulative and highly integrated. They know that its complexity cannot be reliably assessed against standards, indicators of progression, or the provision of nationally nominated exemplars. Education is about more than reaching targets. Accordingly we seek something richer, and in achieving this we are forced to disobey.
If a whole system can’t be reformed, infect what you can with positive initiatives. We are empowered to do this because we are educated, thinking professionals, capable of designing something better. Even at a micro-level our intervention still counts.
The viral infections I discuss here are drawn from my time in different parts of the education system, and all of them happened because I sensed there was something better than what existed. They are not tales of wonder and revolution. I don’t walk on water. However, each of them significantly changed something. They are based on three ideas about assessment. These are: the importance of self-evaluation, reducing the impact of marking, and the need for quality reporting.
Let’s have a look at them.
Some teachers don’t really believe in marking. It has never made sense to them that you can replace the complexity of learning with an abstract grade. As a consequence, over time they design systems where learners appraise their own work and the work of others. We might understand this as a form of self-control. This is because if students can gain increasing insight into their learning, they can break the dependence on external, authority-based assessment. In any education system in which I work, if I can’t convince people that there might be something more effective than examination, I strategically disobey. I ascertain the minimal amount of testing required by those in “authority”, reluctantly tick those boxes and then set about infecting everything else with more effective approaches. Many teachers do this, in lots of different ways. [Here] are a few strategies students have shown me that I have found useful:
Broadly, with the assistance of my classes, I design assessment formats that provide two layers of reflection. The first involves a personal critique by the learner. This is sometimes written but often it occurs in focused discussion. I ask students to outline perceived strengths and limitations in their work and suggest ways that any weaknesses might be addressed.
The second involves peer group critiques of work. These give an outsider viewpoint and also involve other learners in objective considerations of alternative solutions to an assignment their fellow students are also completing. Normally, small panels of peer reviewers are asked to critique three submissions that are not their own. They write a collective review. In return, each student receives a panel assessment of their work.
It sounds very laudable, but of course it can be challenging to handle. Often students have become deeply dependent on marks. They go through a kind of withdrawal when they are asked to generate assessments that really count. They often prefer somebody else to take responsibility. They want rewards. It also takes a lot of work to get the system running smoothly, because you need very clear bottom lines about respect and expectations. You also have to negotiate with colleagues who may see you as not supporting a system that works comfortably for them and is the dominant approach of
the institution inside which you are working.
When I taught at secondary school, I tried wherever possible to apply assessment innovations developed in disciplines like art, design and technology, which have historically used cumulative portfolios. I adapted the approach to a diverse range of subjects including English, social studies, biology and history. Where a school insisted on mid-year assessments, I asked that these be framed as formative and any mark “required” for reporting purposes was delivered as a grade range. This sometimes took some negotiating. Where a school insisted on examinations or standardised testing and I couldn’t negotiate anything else, I acquiesced but wrote a letter to the student and parents that offered a full context and description of what was really happening with the learning.
At the end of each year, I give gifts. This practice has a long tradition in education: historically many teachers have given and received small tokens of appreciation at the end of a year spent with their students. At the university where I work the gifts I give are small potted herbs accompanied by a handwritten letter. This letter tries to describe the kind of learner I have experienced during our time together. Each student’s learning is linked metaphorically to lore surrounding the plant, or to something specific about the way it grows. In giving such a gift I try to negate the authority of a mark that will eventually arrive from some external party. In some cases I know these plants won’t make it through the summer holidays; they will die on the windowsill of an unkempt flat, or they will be planted in conditions that would threaten the survival of a radioactive cockroach. But in other cases they grow and years later I hear back from people. I go to their weddings and attend the births of their babies and they remember these small things.
This is a form of “reporting” that surfaces as a consequence of learning together. It does not explain a mark. It accepts that we are not learning information and skills but using these things to develop as human beings. I try to make any feedback personal, and connected not just to the submitted work but also to the learner and the uniqueness of their journey. I take a lot of time to listen, write to and talk with students. They know I think about them — not just about what they produce but also about them as people. It has taken many years for me to learn how important it is to do such things.
The origin of this approach surfaced 30 years ago when I was teaching in a small King Country college where I was trying to work out a way of reporting on learning more effectively. This was because the school report template only had space for a mark, a class ranking and a small rectangle for entering a few words like “Could do better”. When I explained that I would like to write more fully on the students’ learning, I was asked not to continue my comments on to the back of the report sheet because it was felt that in so doing, I was pressuring my colleagues into doing the same. I was not being a good team player.
But, you know, it is very difficult to sum up the complexity of somebody’s learning in a 10mm x 30mm rectangle. I didn’t want to upset my colleagues, so I took to writing discrete letters. These were given out on the same day as the report, in class, quietly, one to each student. The letters talked about how I thought they were learning, how they appeared to solve problems, and the unique nature of their progress and aspirations. Sometimes the letters were poetic, sometimes tender, sometimes critical, but always appreciative.
Interestingly, when the ritual of the report evening arrived, it was these pieces of paper that parents brought with them. They were proud that somebody commented on their child’s acerbic wit or creative excuse making, or their ability to sense anxiety in others and help before anyone else noticed. Marks and grade point averages do not engender such things. What matters is the care one human being takes to talk about another.
I still write in such depth, be it for a class of 30 students or for the supervision of a single postgraduate candidate. Because my teaching is strategically cleared of fragmented assessment tasks and unnecessary marking, I have more time to listen to and talk with the students with whom I work. Such approaches are better than a B+ or “achieved”. They tell people that they matter.
I do not claim that these approaches to assessment are transferable. They are not a template for a working alternative. They are idiosyncratic, but also indicative of things I see all around me when I watch talented teachers make space for more humane and insightful approaches to assessment. They are disobedient interventions in a system that has lost its sense of priority.
I do not believe that testing has no place in schools, but its current, disproportionate prominence is damaging. It has filled too many spaces and priorities, so that teachers, in a scramble to meet testing requirements, have little time to develop deeply reflective learning environments. A friend of mine teaching in a small rural school once summed up her predicament this way: “We spend so much time weighing the sheep that we have no time to feed it.”
While the image may cause us to smile, the reality doesn’t. Yet despite the cramping nature of too much testing and the prosaic teaching models it reinforces, certain teachers continue to systematically disobey dominant practice in order to transform learning by using assessment as an informal, diagnostic tool that is embedded in a process of observing, discussing, co-creating and listening. They can describe what is emerging, consider what they encounter against a range of social and personal contexts, and then edge their way intuitively forward. This kind of assessment is not a ruler. It is not absolute and is rarely recordable. But it is the foundation of evaluation, and certainly the most active agent in understanding student progress.
The problem is that such immersive forms of assessment are rarely rewarded outside of the intimate relationship established between the learner and the teacher. In our mistaken assumption that learning produces a valid, measurable product we end up rewarding only that which can be turned into performance. This is a senseless and toxic situation. It elevates measurement above knowing, and no effective education system can develop from such a position.
It is very difficult to sum up the complexity of somebody’s learning in a 10mm x 30mm rectangle.
An edited extract from Disobedient Teaching by Welby Ings (Otago University Press, $35). Ings is professor of graphic design at AUT University, Auckland.