Feed­back: The Bro­ken Loop in Higher Ed­u­ca­tion – and How to Fix It

Learn­ing sci­ence and teach­ing prac­tice agree on the power of feed­back to en­able and en­hance learn­ing. So why is feed­back so spotty and ill-timed in higher ed­u­ca­tion?

Rotman Management Magazine - - CONTENT - By Mi­h­nea Moldoveanu and Maja Dji­kic

Feed­back is a proven en­abler of learn­ing. Yet in higher ed­u­ca­tion to­day, it is spotty, ill-timed or ut­terly miss­ing. Here’s what to do about it.

WHEN WE WANT TOLEARN a new skill in life, we try out a new be­hav­iour. Some be­hav­iours ful­fill their in­tended pur­pose, while oth­ers do not. Feed­back — sig­nals from the en­vi­ron­ment that tell us whether or not the be­hav­iour we pro­duced had the in­tended ef­fect — is es­sen­tial to chang­ing, adapt­ing or mod­i­fy­ing hu­man be­hav­iour. This is what learn­ing is all about.

Whether or not a skill can be re­pro­duced by an al­go­rithm, learn­ing any skill re­quires feed­back and is es­sen­tial to the mea­sure­ment of learn­ing. In the skills en­vi­ron­ment of the Fourth In­dus­trial Rev­o­lu­tion — wherein in­for­ma­tion is free and com­plex, in­ter­per­sonal skills whose de­vel­op­ment re­quires tex­tured, pre­cise, timely per­son­al­ized feed­back have be­come the high­est value con­tri­bu­tions to hu­man cap­i­tal. The re­sult: Feed­back has be­come the crit­i­cal miss­ing link in higher ed­u­ca­tion.

Learn­ing Sci­ence and Prac­tice: A Pic­ture Emerges

The sci­ence of learn­ing and teach­ing of­fers abun­dant ev­i­dence of the crit­i­cal link be­tween feed­back and learn­ing. In re­cent years, it has come to fo­cus on iden­ti­fy­ing the right kinds of feed­back for dif­fer­ent peo­ple and learn­ing en­vi­ron­ments: Whether you are learn­ing a for­eign lan­guage or a com­puter lan­guage; learn­ing to su­press im­pulses; or learn­ing to com­mu­ni­cate co­her­ently, em­path­i­cally and re­spon­sively — each re­quires spe­cific types and se­quences of feed­back.

Time­li­ness, pre­ci­sion, in­tel­li­gi­bil­ity, ac­tion­abil­ity, rep­e­ti­tion — all rep­re­sent fea­tures of learn­ing-en­hanc­ing and en­abling feed­back across dif­fer­ent do­mains of knowl­edge, skill and ex­per­tise. The dis­ci­pline of ma­chine learn­ing has made rapid ad­vances in the last 10 years pre­cisely be­cause of its use

of fast mech­a­nisms that al­low al­go­rithms to ‘learn’ from their own per­for­mance via feed­back that tracks their suc­cesses and fail­ures in repli­cat­ing or pre­dict­ing the data sets they are trained to com­press and repli­cate (or ‘un­der­stand’).

De­spite the mo­men­tous ad­vances in un­der­stand­ing the role that feed­back plays in learn­ing, pro­fes­sional and higher ed­u­ca­tion are lag­ging dan­ger­ously be­hind what is now both pos­si­ble and de­sir­able. The ‘lec­ture-home­work-quiz-exam’ rou­tines that per­vade higher ed­u­ca­tion — whereby feed­back is pro­vided en masse — lag stu­dent per­for­mance by a long time and are not adap­tive or per­son­al­ized to the learner or to her task. As such, cur­rent teach­ing prac­tice — and the learn­ing en­vi­ron­ment it pro­duces — lives in self-suf­fi­cient iso­la­tion from the find­ings of learn­ing sci­ence, deep learn­ing sci­ence and en­gi­neer­ing and the neu­ro­science of learn­ing re­gard­ing the im­pact of feed­back on skill and com­pe­tence de­vel­op­ment.

To­day’s feed­back prac­tices re­sem­ble those in ef­fect 50 and even 100 years ago — an in­er­tia driven partly by the eco­nomics of higher learn­ing and partly by the cul­tural im­per­vi­ous­ness of ped­a­gog­i­cal prac­tice to learn­ing sci­ence and tech­nol­ogy. But the op­por­tu­nity costs of this ‘know­ing-do­ing gap’ are very high and ris­ing quickly: This gap rep­re­sents both a sig­nif­i­cant drag on the learn­ing curve of stu­dents and an im­por­tant op­por­tu­nity for dis­rupt­ing the $2 tril­lion (2016 dol­lars) higher ed­u­ca­tion in­dus­try. It is one that some or­ga­ni­za­tions, as we will see, have al­ready laid the ground­work for.

Miss­ing and Coun­ter­pro­duc­tive Feed­back Pat­terns

To un­der­stand how the cur­rent feed­back land­scape of higher ed­u­ca­tion fails learn­ers by fall­ing short of state-of-the-art learn­ing sci­ence, let’s re­turn for a mo­ment to the cen­tury-old lec­ture-home­work-quiz-exam rou­tine that is the cen­tral model for learn­ing to­day.

Lec­tures present con­cepts, mod­els, meth­ods, heuris­tics, along with their deriva­tions thereof and ap­pli­ca­tions. Home­work prob­lems, quizzes and ex­ams of­ten test for the nor­ma­tive or cor­rect ap­pli­ca­tion of a skill or method to an un­fa­mil­iar prob­lem. Feed­back on the ex­er­cise of the skill by the learner is, for the most part, given by teach­ing as­sis­tants on prob­lem sets, quizzes and ex­ams turned in by learn­ers — in batches, and days or weeks fol­low­ing the com­ple­tion of the work. This is the ex­act op­po­site of what learn­ing sci­ence tells us about feed­back that max­i­mizes skill de­vel­op­ment. Specif­i­cally, feed­back in our cur­rent model is.

• OO LATE: Graders typ­i­cally take days or weeks to de­liver feed­back to learn­ers, in sharp con­trast to the re­sults of stud­ies that in­di­cate the im­por­tance of im­me­di­ate feed­back in the de­vel­op­ment of skill.

• TOO RARE: Feed­back is in­fre­quent rel­a­tive to both the weight it should re­ceive vis à vis other learn­ing ac­tiv­i­ties (such as lis­ten­ing and tak­ing notes) — given its im­por­tance to learn­ing, and es­pe­cially to the learn­ing of com­plex skills. An ar­ti­fi­cial neu­ral net­work can ‘shat­ter’ — or, learn to clas­sify — a large data set con­tain­ing lots of non-lin­ear re­la­tion­ships only if it re­ceives pro­fuse feed­back about its per­for­mance as it ‘learns’. Why would a real neu­ral net­work be any dif­fer­ent?

• TOO IM­PER­SONAL OR GEN­ERAL: The feed­back the learner re­ceives is not adap­tive to her spe­cific pat­terns of thought or be­hav­iour. Be­cause of the ‘eco­nomics of feed­back’ in higher ed­u­ca­tion, there is lit­tle time or scope to adapt the feed­back to the learner’s spe­cific goals and stock of ex­ist­ing skills, which sig­nif­i­cantly de­creases the ac­tion­abil­ity of the feed­back for the learner.

• TOO IM­PRE­CISE: Learn­ers usu­ally re­ceive ‘0 or 1’- type feed­back (i.e. cor­rect/not cor­rect) re­lat­ing to the de­gree to which they an­swer a ques­tion or solve a prob­lem as a whole — but not on the spe­cific pit­falls of the think­ing or rea­son­ing un­der­ly­ing an in­cor­rect or par­tially-cor­rect an­swer. This makes the feed­back sig­nal dif­fi­cult to in­ter­pret as an ac­tion-guid­ing and be­hav­iour-cor­rect­ing in­put.

• TOO NOISY: Much of the feed­back learn­ers re­ceive is heav­ily de­pen­dent on the rapidly-chang­ing and idio­syn­cratic bi­ases, moods, dis­po­si­tions and phys­i­o­log­i­cal states of the graders. Dif­fer­ent graders can dis­agree sharply on the qual­ity of

Ma­chine learn­ing has made rapid ad­vances in the last 10 years due to al­go­rithms that ‘learn’ from their own per­for­mance, via feed­back.

a par­tic­u­lar piece of work, and a sin­gle grader’s feed­back can be more or less favourable, pre­cise and co­gent at dif­fer­ent times of the day, be­fore or af­ter meals, and be­fore or af­ter sleep.

• MIS-CON­STRUED AND MIS-CON­STRUCTED: Most of the feed­back in higher ed­u­ca­tion is given — and in­ter­preted as hav­ing been given — for pur­poses of eval­u­a­tion, fil­ter­ing and se­lec­tion, as op­posed to be­ing ori­ented to learn­ing and be­havioural change. It is eval­u­a­tive and se­lec­tion-ori­ented — some­thing that ed­u­ca­tional re­search has stead­fastly shown to un­der­mine the ef­fec­tive­ness of feed­back as an en­abler and fa­cil­i­ta­tor of learn­ing, which de­vel­op­men­tal feed­back en­cour­ages and fa­cil­i­tates.

The Right Feed­back at the Right Time

We cur­rently have the means at our dis­posal to fix this ‘bro­ken feed­back loop’: Con­verg­ing mod­els and ev­i­dence from cog­ni­tive sci­ence, deep learn­ing the­ory and prac­tice, and the neu­ro­science of learn­ing (to­gether mak­ing up ‘feed­back sci­ence’) doc­u­ment the qual­i­ties of feed­back that is max­i­mally con­ducive to learn­ing for most skill sets. We can — and should — turn this knowl­edge into a set of prin­ci­ples for the de­sign of feed­back pro­to­cols that fix the bro­ken loop.

Not all feed­back is equally use­ful or good; and some is ac­tu­ally coun­ter­pro­duc­tive, un­in­for­ma­tive and use­less. What kind of feed­back is most use­ful to learn­ing? Some of the an­swers are in­tu­itive, oth­ers less so. Learn­ing-en­abling feed­back is:

• TIMELY: It fol­lows promptly in the foot­steps of the learner’s be­hav­iour. Feed­back given in a week is far in­fe­rior to feed­back the next hour or the next day. In fact, neu­ro­sci­en­tists have found that for cog­ni­tive tasks—like learn­ing the gram­mar of a mod­er­ately com­plex lan­guage—in­stan­ta­neous feed­back trumps feed­back that is given even a few sec­onds later;

• SPE­CIFIC: Feed­back that en­ables learn­ing is not gen­eral or fuzzy. It does not evince the clue­less­ness of cur­rently com­mon grad­ing prac­tices, in which the grader strug­gles for some­thing mean­ing­ful to say to jus­tify a let­ter or num­ber grade ar­rived at on ac­count of causes that have noth­ing to do with the rea­sons given for the grade. It is spe­cific to the fol­low­ing:

• To be­hav­iour or out­put — to the de­tails of the learner’s writ­ten an­swer or ver­bal and non-ver­bal be­hav­iour, and to the com­po­nents of the out­put that can be use­fully mod­i­fied.

• To the con­text in which the writ­ten an­swer or ver­bal or non-ver­bal be­hav­iour is em­bed­ded. Good feed­back points out, for in­stance, ways in which the learner mis­con­strued the sit­u­a­tion or the ques­tion.

• To tim­ing — to the or­der or se­quence in which the learner’s an­swer or ver­bal or non-ver­bal be­hav­iour oc­curs. Good feed­back sin­gles out the spe­cific points in the learner’s pat­tern of rea­son­ing or be­hav­iour that make the great­est con­tri­bu­tion to the qual­ity of the work. If a learner can­not dif­fer­en­ti­ate con­tin­u­ous func­tions, for in­stance, and tak­ing de­riv­a­tives is an in­te­gral part of the chain of rea­son­ing that leads to the right an­swer on an equi­lib­rium cal­cu­la­tion prob­lem, then feed­back that pro­motes learn­ing should sin­gle out the learner’s skill gap in dif­fer­en­tial cal­cu­lus.

• To the learner her­self — to pat­terns of rea­son­ing, cal­cu­la­tion or be­hav­iour that are spe­cific to the learner’s own way of think­ing or be­ing. Good feed­back is not generic — it is highly tuned into the learner’s pat­terns of think­ing and be­hav­ing.

• To the con­se­quences of be­hav­iour or out­put and their in­ter­pre­ta­tions. Good feed­back on in­ter­per­sonal, so­cial or re­la­tional tasks points out the con­se­quences of the learner’s be­hav­iour on oth­ers’ feel­ings, be­hav­iour and likely thoughts, al­low­ing the learner to make tex­tured in­fer­ences about the causal chain that links her be­hav­iour to their so­cial con­se­quences.

The high­est-value tasks per­formed by hu­mans have be­come pre­dom­i­nantly so­cial, re­la­tional and in­ter­ac­tive.

• AC­TION­ABLE: Good feed­back pro­vides prompts for be­havioural or con­cep­tual changes that are in­tel­li­gi­ble, clear and ex­e­cutable by the learner. It does not merely pro­vide an ap­praisal of how suc­cess­ful an an­swer or be­hav­iour was, but also a set of sug­ges­tions or in­junc­tions for chang­ing thought or be­hav­iour pat­terns which are likely to lead to a bet­ter re­sult;.

• CRED­I­BLE: Good feed­back is per­sua­sive to the learner in virtue of be­ing:

• Le­git­i­mate. It is con­nected to the learn­ing ob­jec­tives of the course or mod­ule or learn­ing ex­pe­ri­ence and to the learn­ing ob­jec­tives of the learner;

• Jus­ti­fied. It is but­tressed by valid rea­sons, drawn from dis­ci­plinary re­search and/or re­search on op­ti­mal learn­ing;

• Ob­jec­tive or im­par­tial. Good feed­back can be val­i­dated by oth­ers of com­pa­ra­ble ex­per­tise to the feed­back giver, and is not thus prone to per­sonal bi­ases that ren­der it par­tial or un­fairly slanted.

• CRED­I­BLE: Its in­tent is to help the learner im­prove her per­for­mance on a task, or en­hance her skill or com­pe­tence in a do­main — rather than merely to pro­vide an or­di­nal or car­di­nal rank­ing of learn­ers’ ef­fort and tal­ent lev­els for the pur­pose of pro­vid­ing dis­crim­i­nant value to re­cruiters or other pro­grams of train­ing.

• ITERATIVE: Good feed­back is not a one-shot deal. It pro­ceeds in iterative fash­ion. Just as neu­ral net­works and au­tom­ata learn from mul­ti­ple rounds of feed­back that build on each other, learn­ers re­quire se­quences of feed­back ses­sions that help them re­fine their skill or ca­pa­bil­ity.

• RE­SPON­SIVE: Good feed­back is re­spon­sive to the learner’s ob­jec­tions or in­ter­pre­ta­tions of the feed­back. It is nei­ther opaque nor de­fin­i­tive, even if and when it is le­git­i­mate and im­par­tial.

Two Routes to One Big Op­por­tu­nity

As in­di­cated herein, the cur­rent sys­tem of pro­fes­sional and higher ed­u­ca­tion is very far from em­body­ing the in­sights of feed­back sci­ence. Given the foun­da­tional im­por­tance of feed­back to learn­ing and the gap be­tween cur­rent and op­ti­mal feed­back prac­tices, we are faced with a sig­nif­i­cant op­por­tu­nity to make a $2 tril­lion-in­dus­try mas­sively more ef­fec­tive by chang­ing its feed­back prac­tices.

What if the learn­ing out­comes that the cur­rent lec­ture­home­work-quiz-exam course achieves in 25 hours of lec­tures and 50 hours of home­work and test­ing can be repli­cated in a feed­back-in­ten­sive en­vi­ron­ment with just four-to-six hours of learner-teacher time? The op­por­tu­nity is sig­nif­i­cant both ed­u­ca­tion­ally and fi­nan­cially. Sev­eral or­ga­ni­za­tions with Face­book­sized rev­enue streams could live well from even a 10 to 20 per cent re­duc­tion in the costs of ed­u­ca­tion driven by changes in feed­back prac­tices.

There are two routes to the re­al­iza­tion of this op­por­tu­nity, and both are likely to emerge and de­velop within the next five years. Each has the po­ten­tial to rad­i­cally change the way teach­ing and learn­ing are done. The first makes use of the se­man­tic, di­a­log­i­cal and con­ver­sa­tional ca­pa­bil­i­ties of AI agents and en­hanced for­mal and nat­u­ral lan­guage-pro­cess­ing tech­nolo­gies, while the sec­ond re­lies on a new gen­er­a­tion of teach­ers and ed­u­ca­tors mak­ing feed­back the cen­ter­piece of their cur­ric­u­lar de­signs and teach­ing plans. Let’s take a closer look at each.

1. FEED­BACK BE­COMES AL­GO­RITH­MIC. Walk­ing in the foot­steps of IBM’S Wat­son and Bluemix, and mak­ing use of deep learn­ing ecolo­gies of al­go­rithms and plat­forms like Google’s Ten­sor­flow and Mi­crosoft’s Cog­ni­tive Ser­vices, adap­tive feed­back agents (AFA’S) will take the learner’s ‘stream of thought’ at­tempt to solve

a prob­lem and give tar­geted, im­me­di­ate, spe­cific, ob­jec­tive, ac­cu­rate feed­back on each step of that learner’s process of rea­son­ing or cal­cu­la­tion, along with sug­ges­tions for re­me­dial ex­er­cises or drills that de­velop each sub-skill or com­pe­tency re­quired for the suc­cess­ful ex­e­cu­tion of a task.

Pow­ered by a data­base of ques­tions, prob­lems, an­swers and solutions from some 58 mil­lion learn­ers tak­ing some 13,000 mas­sively open (MOOC) and small pri­vate on­line cour­ses (SPOC) of­fered by 700 uni­ver­si­ties around the clock, AFA’S will be trained to ad­dress pat­terns of er­rors, idio­syn­cra­sies and rea­son­ing styles that learn­ers ex­hibit. New re­sults from feed­back sci­ence can be em­bed­ded into feed­back prac­tice via up­dates to al­go­rith­mic plat­forms with­out the need to train up armies of teach­ing as­sis­tants and graders. Feed­back can thus be lib­er­ated from the fluc­tu­a­tions of qual­ity, mood, re­sources and acu­men of hu­man graders, for those skills that are suf­fi­ciently ex­plicit and cog­ni­tive in na­ture to be tracked by al­go­rith­mic agents.

2. THE FEED­BACK-CEN­TRIC LEARN­ING FA­CIL­I­TA­TOR EMERGES The Fourth In­dus­trial Rev­o­lu­tion is not only one in which many tasks pre­vi­ously per­formed by hu­mans can be per­formed by al­go­rith­mic agents hooked up to server farms, but also one in which the na­ture of the high­est-value tasks per­formed by hu­mans have changed, be­com­ing pre­dom­i­nantly so­cial, re­la­tional and in­ter­ac­tive.

Eighty per cent of the work man­agers now do in or­ga­ni­za­tions is per­formed in groups and teams, and hence, the skills most prized by or­ga­ni­za­tions are com­mu­nica­tive and re­la­tional in na­ture. They com­prise as many and even more af­fec­tive skills (em­pathic ac­cu­racy, ex­pres­sive­ness) and ex­ec­u­tive skills (like prob­lem struc­tur­ing and quick task switch­ing) as they do cog­ni­tive skills.

With af­fec­tive com­put­ing still in a tur­bu­lent — though promis­ing — in­fancy, there is a need to rapidly de­velop the lan­guage and base of ex­per­tise for giv­ing feed­back on in­ter­per­sonal, re­la­tional and com­mu­nica­tive ‘gen­res’ of work — such as board pre­sen­ta­tions, sales pitches, ne­go­ti­a­tions, de­lib­er­a­tions, pro­cesses of col­lab­o­ra­tive in­quiry and de­bate — that will en­able and foster real learn­ing of skills that are (still) quintessen­tially hu­man and very ‘hot’ in the labour mar­ket.

‘Com­mu­ni­ca­tion skill’ is now used as a catch-all la­bel, which makes the de­vel­op­ment of all of the skills that go into ‘com­mu­ni­cat­ing’ very far from the elab­o­rate eval­u­a­tion rubrics that have been de­vel­oped over a cen­tury of prac­tice in teach­ing and grad­ing Cal­cu­lus, Mi­croe­co­nomics, struc­tured lan­guage pro­gram­ming or ther­mal sys­tem de­sign quizzes. But progress on cre­at­ing the prac­tices that will pro­mote the rapid ac­qui­si­tion and trans­fer of these in-de­mand skills re­quires that we think care­fully about the se­man­tic and syn­tac­tic (e.g. co­her­ence and com­plete­ness) and di­a­log­i­cal and in­ter­ac­tive (e.g. re­spon­sive­ness, in­for­ma­tive­ness) as­pects of the learner’s be­hav­iour in a so­cial con­text — and that our feed­back prac­tices re­flect a much higher level of pre­ci­sion.

Mi­h­nea Moldoveanu is the De­sau­tels Pro­fes­sor of In­te­gra­tive Think­ing, Pro­fes­sor of Busi­ness Eco­nomics, Vice-dean of Learn­ing, In­no­va­tion and Ex­ec­u­tive Pro­grams, Di­rec­tor of the Mind Brain Be­hav­iour Hive and Aca­demic Di­rec­tor of the Self De­vel­op­ment Lab­o­ra­to­rytm and the Lead­er­ship De­vel­op­ment Lab­o­ra­tory at the Rot­man School of Man­age­ment, as well as Vis­it­ing Pro­fes­sor at Har­vard Busi­ness School. Maja Dji­kic is As­so­ciate Pro­fes­sor and the Ex­ec­u­tive Di­rec­tor of the Self De­vel­op­ment Lab­o­ra­tory at the Rot­man School of Man­age­ment. Over the past six years, they have de­signed, de­vel­oped and im­ple­mented feed­back sci­ence-based learn­ing in the Self-de­vel­op­ment Lab­o­ra­tory — the Rot­man School’s per­sonal de­vel­op­ment en­gine for its pro­fes­sional stu­dents.

Rot­man fac­ulty re­search is ranked #3 glob­ally by the Fi­nan­cial Times.

Newspapers in English

Newspapers from Canada

© PressReader. All rights reserved.