Wear­able tech­nol­ogy boosts ‘sight’ for vis­ually im­paired

Re­searchers say de­vice makes life eas­ier for the blind

Baltimore Sun Sunday - - NEWS - By Jonathan M. Pitts

Jake Sch­mude was born with­out op­ti­cal nerves — a rare con­di­tion that left him blind. But the talk­a­tive com­puter whiz has coped ex­ceed­ingly well.

Sch­mude, 32, a net­work ad­min­is­tra­tor at a non­profit or­ga­ni­za­tion for the vis­ually im­paired, lives in­de­pen­dently in Gwynn Oak. He com­mutes to work by Uber and Lyft. And he has long used smart­phone apps that can scan text and “read” it aloud.

But it wasn't un­til 2016 that he felt fully com­fort­able car­ry­ing out many of his nec­es­sary daily tasks.

That spring, Sch­mude bought a cut­tingedge as­sis­tive de­vice that fits atop the frames of an or­di­nary pair of glasses.

Known as the OrCam MyEye, the de­vice con­sists of two parts: A one-ounce cam­era that mounts on his spec­ta­cles and a six-ounce com­puter clipped to his belt. It’s the first op­ti­cal char­ac­ter recog­ni­tion tech­nol­ogy that can be worn on the head.

By di­rect­ing its gaze to­ward a block of text or ob­ject, or by sim­ply point­ing a finger at the tar­get, he ac­ti­vates a process by which MyEye turns scanned vis­ual in­for­ma­tion into spo­ken lan­guage. He hears the voice through an ear­piece.

The process hap­pens al­most in­stantly, and with an ac­cu­racy that in­de­pen­dent stud­ies have shown to ex­ceed that of other tech­nolo­gies on the mar­ket.

Sch­mude says he can now com­fort­ably carry out tasks that had al­ways been chal­leng­ing or im­pos­si­ble, from read­ing the text on a curved can or a dis­tant bill­board to dis­tin­guish­ing faces or the color of an ob­ject.

“It has made a lot of what I do eas­ier,” he says.

Sch­mude’s em­ployer, Blind In­dus­tries and Ser­vices of Mary­land in Halethorpe, was so im­pressed with the de­vice that it set up a pay­ment plan to help its 250 legally blind em­ploy­ees meet the price of $3,500. More than 70 staff mem­bers have pur­chased the de­vice.

Fred­er­ick J. Puente, pres­i­dent of Blind In­dus­tries, per­suaded the board of trustees to make the in­vest­ment.

“When an as­so­ciate comes to you with tears in their eyes and tells you that for the first time they are able to read the news­pa­per them­selves, it makes the de­ci­sion to of­fer this pro­gram a very easy one,” Puente says.

In­de­pen­dent stud­ies sug­gest that the de­vice brings a new level of ease to daily ac­tiv­i­ties. much

Re­searchers at the Uni­ver­sity of Cal­i­for­nia, Davis, for ex­am­ple, showed that 12 legally blind adults could per­form rou­tine daily tasks with sig­nif­i­cantly greater ease while us­ing MyEye than while us­ing other de­vices with which they were fa­mil­iar.

An ar­ti­cle de­tail­ing the re­sults ap­peared in the jour­nal JAMA Oph­thal­mol­ogy last May.

“Pa­tients with low vi­sion of­ten are de­pen­dent on hand­held or elec­tronic mag­ni­fiers, which may be some­what cum­ber­some to use,” Dr. Elad Mois­seiev, a co-au­thor and a UC Davis vit­re­o­reti­nal surgery fel­low, said when the study was com­plete. “This is the first in­de­pen­dent clin­i­cal study to eval­u­ate this new low-vi­sion­aid de­vice based on novel op­ti­cal char­ac­ter recog­ni­tion tech­nol­ogy.

“Our re­sults show that it can be a very use­ful aid for pa­tients with low vi­sion in per­form­ing ac­tiv­i­ties of daily liv­ing, and in­crease their in­de­pen­dence.”

Op­ti­cal char­ac­ter recog­ni­tion — a branch of com­puter sci­ence that in­volves trans­lat­ing op­ti­cally scanned printed or writ­ten text char­ac­ters into a coded form that can be ma­nip­u­lated by a com­puter — is not new. An Is­raeli in­ven­tor, Em­manuel Gold­berg, cre­ated a ma­chine a cen­tury ago that could read char­ac­ters and convert them into tele­graph code. The Ir­ish physi­cist Ed­mund d'Albe in­vented one that could scan text and trans­late it into au­di­ble tones.

By the 1970s, the Amer­i­can com­puter scientist Ray Kurzweil was work­ing to per­fect sys­tems that could read text in any font. It was his com­pany, Kurzweil Com­puter Prod­ucts Inc., that com­bined the use of two new tech­nolo­gies — the CCD flatbed scan­ner and the text-to-speech syn­the­sizer — into the first op­ti­cal char­ac­ter recog­ni­tion com­puter pro­gram in 1976.

In more re­cent years, scores of de­vices have hit the mar­ket that adapt the tech­nol­ogy for use by the blind and vis­ually im­paired. Smart­phone apps help users count cash (LookTel Money Reader), de­tect col­ors (Color Iden­ti­fier) and hear the prompts on their phones read aloud (VoiceOver).

OrCam co-founder Am­non Shashua ear­lier founded Mo­bil­eye, a firm that de­vel­oped tech­nol­ogy that could use a cam­era and soft­ware al­go­rithms to de­tect the pres­ence of ve­hi­cles on the road.

Com­puter gi­ant In­tel bought Mo­bil­Eye for $15.3 bil­lion, and Shashua and his team of 100 re­search and de­vel­op­ment as­so­ci­ates turned the tech­nol­ogy to­ward help­ing the blind. Orcam spokes­woman Anat Nul­man says the com­pany's al­go­rithms al­lowed de­sign­ers to make the MyEye cam­era a mere 3 inches long and the com­puter-and-bat­tery pack 5 inches long, and thus mount­able on the rims of glasses.

In or­der to take the needed “pic­ture,” the user em­ploys much the same mo­tion a sighted per­son would use to “look,” di­rect­ing his or her “gaze” to­ward the tar­get, then trig­gers the ac­tion in one of two ways: by flip­ping a switch on the pack or by hold­ing that gaze for a pre­s­e­lected num­ber of sec­onds.

To “view” a tighter area, the user points at the tar­get, and the de­vice, which is loaded with thou­sands of pic­tures of the hu­man finger, rec­og­nizes the ges­ture as an ac­ti­va­tion com­mand.

The al­go­rithms also en­able the de­vice to read print from curved sur­faces — a boon for the blind as they pe­ruse cans in gro­cery stores — as well as to read text as large as the let­ter­ing on a bill­board or as small as the writ­ing on a pass­port.

As with all OCR de­vices, Nul­man says, its de­gree of ac­cu­racy de­pends on ex­ter­nal fac­tors such as light­ing and dis­tance. Stud­ies have shown ar­eas of needed im­prove­ment.

But Sch­mude, the IT maven at Blind In­dus­tries who turned his bosses and co-work­ers on to the tech­nol­ogy, is im­pressed.

“Just a few years ago you’d have needed a back­pack to carry enough equip­ment to do what this de­vice does, and it would have had to off­load the in­for­ma­tion to a server,” he says. “What they’ve man­aged to pack into this de­vice is hon­estly in­cred­i­ble.”

His col­league, Chris Jones, a ship­ping and re­ceiv­ing clerk, is less tech-savvy, but he agrees with the larger point: the de­vice helps him in ways oth­ers have not.

Legally blind with very lim­ited vi­sion, Jones, 27, can see larger shapes such as peo­ple, streets and mov­ing cars, but not the kind of de­tail that would iden­tify them fur­ther.

A fit­ness nut, he en­joys bik­ing to work, and MyEye — which he wears atop his sun­glasses — al­lows him to vary his route based on the work­out he wants.

Be­fore he got the de­vice, he couldn’t read street signs. Now he points and hears their names read into his ear­phones.

“The voice will say, ‘Lans­downe Road,’ or ‘Ham­monds Ferry Road,’ and I know ex­actly where I am,” he says.

Jones says he can now read his own bank state­ments and work or­ders with­out both­er­ing his sighted friends, and MyEye even helps him keep tabs on his two young daugh­ters.

At times they’ll play­fully try to hide from him in the aisles at Wal-Mart, he says, but to their shock, the fa­cial recog­ni­tion func­tion helps him “nail” them by name. He laughs. “This tech­nol­ogy just helps you be more in­de­pen­dent and self-suf­fi­cient,” he says. “I'm all about that."


The OrCam MyEye is the first op­ti­cal char­ac­ter recog­ni­tion tech­nol­ogy that can be worn on the head. Con­vert­ing scanned vis­ual in­for­ma­tion into words hap­pens al­most in­stantly.

Newspapers in English

Newspapers from USA

© PressReader. All rights reserved.