CAD soft­ware-What's wrong with this pic­ture?

De­tec­tion sys­tems draw mixed re­views in ra­di­ol­ogy

Modern Healthcare - - Front Page -

Early de­tec­tion” has long been the buz­zword in can­cer care. So when com­puter-as­sisted de­tec­tion, or CAD, soft­ware hit the mar­ket­place 12 years ago, man­u­fac­tur­ers and providers alike en­vi­sioned a day when the tech­nol­ogy would be­come so so­phis­ti­cated it might all but elim­i­nate the pos­si­bil­ity of miss­ing an early can­cer.

To­day CAD, which uses com­puter-gen­er­ated mark­ings to draw a ra­di­ol­o­gist’s at­ten­tion to ques­tion­able ar­eas on an im­age, is far from be­ing flaw­less in its func­tion or ubiq­ui­tous in its use. But a grow­ing num­ber of provider or­ga­ni­za­tions are mak­ing the tech­nol­ogy part of their im­age-read­ing op­tions and pro­to­col.

Ac­cord­ing to a Fe­bru­ary 2005 study pub­lished in the Amer­i­can Jour­nal of Roentgenol­ogy, 10% of mam­mog­ra­phy fa­cil­i­ties in the U.S. had adopted CAD tech­nol­ogy within three years of the U.S. Food and Drug Ad­min­is­tra­tion’s ap­proval of the first CAD de­vice in 2002.

“A lot of the CAD sys­tems are act­ing as an as­sis­tant to ra­di­ol­o­gists—es­pe­cially in breast imag­ing—so that we can ac­cu­rately as­sess the im­age,” say Gil­lian New­stead, a pro­fes­sor of ra­di­ol­ogy at the 548-bed Uni­ver­sity of Chicago Med­i­cal Cen­ter.

CAD cur­rently is be­ing used in breast mam­mog­ra­phy, lung imag­ing and colon-can­cer screen­ing, but de­spite its prom­ise, many pro­po­nents of the tech­nol­ogy say its de­vel­op­ment has stalled in re­cent years, and avail­able CAD sys­tems haven’t kept pace with the imag­ing modal­i­ties with which they are paired.

“I did an eval­u­a­tion of mam­mog­ra­phy CAD sys­tems in 2002, and since then the tech­nol­ogy’s un­der­ly­ing al­go­rithms haven’t changed at all,” says Ja­son Laun­ders, med­i­cal physi­cist and se­nior project of­fi­cer with the health­care-tech­nol­ogy anal­y­sis group ECRI In­sti­tute based in Ply­mouth Meet­ing, Pa.

Other in­dus­try ex­perts ex­press sim­i­lar con­cern about the state of CAD tech­nol­ogy. “There have been no new CAD de­vice applications cleared by the FDA since 2006,” says David Fisher, manag­ing di­rec­tor of the Med­i­cal Imag­ing & Tech­nol­ogy Al­liance, a group rep­re­sent­ing ra­di­ol­ogy equip­ment man­u­fac­tur­ers. “We es­ti­mate that be­tween 10 and 20 CAD applications have gone to the FDA, but none have been cleared.”

Fisher de­clined to say which com­pa­nies have sub­mit­ted CAD prod­ucts for FDA re­view within the past four years, but he says he be­lieves that man­u­fac­tur­ers’ and FDA re­view­ers’ con­fu­sion over the pre­mar­ket ap­proval process is to blame for the lack of prod­uct ap­provals.

Now some ob­servers are wor­ried that the FDA’s pro­posed new guid­ance on the CAD tech­nol­ogy ap­proval process could make get­ting new and up­dated prod­ucts into the hands of providers nearly im­pos­si­ble. They say added re­quire­ments for clin­i­cal stud­ies and de­sign ver­i­fi­ca­tion will make CAD de­vel­op­ment too costly, par­tic­u­larly since the tech­nol­ogy pro­vides lim­ited re­im­burse­ment op­por­tu­ni­ties for providers. Mam­mog­ra­phy CAD is the most widely cov­ered form of the de­vice’s use, with Medi­care and most pri­vate pay­ers re­im­burs­ing providers for CAD ser­vices. Lung CT and colon-screen­ing CAD is less of­ten cov­ered.

“Th­ese can be ex­pen­sive add-on soft­ware sys­tems to buy,” Laun­ders says. “It can cost $50,000, so it’s the kind of thing hos­pi­tals knock off the or­der when they’re looking to buy new imag­ing tech­nol­ogy.”

Throw­ing down a new gaunt­let

In Oc­to­ber 2009, af­ter two years of work by a ra­di­o­log­i­cal de­vice ad­vi­sory panel, the FDA re­leased pro­posed guid­ance for its CAD tech­nol­ogy ap­proval process. Con­tained in two doc­u­ments, the draft guid­ance is the agency’s first at­tempt to stan­dard­ize and clar­ify the CAD prod­uct-ap­proval ap­pli­ca­tion process.

“The goal is to let in­dus­try know what we hope to see in an ap­pli­ca­tion and what cri­te­ria we need them to meet,” says Ni­cholas Pet­rick, deputy di­rec­tor of the FDA’s Cen­ter for De­vices and Ra­di­o­log­i­cal Health’s sci­ence and en­gi­neer­ing lab­o­ra­to­ries of­fice.

The first of the two doc­u­ments de­tails pro­posed new rules for the 510(k) sub­mis­sion process—which cov­ers CAD prod­uct up­dates, moderate-risk de­vices and new prod­ucts that are equiv­a­lent to al­ready ap­proved CAD tech­nol­ogy. The sec­ond doc­u­ment di­rects man­u­fac­tur­ers on how to de­sign and con­duct their clin­i­cal-per­for­mance as­sess­ment for both 510(k) and new, high-risk CAD prod­ucts ap­proved un­der the pre-mar­ket ap­pli­ca­tion, or PMA, process.

“We think the process they’ve put for­ward is ex­ces­sively bur­den­some,” Fisher says of the draft guid­ance.

Un­der FDA reg­u­la­tions, com­pa­nies are re­quired to pro­vide more per­for­mance ev­i­dence and clin­i­cal-test­ing data for PMA applications than for those ap­proved through the 510(k) process. But crit­ics say that the bar for prov­ing the safety and ef­fi­cacy of CAD prod­ucts un­der the 510(k) process would be raised too high if the agency’s cur­rent draft guid­ance is ap­proved.

“This sub­stan­tially bumps the re­quire­ments of a 510(k) de­vice up to that of a PMA,” says Robert Nishikawa, a Uni­ver­sity of Chicago med­i­cal physi­cist who has con­sulted with a num­ber of com­pa­nies on ra­di­ol­ogy prod­uct

de­vel­op­ment. The change would likely add time and ex­pense to the ap­proval process.

Un­der the pro­posed 510(k) ap­pli­ca­tion guide­lines, man­u­fac­tur­ers would be re­quired to de­tail how they de­signed the al­go­rithms that guide their CAD tech­nol­ogy to iden­tify po­ten­tial ar­eas of con­cern on an im­age. CAD de­vice­mak­ers are wor­ried, how­ever, that the re­quire­ment would force them to dis­close pro­pri­etary in­for­ma­tion.

“We spend mil­lions and mil­lions of dol­lars to de­sign those al­go­rithms, and we don’t want to re­veal those trade se­crets,” says Ju­lian Mar­shall, di­rec­tor of prod­uct man­age­ment for work­flow so­lu­tions for Ho­logic, a Bed­ford, Mass.-based de­vel­oper of mam­mog­ra­phy CAD soft­ware. He ar­gues that the FDA should be able to glean safety and ef­fi­cacy in­for­ma­tion from stand-alone test­ing of CAD de­vices. But the va­lid­ity of stand-alone test­ing is also be­ing chal­lenged by the ad­vi­sory panel that wrote the guide­lines.

Pre­vi­ously, CAD man­u­fac­tur­ers have been able to sub­mit stand-alone test anal­y­sis—which looks at how ac­cu­rately a CAD sys­tem marks im­ages from pre­vi­ously de­ter­mined cases—with their 510(k) applications to demon­strate that their de­vices per­form on par with oth­ers al­ready on the mar­ket. But the FDA’s pro­posed guid­ance would also re­quire de­vice­mak­ers to per­form clin­i­cal stud­ies on 510(k)-el­i­gi­ble de­vices. The guid­ance would, in most cases, re­quire de­vice­mak­ers to con­duct mul­ti­ple reader, multi- ple case stud­ies where ra­di­ol­o­gists’ read­ings of an im­age would be com­pared against the new CAD sys­tem’s as­sess­ment of an im­age.

“One of the prob­lems is that the man­u­fac­tur­ers don’t have ac­cess to their com­peti­tors’ al­go­rithms, so it’s dif­fi­cult for them to know if their prod­uct is equiv­a­lent,” Pet­rick says. “So in those cases, I think we’ll def­i­nitely need to have reader stud­ies that show the prod­uct is equiv­a­lent.” Crit­ics of the draft guid­ance say the re­quire­ment is not re­al­is­tic. “Hav­ing run mul­ti­ple reader, mul­ti­ple case tri­als, I’m very sen­si­tive to how dif­fi­cult lo­gis­ti­cally it is to put to­gether th­ese stud­ies,” says Abe Dach­man, pro­fes­sor of ra­di­ol­ogy with the Uni­ver­sity of Chicago Med­i­cal Cen­ter. “Find­ing ra­di­ol­o­gists who are will­ing to read the cases two and three times is dif­fi­cult, and com­pa­nies want to avoid do­ing mul­ti­ple reader, mul­ti­ple case stud­ies be­cause it’s so ex­pen­sive.”

Dach­man was un­able to es­ti­mate the cost of mul­ti­ple reader clin­i­cal stud­ies, but Ho­logic’s Mar­shall says the ex­pense eas­ily runs into hun­dreds of thou­sands of dol­lars.

The FDA also would re­quire that 510(k) clin­i­cal stud­ies in­clude a con­trol group to show if there is a dif­fer­ence be­tween how ac­cu­rately ra­di­ol­o­gists as­sess im­ages when they are aided by CAD tech­nol­ogy on a first read and when they are un­aided by the tech­nol­ogy on a first read.

Cer­tainly, the find­ings on the ben­e­fits of CAD are mixed. An April 5, 2007


New Eng­land Jour­nal of Medicine In­flu­ence of Com­puter-Aided De­tec­tion on Per­for­mance of Screen­ing Mam­mog­ra­phy,

found for ex­am­ple that the use of CAD was as­so­ci­ated with re­duced im­age in­ter­pre­ta­tion ac­cu­racy by physi­cians and in­creased rates of biopsy, but was not clearly as­so­ci­ated with im­proved de­tec­tion of in­va­sive breast can­cer.

An­other study,

Sin­gle Read­ing With Com­puter-Aided De­tec­tion for Screen­ing Mam­mog­ra­phy,

pub­lished Oct. 16, 2008, in the same med­i­cal jour­nal, found that dou­ble read­ing of a mam­mo­gram im­age by two ra­di­ol­o­gists pro­duced more ac­cu­rate con­clu­sions than a sin­gle read of an im­age cou­pled with CAD as­sis­tance.

New­stead says the chal­lenge of CAD tech­nol­ogy is that be­cause it is more sen­si­tive to po­ten­tial anom­alies than the hu­man eye, it of­ten calls at­ten­tion to per­fectly healthy tis­sue. Some stud­ies have found, for ex­am­ple, that CAD typ­i­cally gen­er­ates two to four false-pos­i­tive marks on a sin­gle mam­mog­ra­phy study. That means more pa­tients are likely to be re­called for un­nec­es­sary biop­sies and other tests, par­tic­u­larly when CAD is be­ing used by a less-ex­pe­ri­enced ra­di­ol­o­gist who may not be skilled in de­ter­min­ing if a CAD mark­ing is su­per­flu­ous.

“CAD is not per­fect,” says Chris Flow­ers, an as­so­ciate pro­fes­sor of ra­di­ol­ogy with the Uni­ver­sity of Cal­i­for­nia at San Fran­cisco’s breas­t­imag­ing pro­gram. “It can mark things that aren’t an area of con­cern, and it can miss things that are and that can be danger­ous. It can be a dis­trac­tion to the reader when there’s a pos­i­tive find­ing that ac­tu­ally rep­re­sents noth­ing.”

Flow­ers be­lieves that the FDA’s re­quest for in­for­ma­tion about al­go­rithm de­signs and ad­di­tional clin­i­cal stud­ies for CAD 510(k) applications are pos­i­tive moves given un­cer­tainty about the ben­e­fits of CAD. “I’m con­cerned that any­time you are go­ing to al­low a com­puter-aided de­vice to be in­volved in de­tec­tion you have to be care­ful to make sure the al­go­rithms are ac­cu­rate.”

Maryellen Giger is a pro­fes­sor of ra­di­ol­ogy at the Uni­ver­sity of Chicago and a mem­ber of the med­i­cal cen­ter’s CAD imag­ing team.

New­stead: CAD is used as “an as­sis­tant to ra­di­ol­o­gists.”

Newspapers in English

Newspapers from USA

© PressReader. All rights reserved.