Will better test mean better docs?
Revised test to focus on more than science acumen
For July 2020, some envision a new type of doctor entering the workforce: One who may not write very well but has extensive biochemistry knowledge, has a solid grasp of behavioral and social sciences, and has critical analysis and reasoning powers never seen before in the halls of medicine.
A 21-person Association of American Medical Colleges committee began the preliminary work on creating this vision three years ago, and the panel’s recommendations for revising the Medical College Admission Test will be put to a vote during a Feb. 15-16 meeting of the AAMC board. If approved, a new test will be built and would be expected to be ready for students to take in 2015.
Some say the new test may replace the stereotypical med-school applicant pool from the driven, Type A personalities who know just about everything except how to relate to other people with a new breed of student full of empathy and compassion. Others say it may lead to the Type A’s being better trained to deal with others who don’t match their intellect, but at the expense of either pushing nonscience majors away from considering medicine or forcing them to take five years of undergraduate schooling in order to squeeze in the required coursework.
And then there are those who say, “Relax, it’s just a test.”
The effort has been dubbed “MR5” because, according to the AAMC, this is the fifth comprehensive revision of the MCAT since its orig- inal version was used in 1928. The current edition has been in use since 1991, and it’s likely that a revised test will be used until 2030. While people are generally aware of the test’s strengths and limitations, there appears to be a desire to use it for something deeper than just measuring a student’s ability to memorize scientific facts.
Dr. Darrell Kirch, the AAMC’S president and CEO, touched upon this in his address at the organization’s annual meeting this month.
“The MCAT exam is certainly a reliable tool to measure cognitive ability—that is, ‘brightness’—in certain areas, but we all know how little it tells us about the attitudes, values and experiences that may make an applicant truly among the best,” Kirch said in his speech.
“Our own AAMC public opinion surveys show this dichotomy. While the people we serve have a high level of confidence in the medical knowledge of our graduates, a significant percentage of them express real concern about the bedside manner of the doctors we produce. In essence, the public is more confident in our ability to bring the ‘brightest’ to medicine than in our ability to find and educate the ‘best.’”
Recommendations for the new exam call for eliminating a required writing sample, and creating four test sections that would cover biological and biochemical foundations of living systems; chemical and physical foundations of biological systems; psychological, social and biological foundations of behavior; and critical analysis and reasoning skills.
The last two sections would be new additions to the test and, according to an AAMC MCAT “preview guide,” the first three sections of the proposed test are organized around “big ideas” in science and emphasize “deep knowledge of the most important scientific concepts over shallow knowledge of many discrete scientific facts.” The preview guide also states how, “Before drawing a conclusion, scientists identify key assumptions and evaluate evidence relative to their claim.” The fourth section measures an applicant’s ability to do so.
In addition to the main panel working on the revisions, a smaller group formed an “innovation lab” that last month issued its own recommendations about what “personal competencies” a new test should measure in order to predict successful performance in medical school. These six competencies were identified as integrity and ethics; reliability and dependability; service orientation; social, interpersonal and teamwork skills; desire to learn; and resilience and adaptability.
According to Karen Mitchell, MCAT senior director and staff leader of the MR5 initiative, 90 outreach events were held, and the committee received plenty of recommendations.
“There certainly was a wide range of views of what the MCAT should do and test for,” Mitchell says, including those who thought medical-school students should be required to have advanced scientific knowledge upon arrival and those who sought a more “well-read and well-rounded” applicant pool who could be taught the science they need to know when they got there. “I think certainly one of the big messages the test is communicating is to have balanced preparation,” Mitchell says, adding that the new MCAT will require applicants to demonstrate “not only mastery of the content, but being able to use that knowledge” with scientific problem-solving and statistical analysis.
Dr. George Thibault, president of the Josiah Macy Jr. Foundation medical education advocacy group and former chief medical officer at Brigham and Women’s Hospital in Boston, says he’s encouraged by the work of the MR5 panel, but he thinks the test’s potential impact needs to be kept in perspective and that the revised MCAT cannot be seen as the “end all and be all” in determining how to develop a physician workforce that will meet society’s near-future needs.
“I don’t think the test alone will be a sufficient tool to bring about the change. … Tests are not all that good or powerful—no matter how well-devised,” Thibault says. “Revision of the test is a small piece of a holistic process.”
Richard Macdonald, professor of biochemistry and molecular biology at the University of