Costly and Unnecessary?
By Kate Harrington Texas Medicine December 2016

Medical Students, Physicians Call for an End to USMLE Step 2 Clinical Skills Exam

Texas Medicine Logo(1)

Education Feature — December 2016

By Kate Harrington

Many U.S. medical students, residents, and physicians — more than 17,000 — want to see the elimination of the U.S. Medical Licensing Exam (USMLE) Step 2 Clinical Skills (Step 2 CS) exam, which tests fourth-year medical students' and international medical graduates' patient care skills. Students, as well as some physicians, say the test's high price tag of $1,275 to register plus travel and lodging expenses, coupled with its 95-percent first-attempt pass rate, make it a costly and unnecessary exam. 

"At least three-quarters of U.S. medical schools already require a clinical skills exam passage to graduate," said Kevin McKinney, MD, an associate professor of endocrinology at The University of Texas Medical Branch School of Medicine in Galveston. "Medical students feel like they are already getting that kind of evaluation locally. It is such an expense, and it takes so much time. You might try to find a way to get it done in a high-quality local environment."

Dr. McKinney was quite vocal about his stance on the exam at the Annual Meeting of the American Medical Association House of Delegates in June. 

"That exam is just not a good enough discriminator between good students and bad students," he testified. 

AMA delegates agreed and adopted a plan to facilitate a transfer of jurisdiction for CS exams directly to the nation's accredited medical schools. The Texas Medical Association also has policy that supports clinical skills assessments being conducted by individual medical schools.

But not everyone is on the test-reform bandwagon. Those who support the test as it is currently given say it allows for an assessment of students' physical exam aptitude and helps ensure medical schools are teaching patient care skills. Some go so far as to say giving medical schools administrative control of the exam would put patients in jeopardy. 

An Unpopular Exam

The Step 2 CS exam originally began as a test known as Clinical Skills Assessment, which tested the English language proficiency of international medical students. That changed in 2004, when the two groups that oversee USMLE, the National Board of Medical Examiners (NBME) and the Federation of State Medical Boards (FSMB), added the clinical skills exam to the Step 2 examination. 

All fourth-year medical students must travel to one of five testing sites in Houston, Atlanta, Chicago, Los Angeles, or Philadelphia and pay the exam fee to take the Step 2 CS. It also means the exam is now a requirement of state medical licensure. 

Many students say the exam, taken in a staged setting with actors portraying patients, doesn't accurately reflect the experience of interacting with and treating patients in reality. Compounding its unpopularity is the fact that students receive a "pass" or "fail" designation for the exam, with no feedback or nuanced score.

As of September, more than 17,000 students had signed a petition calling for the elimination of the Step 2 CS exam. The petition was started by a group of fourth-year students at Harvard Medical School. The movement includes students, residents, and physicians from more than 130 medical schools in 45 states plus the District of Columbia and Puerto Rico.

Research published in the New England Journal of Medicine estimates a cost of $1.1 million to catch a single student who fails the Step 2 CS on back-to-back attempts. Given the fraction of examinees who fail Step 2 CS relative to its exorbitant costs, the exam offers a poor return on investment, the petition supporters say. Similarly, the petition concludes, "Step 2 CS adds no provable benefit to U.S. medical education or patient safety at an enormous cost. The exam should not be required for licensing of U.S. medical school graduates."

"In reality, the exam is a waste of time and money for U.S. medical school students," says Jerome Jeevarajan, a third-year medical student at The University of Texas Southwestern Medical Center in Dallas. "There have been studies that have shown there's a weak correlation between scores and the evaluations everyone gets at the end of the first intern year. And studies show that there's no predictive value beyond written exams … yet the Step 2 CS is the most expensive exam."

Mr. Jeevarajan, the AMA delegate coleader of TMA's Medical Student Section Executive Council, worked to gather support for the AMA resolution on the Step 2 CS exam that delegates adopted in June. 

"One of our main goals was to reach out to physicians in our delegation," Mr. Jeevarajan said. "We spent a lot of time answering questions and going over details with members of the Texas delegation and eventually garnered their full support for the measure. It's amazing to see how quickly this type of change can happen. Leaders from California to Harvard are working together with physicians to make this big change in medical education."

During that June meeting, the AMA House of Delegates adopted new policy that calls for working with stakeholders to pursue a transition away from the current Step 2 CS exam toward a medical school-administered exam.

AMA's adopted policy states it will work:

 

  • With FSMB, NBME, state medical societies, state medical boards, and other key stakeholders to pursue the transition from and replacement for the current USMLE Step 2 CS examination and the Comprehensive Osteopathic Medical Licensing Examination (COMLEX) Level 2-Performance Examination, with a requirement to pass a Liaison Committee on Medical Education (LCME)-accredited or Commission on Osteopathic College Accreditation-accredited medical school-administered, clinical skills examination. 
  • To ensure rapid yet carefully considered changes to the current examination process to reduce costs, including travel expenses, as well as time away from educational pursuits, through immediate steps by FSMB and NBME; encourage a significant and expeditious increase in the number of available testing sites; allow international students and graduates to take the same examination at any available testing site; engage in a transparent evaluation of basing this examination within our nation's medical schools, rather than administered by an external organization; and include active participation by faculty leaders and assessment experts from U.S. medical schools as they work to develop new and improved methods of assessing medical student competence for  advancement into residency. 

 

TMA has similar policy, dating back to 2003, which supports clinical skills exams when given by medical schools. It states: "Given the importance of assessing clinical competency, the Texas Medical Association joins the American Medical Association in strongly urging the LCME and the American Osteopathic Association to modify and enforce uniform accreditation standards as soon as possible to require that all medical schools rigorously and consistently assess clinical skills of all students as a requirement for advancement and graduation."

Further, TMA's policy supports continued efforts to develop and validate methods for assessment of clinical skills. TMA's policy opposes the use of these methods in evaluation for licensure of graduates of LCME- and American Osteopathic Association-accredited medical schools, believing that clinical skills assessment is best performed using a rigorous and consistent examination administered by the medical school.

Steven Hays, MD, chair of TMA's Council on Medical Education, says the best way to discriminate between good and bad students "is for medical schools themselves to administer the exam, not the USMLE." He says his council "felt that the CS assessment for students is best performed using a test that is rigorous and designed by medical schools. All this money is being paid, and we need to know what is the good of all this if everyone passes."

Lynne Kirk, MD, a professor in the Internal Medicine, Family and Community Medicine Department at UT-Southwestern and a member of the AMA Council on Medical Education, says medical student input was critical in crafting and ultimately passing the new AMA policy. 

"There were lots of people speaking about this issue from both sides at the Reference Committee [on Medical Education]," Dr. Kirk said. "I thought it was a high-quality discussion, and, fortunately, representatives from the federation and national board were there and could listen. There were residents who recently had experienced taking the exam who were able to talk about the challenges, both financial and timewise, and their perception that they don't get as much in return as they could. And then on the other side, we heard reminders that we're a self-regulating profession, and these are skills the public values."

In Defense of the Exam

Not all physicians think changing the way the Step 2 CS is administered is a good idea, however. 

"It's been contested from day one by medical students," said William Fleming, MD, a Houston neurologist and former TMA and FSMB president who once served on NBME. "But the board members — the state medical boards across the country — are in favor of the exam. The federation just did a survey earlier this year among medical boards, and 83 percent of medical boards strongly agreed, with 15 percent agreeing that the USMLE Step 2 CS provides valuable medical knowledge and skills."

Dr. Fleming acknowledges the fail rate is low, at approximately 4 percent, but points out that 4 percent is still a large number of medical students who will someday interact with and treat patients. 

"That 4 percent represents more than 800 physicians who fail on the first attempt each year who otherwise would go on and become licensed and go on to practice medicine," Dr. Fleming said.  

As for medical schools taking over the administration of a clinical skills test, Dr. Fleming says it's inaccurate to say all medical schools currently teach and test those skills. U.S. medical schools haven't been equipped, or at least not uniformly equipped, to give the Step 2 CS in the past, so putting such an important task in their hands could be risky, he says. 

"I think patient safety would be at stake if the exam was abolished," Dr. Fleming says. "The medical boards have found a significant number of disciplinary actions are the result of a lack of interpersonal skills. The medical board's feeling is that we can't delegate that responsibility to medical schools, when only 74 percent of our 140 medical schools require a CS exam for graduation." 

NBME and FSMB echoed Dr. Fleming's concerns. Medical schools are not uniformly equipped, nor are their testing staffs uniformly trained, to ensure the level of standardization, reliability, and exam security provided by the USMLE program for Step 2 CS, says Drew Carlson, FSMB director of communications. FSMB responded on behalf of NBME, with which it cosponsors administration of Step 2 CS. 

"Furthermore, over 140 medical schools, each with its own curriculum, standards, and priorities, simply cannot provide the same assurance to the American public that all doctors, no matter where they were educated, have demonstrated their competency to the same standard," Mr. Carlson said.

NBME and FSMB are working to see how it might provide more feedback to medical schools in response to students' requests for more information about their performance on the exam than what they learn form a simple pass/fail designation.

"In general, however, many of the criticisms of Step 2 CS confuse the exam with those given in school," Mr. Carlson says. "Step 2 CS is a component of a licensing exam, which exists to assure the public that their physicians have demonstrated competence to the same fair standard before they can practice medicine. Step 2 CS is not a teaching tool."

Next Steps

No matter what changes are or aren't in store for Step 2 CS, NBME can't fail to heed medical students' concerns about making Step 2 CS a better exam, says Dr. Kirk. 

"Whether it's the national board working with schools to develop a better exam or looking at areas where it could be improved, all those things are options worth pursuing," she said. "Licensing is regulated by state law, and all of the state boards do require that exam, so if there was something else instead of the USMLE Step 2 CS, it would have to be as rigorous as that exam."

Medical students don't want to avoid taking a rigorous exam, say Dr. McKinney and Mr. Jeevarajan. They just don't want the expense of taking what they see as a duplicative exam. 

"The goal [of the new AMA policy] is to have an improvement in medical education," Mr. Jeevarajan says. "Since medical students are directly affected, we see the inefficiencies. This is something we all saw — a waste of money for students, schools, and, indirectly, for the government. We thought this would be a good way to save money and improve the process of medical education so we can learn more, be more effectively tested and taught, and so we can all be better doctors." 

Kate Harrington is an Austin-based writer and principal of Thumbtack Communications.

December 2016 Texas Medicine Contents
Texas Medicine Main Page

 

Last Updated On

December 01, 2016

Originally Published On

November 23, 2016

Related Content

Medical Education