AMSA Pharm-free Scorecard 2014

Executive Summary

Updated June 26, 2014

This document reflects school assessments as of the above date. For the most current assessments, please reference the main Scorecard page.

Why we need a Scorecard:

The American Medical Student Association (AMSA) launched the first PharmFree Scorecard in 2007 with the 2014 Scorecard being its seventh iteration. Since its inception, the AMSA Scorecard team has been committed to objectively evaluating conflict-of-interest policies and curricula at the 161 allopathic and osteopathic medical colleges in the United States and Puerto Rico.

The Scorecard is an evolving tool that uses letter grades to assess schools’ performance in fourteen potential areas of conflict of interest.  It offers a comprehensive look at the changing landscape of conflict-of-interest policies across U.S. medical education, as well as in-depth assessments of individual policies that govern industry interaction between students, faculty, and the pharmaceutical and medical device industries. 

While the relationships between academic physicians and industry benefit medical research and treatment, they can bias medical education in favor of specific products. Studies have shown that industry influence—whether in the form of gifts, commercially supported education, or simply visits with pharmaceutical representatives—can lead to more expensive and less evidence-based prescribing practices.  Medical schools and academic medical centers have been leaders in setting new standards for policies on potential conflicts of interest, as supported by strong guidelines set by the Association of American Medical Colleges (AAMC) in 2008 and the Institute of Medicine in 2009.  In 2012, The Pew Charitable Trusts continued this tradition by convening an expert task force of leaders from academic medical centers to create a new set of recommendations for best practices in conflict of interest policies at academic medical centers

AMSA revised its Scorecard to reflect many of the recommendations of this task force.  These changes represent an intentional strengthening of the standards of excellence with which policies are evaluated and scored. With the impending release of a federal database disclosing all financial relationships between physicians and industry, greater scrutiny of the appropriateness of these certain industry activities is particularly critical.

The new Scorecard adds three categories not present in prior years, including:

  • Extension of COI policies to community affiliates;
  • Ghostwriting and honorary authorship; and
  • Enforcement and Sanctions of Policies.

Moreover, Site access to pharmaceutical sales representatives and medical device representatives are now scored as two separate domains.  

Some institutions will notice a decline in their grades this year, because the new scorecard envisions an ideal, conflict-free medical education environment—one that will not be easy for all medical schools, even those with good policies, to achieve. Through its Scorecard, AMSA seeks to encourage schools to adopt aspirational standards in conflict of interest regulation.

You can also download archives of the 2008, 2009, and 2010 policy assessments.

Summary of Results:

As of June 28, 2014, 77 medical schools submitted policies to AMSA for scoring, a 48% response rate. This year’s relatively low participation rate is likely due to the first major change in methodology since the Scorecard’s creation. In prior years, schools were merely asked to submit policy updates. This year, because of new domains and changes to old ones, we requested entirely new policy submissions. As was true in the past, we expect participation rates to increase significantly over subsequent years. In order to continue our tradition of serving as a comprehensive database of U.S. medical schools, we nevertheless scored non-reporting schools by searching for publically available documents online (see Methodology for more details.)

Of the 161 US medical schools, 26 receive “A”s (16%), 82 “B”s (51%), 25 “C”s (16%), and 26 Incompletes (16%) This year the grades of “D” and “F” were removed because we found that trying to assign these grades to a small number of schools in the lower end of the spectrum involved difficult and potentially arbitrary decisions.  In general, schools’ conflict of interest policies fall into three major categories of policy quality:

  1. Excellent or model policies (A) in which at least half of the 14 conflict of interest domains met criteria for best practices (a domain score of 3 out of 3);  
  2. Good or solid policies (B), in which at least all 14 domains met criteria for good practices (a domain score of 2);
  3. Poor or deficient policies (C), in which more than one domain scored as a poor practice (1).

In addition, 26 schools (16%) received “I” for Incomplete. These were schools that had not submitted policies, and for which our web search yielded so little information that grading them was unlikely to be a meaningful measure of policy strength. While in prior years such schools would have been assigned Ds or Fs, this likely overestimated the number of schools with poor, as opposed to incomplete policies.  These institutions are encouraged to share updated policy information with AMSA for a possible re-grade.  After a one-year period, institutions that have not updated their information will receive a grade of “D”.

Trends and Analysis:

Consistent with the more stringent criteria of this year’s Scorecard, fewer schools received grades of A or B this year than last: In 2013, 72% of schools received the top two grades, whereas in 2014 this proportion dropped slightly to 67%. This overall decrease was entirely due to a drop off in the number of A grades (40 in 2013, and 25 in 2014). In fact, the number of schools with solid B policies increased from 74 in 2013 to 83 this year.

Changes in the Scorecard this year make it difficult to compare scores to years past. Nonetheless, of the 25 institutions receiving “A” scores, three deserve particular attention for submitting newly drafted strong policies improving their grades from Cs, Ds, and “In Process”. These schools are: Edward Via College of Osteopathic Medicine, Pacific Northwest University of Health Sciences College of Osteopathic Medicine, and Hofstra North Shore-LIJ College of Medicine.

“B” policies on the verge of excellence

Many of the solid B schools’ policies would reach “A” status with relatively minor improvements. For example, 10 “B” schools with particularly strong policies narrowly missed model status because only 6, rather than 7 (half), of their COI domains had perfect scores—these schools included: Albert Einstein College of Medicine-Yeshiva University, East Tennessee State University James H. Quillen College of Medicine, New York University School of Medicine, Southern Illinois University School of Medicine, The Commonwealth Medical College, University of Arizona College of Medicine – Tucson, University of Oklahoma College of Medicine, University of Pittsburgh School of Medicine, Virginia Commonwealth University School of Medicine, and Yale University School of Medicine.

Depending on the school, one of the following relatively minor policy adjustments would propel them from a B to an A score:

  • Requiring faculty to disclose their conflicts of interest directly to students and trainees
  • Requiring faculty to pay for their own meals in the context of industry supported continuing medical education activities
  • Discouraging faculty and trainee attendance at industry marketing programs

Trends by Domain:

Domains with similar criteria to prior Scorecards

Two domains were essentially unchanged from prior years and therefore allow year to year comparisons: industry-funded promotional speaking and site access for pharmaceutical sales representatives.

  • Industry-funded promotional speaking: The most dramatic improvement is in policies regulating promotional speaking. 79 schools now effectively ban their faculty from serving on industry promotional speaker’s bureaus, up from 44 in 2013, 31 in 2011-12, 19 in 2010, 10 in 2009, and 4 in 2008.
  • Pharmaceutical sales representative access: Nine schools now either forbid site access to pharmaceutical sales reps, or have policies in place effectively preventing them from engaging in marketing activities. This is improved from 4 in 2013, 2 in 2011-12 and 2010, and 1 in 2009 and 2008.

Domains with more stringent standards this year

Several domains have more stringent criteria this year, resulting in fewer schools receiving perfect scores.  However, the disclosure domain experienced improvements across institutions this year.

  • Scholarships: A perfect score requires that schools not accept industry funds for sending medical students to conferences, whereas in prior years industry funding was deemed acceptable as long as the funder had no role in selection of recipients. Surprisingly, very few schools (Harvard, Edward Via, and Uniformed Services) forbid industry funding for trainee travel to conferences, though the vast majority of schools (111) have robust policies preventing industry from earmarking funds to specific students. Most schools evidently believe that students are unlikely to develop a feeling of reciprocity toward a company that provides substantial funding to them.

Scholarship perfect scores 2014: 3, down from 123 in 2013.

  • Industry funding of CME: A perfect score requires no industry funding for the support of accredited CME except in clearly defined circumstances, such as through a central, undesignated, blinded pool of funds.  In prior years, centralized funding was required, but not a blinded pool that prevents any designation of the topic or recipient department. Only five schools (Edward Via, Pacific Northwest, Stanford, University of Michigan, and Brody/East Carolina) effectively prohibit any industry funding of CME that allows companies to direct funds to specific topics, courses, or departments. However, 52 schools now require that all funding must be channeled through a central CME office, which provides a greater level of oversight than systems allowing funds to go directly to departments or divisions.

Industry funding of CME perfect scores 2014: 5, down from 28 in 2013.

  • Gifts: A perfect score requires no industry-funded gifts of any nature or value, including textbooks even if paid for indirectly through a department. This textbook stipulation was absent last year, and likely accounts for the lower number of schools with a perfect gift score for 2014.

Gifts perfect score 2014: 79, down from 93 in 2013.

  • Meals: A perfect score requires that no industry-funded meals of any nature or value are allowed, including indirect industry-funded meals in the context of CME courses. In prior years, the CME specifier was not present, and including this prohibition significantly decreased the numer of schools with perfect scores on the meals domain. This year, 95 schools prohibit non-CME meals.

Meals perfect score 2014: 24, down from 93 in 2013.

  • Attendance of Industry-Sponsored Promotional Programs: A perfect score requires prohibition or discouragement of faculty, students and trainees to attend promotional programs.  In prior years, attendance was allowed, but attendees could not accept industry compensation for travel expenses. A significant number of schools (25) have opted for the more restrictive policy.

Attendance of Industry-Sponsored Promotional Programs perfect score 2014: 25, down from 101 in 2013.

  • Consulting and Advising Relationships: A perfect score requires prohibition or active discouragement of consulting or advising for purely commercial or marketing purposes.  In prior years, this domain required only elements demonstrating a legitimate contract, such as clear deliverables and fair market compensation.

Consulting and Advising Relationships perfect score 2014: 24, down from 71 in 2013.

  • Disclosure: A perfect score requires internal disclosure to the institution and external disclosure to trainees/audiences.  In prior years, this domain required internal disclosure on a publically-available website or disclosure to patients.

Disclosure perfect score 2014: 51, up from 41 in 2013

  • COI Curriculum: A perfect score requires a medical school curriculum that reflects and covers most of the curricular content and objectives in the AMSA standards for a “model curriculum.”  In prior years, this was measured more loosely.

COI Curriculum perfect score 2014: 34, down from 79 in 2013

Domains that are new this year:

  • Ghostwriting and honorary authorship: A perfect score requires prohibition of this activity.

Ghostwriting perfect score 2014: 105

  • Medical device representatives: A perfect score permits representatives in patient care areas only for legitimate reasons not related to marketing, such as providing necessary technical assistance and/or training on devices or other equipment already purchased.

Medical device representatives perfect score 2014: 91

  • Extension of COI policies to adjunct faculty and affiliated institutions: A perfect score requires the policies to apply to all employees of the institution, whether full/part-time or volunteer faculty and students/trainees.  The policy must apply to them regardless of the site they are working, even if the site itself does not follow the same policy.

Extension of COI policies perfect score 2014: 50

  • Enforcement and sanctions: This domain was present last year, but scored for the first time this year.  A perfect score requires general oversight to ensure compliance with policies and sanctions for noncompliance.

Enforcement and sanctions perfect score 2014: 126

The following is a streamlined list of all 2014 domains and the number of schools receiving perfect scores in each:

  • Gifts – 79
  • Meals – 24
  • Industry-funded promotional speaking relationships (not ACCME-accredited) – 79
  • Industry-support of ACCME-accredited CME – 5
  • Attendance of industry-sponsored promotional events – 25
  • Industry-supported scholarships and awards – 3
  • Ghostwriting and honorary authorship – 105
  • Consulting and advising relationships – 24
  • Access of pharmaceutical sales representatives – 9
  • Access of medical device representatives – 91
  • Conflict of interest disclosure – 51
  • COI curriculum – 34
  • Extension of COI policies to community affiliates – 50
  • Enforcement and Sanctions of Policies – 126

Methods

For full report see the Methodologies Section

Next Steps and Assistance:

The Scorecard serves not only to measure the strength of policies, but also to provide a valuable resource for institutions to develop and refine new policies. The inclusion of full text policies on the website (where permission has been given) will facilitate this goal.

  • To help medical students promote policy change, AMSA provides toolkits, templates, talks, and training institutes. AMSA is proud to join forces with three other national organizations (The Pew Charitable Trusts, National Physicians Alliance, and Community Catalyst) to create both external and internal pressure for medical schools and academic medical centers to adopt strong conflict-of-interest policies, through the Partnership to Advance Conflict-Free Medical Education (PACME).
  • The National Physicians Alliance (NPA) creates research and education programs that promote active engagement of physicians with their communities to achieve high quality, affordable health care for all. Their Unbranded Doctor campaign provides resources for conflict-free medical practice including an archive of past events including bimonthly Conflict-Free Leadership Calls and National Grand Rounds.
  • Community Catalyst is a national non-profit advocacy organization working to build the consumer and community leadership that is required to transform the American health system. Community Catalyst is in the process of developing a series of toolkits to help institutions improve their policies with examples of language from other model institutions and provides technical assistance to schools via regional roundtable and individual consultation.
  • Another resource is the Association of American Medical Colleges (AAMC) task force report on Industry Funding of Medical Education, PharmedOut, and the Institute of Medicine (IOM).
  • These and other resources may be found at www.pharmfree.org.