AMSA Pharm-free Scorecard 2010

Methodology


I. Methods

II. Rating system

III. Determination of letter grade

IV. Reassessment

V. Assessment Instrument




I. Methods


Each year, all U.S. schools of allopathic and osteopathic medicine are invited to submit policies to the PharmFree Scorecard for assessment. Emails are sent to appropriate contacts at these schools (deans, conflict of interest officers, etc as established by previous year's contact at each school) explaining the Scorecard project and requesting the submission of their policies. The email also contains a check-list identifying relevant policy domains and requesting permission to make policies public, as well as a submission guidance document.

Schools that do not submit policies after mulltiple attempts at contact are assumed to be unchanged from the prior year and retain an unchanged assessment. Schools that decline to participate after multiple attempts at follow-up by repeated email and phone calls receive a grade of F. Any institution submitting draft policies or a formal notification that its policies are currently under review receives a grade of In Process (I), which may stand for up to one year.  After a year of being In Process, if a school does not submit either policies for evaluation or a description of their progress in developing such policies, they receive a grade of F.

Schools that have been newly founded receive a two-year grace period.  This starts from the matriculation of the inaugular class, during which we designate them as In Process (I).  This two-year period coincides with the first two years of medical school, which classically take place in the classroom, rather in the hospitals and clinics.  If there are no policies submitted to us or a description of progress in developing such policies by year three, they receive a grade of F.


II. Rating system


Policies were rated across a range of domains broadly consistent with those identified in Brennan et al. “Health industry practices that create conflicts of interest: A policy proposal for academic medical centers” (JAMA 2006; 295:429-433) and The Prescription Project “Time for Change” 2007 (policy brief). The formal grading and scoring systems appear below.

Each policy was graded by two independent assessors, blinded to the institution of origin.  Any differences in scoring between the two assessors were resolved by a consensus process. The assessors received formal training in the use of the scoring system, independently evaluating and coming to a consensus on five training policies before beginning to evaluate the medical school policies.

This scoring instrument focuses on conflict-of-interest policies directly related to industry marketing and education. While not addressed here, academic medical centers should also have robust policies to ensure the integrity of basic and clinical research, including policies to ensure that academic researchers working in collaboration with industry retain full control of study design, data, analysis and writing.

Policies are rated on each of the domains listed below, using the following general format.

3 = Model policy
2 = Good progress toward model policy
1 = Policy is absent or unlikely to have a substantial effect on behavior
0 = Institutions that do not respond to requests for policies or decline to participate will receive scores of zero in every category.
In exceptional circumstances, a domain may be categorized as Not Applicable and omitted from the scoring.

Policies are rated on how closely they approach the goals, recognizing that there may be many approaches to a single goal. Examples in parentheses are illustrative only.  

Industry refers to the pharmaceutical and medical device industries.

 

Policy Domains:


1. Gifts and individual financial relationships with industry
1A. Gifts (including meals)
1B. Consulting relationships (excluding scientific research and speaking)
1C. Industry-funded speaking relationships
1D. Disclosure
2. Pharmaceutical Samples
3. Purchasing & Formularies
4. Industry Sales Representatives
5. Education
5A. On-site Educational Activities
5B. Compensation for Travel or Attendance at Off-site Lectures & Meetings
5C. Industry Support for Scholarships & Funds for Trainees
5D. Medical school curriculum
6. Enforcement


1. Gifts and individual financial relationships with industry


1A. Gifts (including meals)


Background: Numerous published studies demonstrate that small and large gifts play a role in influencing prescribing decisions, which directly affect patients. Medical personnel consistently underestimate the extent to which they personally are influenced. Industry-sponsored meals are a form of gifting.

3 = All gifts and on-site meals funded by industry are prohibited, regardless of nature or value.

2 = Less stringent limitation on industry-funded gifts (e.g., gifts prohibited above $50/year – or gifts prohibited but meals allowed)

1 = No policy, or policy that would not substantially reduce gifting (e.g., gifts are allowed but discouraged, or limited in a non-specific way to “appropriate,” or primarily for the benefit of patients).


1B. Consulting relationships (excluding scientific research and speaking)


3 = Consulting relationships with industry must be subjected to institutional review or approval. Additionally, they must either be described in a formal contract, or payment for services must be commensurate to the task.
 
2 = As above, without the institutional review or approval requirement.

1 = No policy, or policy that would allow consulting relationships to occur without institutional scrutiny or that would allow relationships in which payments are not commensurate with work.



1C. Industry-funded speaking relationships


Background: Research relationships with industry may entail beneficial public presentations and speeches by individual researchers. However, industry also uses academic physicians to support marketing goals by identifying and cultivating speakers who give a positive message about the drug in question. Such ongoing relationships, sometimes called “speakers bureaus,” are unnecessary and detrimental.

3 = Speaking relationships are prevented from functioning as de facto gifts or marketing. An effective policy must not implicitly permit (a) long-term speaking agreements or (b) industry to have a role in determining presentation content. (Some effective policies may explicitly prohibit participation in a speakers bureau. Other effective policies contain elements such as limits on compensation and reimbursement and a requirement to ensure the scientific integrity of information presented.)
 
2 = Industry-funded speaking relationships are regulated, but with less stringent limits on longevity, content or compensation.

1 = No policy, or policy that does not define the limits on longevity, content or compensation.



1D. Disclosure


3 = Personnel are required to disclose past and present financial ties with industry (e.g., consulting and speaking agreements, research grants) on a publicly-available website and/or disclose such relationships to patients when such a relationship might represent an apparent conflict of interest.

2 = Universally-required, internal disclosure to the medical school or hospital administration. (Policies requiring disclosure only when presenting or publishing do not meet this criterion.)

1 = No policy.
 


2. Pharmaceutical Samples


Background: The U.S. pharmaceutical industry distributes some $18 billion per year in drug samples. Published studies show that a substantial proportion of these samples are used by physicians, staff and their families. Such use is a clear financial conflict of interest that confers no possible benefit on patients.

When sample medications are accepted and dispensed in the clinic setting, the usual standards of inventory control, drug interaction and dosage screening, labeling and documentation may be bypassed (contravening Joint Commission standards for hospital accreditation). Distribution of non-formulary drug samples has the potential to undermine the intent and function of the formulary.
 
Furthermore, the distribution of samples has been shown to lead physicians to prescribe drugs that differ from their preferred drug choice, reducing their prescribing of unadvertised drugs in favor of advertised drugs and decreasing their use of first-line (relative to second-line) therapies.  This implies that the direct distribution of samples to physicians may, in aggregate, increase costs while reducing the safety and effectiveness of prescribing.

3 = Industry samples are prohibited, except under certain narrow circumstances approved by the institution that protect the interests of patients and prevent the use of samples as a marketing tool (e.g., policies that allow samples under limited circumstances with the approval of the Pharmacy and Therapeutics (P&T) Committee or policies that incorporate samples into a larger program designed to ensure the availability of brand-name and generic medications to under-insured patients; if the circumstances of the specific program are not defined, the policy should define the approvals process). Where there is a specific program in place, the policy must prevent samples from being given directly to physicians by pharmaceutical sales representatives.

2 = Samples or vouchers for medications may be provided, but with significant limitations (e.g., samples may not be given directly to physicians, samples must be dispensed or controlled by the pharmacy department).

1 = No policy, or a policy that does not substantially limit the use of samples (e.g., samples limited to formulary items, or samples not for personal use).



3. Purchasing & Formularies


Background: Individuals with financial conflicts of interest should not make institutional purchasing decisions. Decisions influenced by personal conflicts have the potential to adversely affect institutional costs and the quality of patient care. Pharmacy and Therapeutics (P&T) Committees typically decide which drugs will be on the hospital’s “preferred list,” known as a formulary. Other committees may make other purchasing decisions.

3 = Formulary committees and committees overseeing purchases of medical devices should exclude those who have financial relationships with drug or device manufacturers. Exclusion may be specific to participation in particular decisions for which the staff member has a conflict of interest. This policy does not prevent expert clinicians from advising a committee, provided that potential conflicts are disclosed.  (Note: this standard is not intended to prohibit indirect financial interests, such as investments in mutual funds that may own pharmaceutical company shares).

2 = Less stringent policies that do not prohibit individuals with conflicts from participating in purchasing decisions (e.g., policies that require members of committees overseeing purchases merely to disclose potential conflicts of interest).

1 = No policy, or policy that merely cautions against conflicts of interest.



4. Industry Sales Representatives


Background: Industry sales representatives are employed to increase the sales of their company’s drugs. Permitting their access to medical staff is not in the interests of patients or staff.
 
3 = Pharmaceutical and device representatives are not allowed to meet with faculty regardless of location, or are not permitted to market their products anywhere inside the medical center and associated clinics and offices. (Exceptions may be made for non-marketing purposes, such as training on devices or equipment.)
 
2 = Pharmaceutical representatives are permitted to meet with faculty, but with significant limitations (e.g., only in non-patient care areas or only by appointment). Exceptions as above.
 
1 = No policy, or policy that does not substantially limit access.



5. Education


Background: It is essential that financial support not influence the content of educational activities. Where financial support from industry assists in the delivery of educational activities, it must not be linked to an individual company’s interest in promoting specific products. Therefore, a firewall should separate the donor from those developing the educational activity. 

Educational activities take place both “on-site” (that is, within the medical school or hospital campus) and “off-site” (at outside facilities, including professional conferences). Many policies distinguish between on-site and off-site activities.


5A. On-site Educational Activities


3 = Industry is not permitted to provide direct financial support for educational activities, including Continuing Medical Education (CME), directly or through a subsidiary agency.  (However, companies may contribute unrestricted funds to a central fund or oversight body at the academic medical center, which, in turn, would pool and disburse funds for programs that are independent of any industry input or control.)

2 = Less stringent limitations to ensure independence of educational content (e.g., standards to establish freedom from industry influence of content, such as review and approval of presentations; language that prevents industry from selecting the speaker; a requirement that programs adhere to ACCME* standards; or language such as: industry funding may be allocated for a particular topic, but must be provided directly to the department, not to individuals). *Note: ACCME is a non-governmental oversight organization that establishes accreditation standards for CME activities.

1 = No policy, or a policy that would not substantially limit industry influence over educational activities (e.g., industry funding must be disclosed).
 


5B. Compensation for Travel or Attendance at Off-site Lectures & Meetings


3 = Personnel may not accept payment, gifts or financial support from industry to attend lectures and meetings. (An exception may be made for modest meals, if part of a larger program.) Travel support may only be accepted if it is subject to institutional approval or industry is prevented from selecting (“earmarking”) the recipients. Note: speaking and consulting relationships are evaluated separately in domain 1.

2 = Less stringent limitations.

1 = No policy, or a policy that would not substantially limit participation in industry-funded events and meetings.



5C. Industry Support for Scholarships & Funds for Trainees


3 = The policy must either prevent industry from earmarking or awarding funds to support the training of particular individuals (recipients must be chosen by the school or department), or the policy must mandate institutional review of the giving of funds. (This does not preclude grants that fund a specific research project.)

2 = Less stringent limitations.

1 = No policy, or a policy that would not substantially regulate industry funding of scholarships and funds for trainees.



5D. Medical school curriculum (or other documentation of educational objectives/course content)


3 = Students are trained to understand institutional conflict-of-interest policies and recognize how industry promotion can influence clinical judgment.

2 = Curriculum addresses conflict of interest in a more limited way (e.g., training on policies only).

1 = No policy (not addressed in curriculum or elsewhere).



6. Enforcement


A. Is it clear that there is a party responsible for general oversight to ensure compliance?  (Y/N)        

B. Is it clear there are sanctions for noncompliance?  (Y/N) 

 
 

III. Determination of Letter Grade


Scores for domains 1 and 5 are calculated as the average of the best three sub-domain scores. Domain 6 (Enforcement) is not incorporated into the final score.

Final score is calculated as the total cumulative score for domains 1 through 5 divided by total possible score x 100.  (Total possible score = 15, unless a domain is classified as NA.) Grades are assigned as follows:

A ≥ 85%
B ≥ 70%
C ≥ 60%
D ≥ 40%
F <  40%
I = In Process

C* is assigned in two instances: (1) Policies that apply only to medical students and/or residents, but not faculty, may achieve a maximum grade of C*; (2) Policies that are only guidelines, without formal requirements, may achieve a maximum grade of C*.

An In Process grade will be assigned to any institution that submits a formal notification that its policies are currently under revision. This grade may stand for up to one year.

Non-reporting institutions will receive a grade of F.


 

IV. Reassessment


An institution may request an explanation or reassessment of its score within 30 days of the scorecard release if policies change or if it feels it has been scored inappropriately.  The reassessment process requires that a school submit either (a) new or additional policies or (b) a written explanation of why the current assessment is incorrect, specifically addressing each relevant domain and with reference to the grading methodology.  During reassessments, the grade will be changed to In Process and the reassessment completed within 60 days.  Contact scorecard@amsa.org for more details.



 

V. Assessment Instrument

 
The rating system discussed above was translated into a flowchart of guided quesions to facilitate qualitative assessment.


1A: GIFTS


1. Is there any part (or parts) of the policy that applies to gifts and/or on-site meals?  Y:2 N:8

2. Are all gifts and on-site meals prohibited, regardless of nature?  Y:6 N:3

3. Does the policy prohibit only one element (gifts or on-site meals) completely?  Y:7 N:4

4. Are gifts and/or on-site meals restricted by exclusive reliance on the AMA or 2002 PhRMA guidelines?  Y:8 N:5

5. Is there any policy language you think would substantially reduce gifting or on-site industry-sponsored meals?  (e.g. a dollar-value limit on gifts or other restrictions that go beyond PhRMA guidelines)  Y:7 N:8

6. This policy scores a "3" 

7. This policy scores a "2" 

8. This policy scores a "1" 

 

1B: CONSULTING


1. Has the institution provided any part (or parts) of the policy that applies to physicians or medical school faculty who serve as consultants to the pharmaceutical industry?  Y:2 N:8

2. Is there a requirement for institutional review or approval of consulting relationships between AMC personnel and industry?  Y:3 N:5

3. Are consulting relationships required to be described in a formal contract?  Y:6 N:4

4. Is there a provision to ensure that payment is appropriate or commensurate to the task? Y:6 N:7

5. Is there EITHER a provision for a formal contract, OR language to ensure that payment is commensurate/appropriate to the services rendered, or both?  Y:7 N:8

6. This policy scores a "3"

7. This policy scores a "2"

8. This policy scores a "1"

  

1C: SPEAKING


1. Has the institution provided a part (or parts) of the policy that applies to AMC personnel who receive payments from industry for external speaking engagements? (NB: Policies solely governing on-site presentations/events should be scored under “On Site Education)  Y:2 N:8

2. Does this policy explicitly prohibit participation in “speaker’s bureaus”?  Y:4 N:3

3. Is this policy written in a way that would prevent long-term speaking relationships?  Y:4 N:5

4. Does the policy prevent industry from controlling the presentation content? (NB: this should be applicable to all speaking arrangements between AMC personnel and industry, not just on-site presentations.)  Y:6 N:7

5. Does the AMC establish any specific limits on, or institutional review of, speaking relationships?  Y:7 N:8

6. This policy scores a "3"

7. This policy scores a "2"

8. This policy scores a "1"

 

1D: DISCLOSURE


1. Has the institution provided a part (or parts) of the policy that applies to disclosure of financial ties with the pharmaceutical industry?  Y:2 N:7

2. Does the policy/policies mandate public disclosure of all financial ties between physicians and/or medical school faculty and industry?  Y:5 N:3

3. Does the policy/policies require personnel to disclose financial ties to their patients, only when that relationship might represent an apparent conflict of interest?  Y:5 N:4

4. Does the language require any other form of EITHER internal OR external disclosure that all personnel (or all personnel with conflicts) are required to file on a periodic basis?  Y:6 N:7

5. This policy scores a "3"

6. This policy scores a "2"

7. This policy scores a "1"

  

2: SAMPLES


1. Has the institution provided a part (or parts) of the policy that applies to pharmaceutical samples?   Y:2 N:10

2. Are samples completely prohibited from the AMC, or are faculty clearly prohibited from distributing samples (including indirect distribution -- e.g., through a pharmacy)?   Y:8 N:3

3. Does the policy explicitly prevent samples from being given directly to physicians by a pharmaceutical sales representative?   Y:4 N:6

4. Does the policy require that the AMC must specially approve any proposed program that would allow samples in certain narrow circumstances? (If the circumstances are not defined, the policy should define the approvals process.)   Y:8 N:5

5. Does the policy effectively remove the advantage of marketed brand-name samples by also providing samples of generic medication?   Y:8 N:9

6. Does the policy mandate that samples be dispensed or controlled from the pharmacy?   Y:9 N:7

7. Does the policy create any other restrictions that you think would substantially limit the use of samples as a marketing tool (whether or not patient access to samples is limited)?   Y:9 N:10

8. This policy scores a "3"

9. This policy scores a "2"

10. This policy scores a "1"

  

3: PURCHASING / FORMULARIES


1. Has the institution provided a part (or parts) of the policy that applies to individuals making purchasing decisions? (Often as members of purchasing or formulary committees, sometimes known as P&T)   Y:2 N:8

2. Are committee members with financial conflicts of interest forbidden from serving on the committee?  Y:6 N:3

3. Are committee members with financial relationships with a drug or device company required to recuse themselves from all committee decisions involving that same company?  Y:6 N:4

4. Are committee members or heads required to disclose their financial conflicts of interest?  Y:7 N:5

5. Does the policy make any other meaningful attempt to mitigate the effects of conflict of interest on those making purchasing decisions? (Simple cautionary language does not count)  Y:7 N:8

6. This policy scores a "3"

7. This policy scores a "2"

8. This policy scores a "1"


 

4: INDUSTRY SALES REPRESENTATIVES


1. Has the institution provided a part (or parts) of the policy that regulates the ability of pharmaceutical sales representatives to enter the medical school or medical center or interact with faculty?   Y:2 N:8

2. Are industry sales representatives prohibited from entering the medical center, or are faculty prohibited from meeting with sales representatives? (The only allowable exception is for training on devices or equipment)  Y:6 N:3

3. Are industry sales representatives allowed to meet with faculty, but only by appointment or invitation?   Y:7 N:4

4. Are industry sales representatives allowed to meet with faculty, but only in non-patient care areas?  Y:7 N:5

5. Is there any other policy language that you think would substantially limit industry sales rep interaction with faculty[GS1] ?   Y:7 N:8

6. This policy scores a "3"

7. This policy scores a "2"

8. This policy scores a "1"
  

5A: ON SITE EDUCATION


1. Has the institution provided a part (or parts) of the policy that applies to on-site educational activities?  Y:2 N:11

2. Does the policy prohibit financial support by industry of all educational activities (including CME)?  Y:9 N:3

3. Does the policy mandate that all contributions be made to a central fund at the institutional (not departmental) level?  Y:4 N:5

4. Does the policy clearly state that all funds will be independent of industry control?  Y:9 N:10

5. Does the policy require institutional review or approval of educational activities?  Y:10 N:6

6. Does the policy cite adherence to ACCME standards as a benchmark for all on-site educational activities (regardless of whether the program is accredited CME)?  Y:10 N:7

7. Does the policy contain any limitations on industry-control of educational activity content or topic? (Such as direct prohibition, or oversight, review, or approval of presentations?)  Y:10 N:8

8. Does the policy effectively prevent industry from selecting the speaker?  Y:10 N:11

9. This policy scores a "3"

10. This policy scores a "2"

11. This policy scores a "1"

 

5B: OFF SITE EDUCATION: Support for travel/attendance


1. Has the institution provided a part (or parts) of the policy that applies to gifts, compensation, or travel for off-site events?  Y:2 N:9

2. Does the policy prohibit the receipt of payment for attendance at off-site lectures and meetings? (NB: Policy should be scored under this domain only as it applies to off-site event attendees, not speakers)  Y:3 N:5

3. Does the policy prohibit all financial support by industry for travel to off-site lectures and meetings?  Y:7 N:4

4. Does the policy require all travel support be subjected to institutional approval, OR does the policy prevent industry from selecting the recipients of travel support/meeting attendees?  Y:7 N:8

5. Are there any specific limitations (financial or non) on gifts, compensation or travel for off-site event attendance?  Y:8 N:6

6. Are there any other parts of the policy that you think would substantially limit participation in industry-funded off-site events?   Y:8 N:9

7. This policy scores a "3"

8. This policy scores a "2"

9. This policy scores a "1"

 

5C: SCHOLARSHIPS / TRAINEE FUNDS


1. Has the institution provided a part (or parts) of the policy that applies to industry support for scholarships and funds for trainees? (NB: policies solely governing speaking or consulting relationships should be scored under those respective domains)  Y:2 N:7

2. Does the policy prevent industry from choosing (“earmarking”) the individual recipient of the funds?  Y:5 N:3

3. Does the policy establish oversight or review of the giving of funds?  Y:5 N:4

4. Are there any other less stringent limitations on industry funding of scholarships?  Y:6 N:7

5. This policy scores a "3"

6. This policy scores a "2"

7. This policy scores a "1"

  

5D: CURRICULUM


1. Does the policy show that the medical school’s curriculum or hospital’s training materials promote understanding of pharmaceutical marketing, financial conflicts of interest, and its effect on health care decision-making by physicians?  Y:3 N:2

2. Does the curriculum or curriculum policy cover industry marketing and conflict of interest in a more limited way? (i.e. does it only describe the institution’s policy without going further into why and how industry can create conflict of interest for medical professionals?)  Y:4 N:5

3. This policy scores a "3"

4. This policy scores a "2"

5. This policy scores a "1"


 

6A: OVERSIGHT


1. Is it clear that there is a party responsible for general oversight to ensure compliance?  (e.g. the committee or individual to whom violations should be reported)  Y/N

 

6B: SANCTIONS


1. Is it clear that there are sanctions for noncompliance? (Description of sanctions not required)  Y/N


Suggested Citation: AMSA PharmFree Scorecard 2011-12. <<Title of Web Page>> (online), <<URL of Specific Web Page>>, Reston, VA:  American Medical Student Association <<Date Accessed>>.