Journal of Nature and Science, Vol.1, No.2, e39, 2015


Social Sciences

 

Resident Understanding of ACGME Core Competencies


Nitesh N. Paryani1, Jennifer L. Peterson1, Cheryll Albold2, Suzanne R Taylor2, Laura Vallow1, Stephen J. Ko1, Steven J. Buskirk1

 

1Department of Radiation Oncology and 2Graduate Medical Education at Mayo Clinic in Jacksonville, Florida, USA


Competency-based training and evaluation is a central requirement of all accredited residency training, yet there is little indication that it is sufficiently recognizable in the transition between undergraduate and graduate medical education. The purpose of this study was to assess applicants¡¯ knowledge of the ACGME¡¯s six core competencies, which guide the curriculum of all the ACGME accredited graduate medical education programs. Between November, 2011 and February, 2012, applicants to ACGME accredited residency programs at a single hospital completed a questionnaire asking them to list the six core competencies and articulate the words in the organization¡¯s acronym. Aggregated responses from 72 applicants were tabulated and recorded. Twenty-seven students (38%) were unable to name a single competency correctly.  Four students (6%) were able to name 5/6 core competencies.  On average, students were able to name fewer than two of the core competencies.  Students fared slightly better when asked to identify what the acronym ¡°ACGME¡± means.  Eight students (11%) were able to name all five components. Three students (4%) were unable to name a single component of the acronym ¡°ACGME.¡±  On average, students were able to name three parts of the acronym. Despite beginning their medical education two years after the widespread implementation of the ACGME core competencies, these graduating medical students had little recall and implied understanding of the core competencies used to evaluate them as residents.  There is a need for more collaboration and alignment between the nation¡¯s medical schools and the ACGME in terms of educating medical students on the increasing importance of competency-based training and evaluation in the medical profession. Journal of Nature and Science, 1(2):e39, 2015.

 

ACGME | core competencies | residency | education

 

In the United States (US), the Accreditation Council on Graduate Medical Education (ACGME) is arguably the most recognized organization involved in setting the standards for the educational experiences that are required of physicians in training to complete graduate medical education (GME) programs and enter independent practice.  As such, both allopathic and osteopathic medical school graduates aspire to enter into ACGME-accredited residency training programs. Although these eager applicants are presumed to have a baseline understanding of graduate and postgraduate physician training, ironically, the expanded name of the largest organization that accredits allopathic specialty and sub-specialty programs and the mandatory six curricular competencies which govern all ACGME residency programs are not easily recalled and articulated by graduating medical students who are also applicants to its accredited programs.

In July 2002, the ACGME and the American Board of Medical Specialties (ABMS) directed US medical residency programs to utilize six core educational competencies when evaluating the ability and proficiency of US medical residents.1  No uniform standards by which to assess resident competency existed prior to the release of these six ACGME core competencies.2 

The implementation of these core competencies was part of a three-step process, known as the Outcome Project.3  The first phase involved the initial drafting of the core competencies, followed by a testing period from 2002-2006 in which residency programs were required to incorporate these core competencies into their learning curricula and explore methods of evaluating residents according to these six competencies.  The ACGME expected that residency programs would have fully incorporated the ACGME core competencies into their training programs by 2006.4 

Medical student graduates of the class of 2012 began their medical education two years after widespread implementation of the ACGME core competencies. Presumably, their home institutions utilized these measures of evaluation with their residents, as well as integrated the competencies into the medical school curriculum. Nevertheless, based on informal surveys of medical students in past years and the limited published data available on this topic, we hypothesized that medical students had limited awareness of the ACGME and its core competencies.

 

Purpose of the study

The purpose of this study was to assess applicants¡¯ knowledge of the ACGME¡¯s six core competencies, which guide the curriculum of all the ACGME accredited graduate medical education programs.  As such, this study used a one-page questionnaire to:

 

1. Assess residency applicants¡¯ ability to accurately name the ACGME six core competencies.

2. Evaluate residency applicants¡¯ ability to recognize and articulate the components of the ACGME acronym, given the primary role of the organization in residency program accreditation and graduate medical education.

 

Study design

Institutional review board (IRB) approval was received to conduct the study.  Additionally, residency program directors provided permission to solicit participation from their candidates during the interview process.  The study participants were fourth year medical students who were applicants to various residency programs at the Mayo Clinic in Florida, an academic medical center.  The institution sponsors 11 campus-based ACGME accredited residency programs. The individual residency program coordinators administered the questionnaire to their program applicants, preferably at the beginning of the interview day.

Each participant received a cover letter describing the study and asking them to complete a one-page questionnaire.  The participants did not provide any identifying information, and the cover letter clearly stated that voluntary participation in the study would play no role in their application or interview process. Data collection was over a period of four months (November 2011 through February 2012). Seventy-two applicants out of 660 completed the confidential, anonymous survey (11%).

The quiz-style questionnaire anonymously assessed respondent knowledge of the six core competencies and ability to articulate the acronym components.  Participants were instructed to name the six ACGME core competencies, prompted by the first letter of each competency. The decision to give the first letter of each competency was made based on previous, informal assessments of residents across the institution, whom we found struggled and were less likely to name a single competency without prompting.  It was our hope that by providing the first letter of each competency, applicants would be more likely to fill out the survey. Additionally, participants were also asked to identify the meaning of the acronym ¡°ACGME,¡± again prompted by the first letter of each word.

Participants were encouraged to guess if they did not know what each of the letters stood for or to leave it blank.  To ensure anonymity, we collected the questionnaires in multiples. Responses were aggregated and embargoed until after the conclusion of the 2012 National Residency Matching Process in March. 

 

Table 1: The number of responses per competency

Core Competency

Respondents (%)

Professionalism

40 (56%)

Patient care

21 (29%)

Medical knowledge

16 (22%)

Systems-based practice

9 (13%)

Practice-based learning and improvement

6 (8%)

Interpersonal and communication skills

2 (3%)

Number of Correct Responses

Respondents (%)

None

27 (38%)

One

21 (29%)

Two

9 (13%)

Three

9 (13%)

Four

2 (3%)

Five

4 (6%)

 

Average: 1.3 correct per respondent

 

Results

A total of 72 questionnaires were completed, and the number of responses per competency was tabulated (Table 1). Twenty-seven students (38%) were unable to name a single competency correctly when provided with the first letter of each of the six competencies.  Four students (6%) were able to name five out of the six core competencies.  No participant was able to name all six.  The most commonly named core competency was ¡°professionalism,¡± with 56% (n=40) of students correctly identifying this competency.  ¡°Patient care¡± followed with 29% (n=21) correct identifications.  Only 3% (n=2) were able to successfully identify the competency of ¡°interpersonal and communication skills.¡±  On average, students were able to name less than two of the six core competencies even when provided with the first letter of each. 

Study participants (N=72) fared slightly better when asked to identify what the acronym ¡°ACGME¡± means, based on common knowledge that the acronym is derived from the first letter of each word in the organization¡¯s moniker.  Eight participants (11%) were able to correctly name all five components.  Sixty-nine participants (96%) were able to identify the words ¡°medical¡± and ¡°education,¡± and 64 (89%) were able to identify the word ¡°graduate.¡±  Sixteen (22%) students successfully identified the word ¡°accreditation,¡± and only 13 (18%) students could identify ¡°council.¡±  Survey participants commonly offered the words ¡°American¡± and ¡°college¡± as answer choices. Finally, three participants (4%) were unable to correctly name a single word of the acronym ¡°ACGME.¡±  On average, study participants were able to name three words of the acronym ¡°ACGME,¡± the large majority of whom identified the words ¡°Graduate Medical Education.¡± 

 

Discussion

Despite having begun their medical education two years after the widespread implementation of the ACGME core competencies, these graduating medical student applicants had difficulty recalling and articulating them.  An implication of our results is that these students have limited understanding of the core competencies that programs will use to evaluate them as residents. They also had limited awareness of the governing body that develops and monitors those standards.  We feel that if medical students entering residency do not recognize the role of the ACGME or understand the competencies by which they will be evaluated, it. is difficult to assess the efficacy, utility, and success of the Outcome Project

To our knowledge, this is only the second study to explore medical student understanding of the ACGME and the six core educational competencies. In 2008, St. Luke¡¯s-Roosevelt Hospital Center conducted a similar study. 5 In contrast to our questionnaire, the survey at St. Luke¡¯s-Roosevelt did not provide applicants with the first letter of each competency.  They also did not survey students on their understanding of the acronym ACGME.   A final notable difference is that our study included a sample of applicants interested in more than a single program as our study participants were applying to multiple specialties.

Nonetheless, the results of this study are congruent with findings from the study by Wasnick et al of applicants to the anesthesiology residency program at St Luke¡¯s-Roosevelt Hospital Center.5  In that study, the authors surveyed 193 residency applicants in the fall of 2008 to assess the applicants¡¯ knowledge of the core competencies.  They found that 39% of students were unable to identify a single core competency and that only 2% of students could correctly name all six.  In their group of students surveyed, they also found that ¡°professionalism¡± was the most widely identified competency, with 39% of students successfully identifying it.  The least commonly identified was ¡°systems-based practice,¡± with only 10% of students naming this competency.  When comparing these two studies, it is interesting to note that in both groups, students most commonly identified the core competency of ¡°professionalism.¡±  We conclude that this is likely due to an increased emphasis on teaching professionalism as part of integrated medical ethics courses that are becoming a standard component of the pre-clinical education. 

Both studies found that students had significant trouble identifying ¡°systems-based practice¡± and ¡°practice-based learning and improvement¡± as elements of the six core competencies, with approximately 10% of students in either group correctly identifying these competencies.  This may be due in part to the vague nature of these competencies, which tends to represent broader concepts rather than more straightforward competencies such as ¡°medical knowledge,¡± ¡°patient care,¡± and ¡°professionalism.¡±  A study of faculty members of the Uniformed Services Academy of Family Physicians (USAFP) also found a lack of awareness of these two competencies by attending faculty.2 

The current study found that students had the most difficulty with identifying the ¡°interpersonal and communication skills¡± as a core competency.  This contrasts with the St-Luke¡¯s-Roosevelt study in which 15% of respondents identified communication as one of the six.  This may be due to the fact that we did provide applicants with the first letter of each competency, and they may not have recognized the communication skill as beginning with an ¡°I.¡±  It is possible that more students may have known that communication skills were part of the ACGME core competencies but perhaps not the specifics of interpersonal communications.

Our study is subject to several limitations.  The sample size was small, owing to the fact that we have relatively few residency programs at our hospital and proportionately  interview a small number of applicants per year.  Additionally, participation in the survey was completely voluntary, and therefore not all applicants participated.  Finally, applicants may have been focused on their pending interviews. 

We did not assess the nature or amount of previous exposure that each participant had to the ACGME and competency-based education.  It is also possible that participants are more knowledgeable than our results imply, but simply were not able to recall the details, especially if they were also anxious about their upcoming interviews..  Conversely, by giving students the first letter of each competency, we may have artificially inflated the number of correct answers given by respondents.  Additionally, with a high number of non-respondents and no penalty for non-response, it is possible that our study selected those applicants with the strongest knowledge of the core competencies.  In order to protect the anonymity and privacy of the applicants, we did not collect any demographic information on responders to compare to that of non-responders; this further limits our interpretation of the data and may have resulted in our sample consisting of applicants with more knowledge about the ACGME and its core competencies.


   Future studies should use our platform to investigate the depth of exposure that medical students receive to ACGME and the core competencies and whether an in-depth understanding of the core competencies is something that would improve medical students¡¯ expectations of the residency training process. Tracking this kind of information on a prospective basis may provide evidence regarding the value of emphasizing the competencies in helping to build concordance between undergraduate and graduate training.

 

Study Conclusions

In summary, our study participants were not able to easily recall and articulate the ACGME core competencies, despite the important role they play in undergraduate and graduate medical education.  Given that our results were congruent with that of a noted past study performed in a similar setting, clearly there is room for increased collaboration and alignment between the nation¡¯s medical schools and the ACGME in terms of educating medical students on the increasing importance of competency-based training and evaluation in the medical profession. Consideration should be given to incorporation of competency-based training, and education about the ACGME core competencies, as part of the medical school curriculum.

 

Acknowledgements

The authors would like to thank Kelly Viola, BS, ELS and Victoria L. Jackson, MLIS in the Department of Academic and Research Support for their help with the preparation of the manuscript.

 


 



__________

Conflict of interest: No conflicts declared.

* Correspondence author: Steven J Buskirk, MD, Chair, Department of Radiation Oncology, Mayo Clinic, 4500 San Pablo Road South, Jacksonville, FL 32224£¬USA. Tel: 904-953-1000; Fax: 904-953-1010

E-mail: Buskirk.steven@mayo.edu

© 2015 by the Journal of Nature and Science (JNSCI).