Meeting the Accreditation Council for Graduate Medical Education competencies using established residency training program assessment tools

Background: Most existing residency evaluation tools were constructed to evaluate the Accreditation Council for Graduate Medical Education (ACGME) competencies. Methods: Before ACGME's six competency based assessment requirements for resident performance were developed, we created a residency evaluation tool with 5 domains important to successful surgical resident performance. Reliability was determined after 6 months of use. Factor analysis assessed whether the evaluation tool was a construct-valid measure of the ACGME competencies. Results: Three hundred forty-three evaluations for 36 surgical residents were tested. The original evaluation tool was highly reliable with an overall reliability of 0.97. Factor analysis defined 4 new combinations of questions analogous to 4 of the ACGME competencies: professionalism (reliability 0.95), patient care (reliability 0.93), medical knowledge (reliability 0.92), and communication (reliability 0.92). The new competency clusters were correlated with each other to a moderate degree. Conclusions: Our locally developed tool demonstrated high reliability and construct validity for 4 of 6 ACGME competencies. The correlation between factors suggests overlap between competencies. (C) 2004 Excerpta Medica, Inc. All rights reserved.