Frazer A, Tanzer M. Hanging up the surgical cap: Assessing the competence of aging surgeons. World J Orthop 2021; 12(4): 234-245 [PMID: 33959487 DOI: 10.5312/wjo.v12.i4.234]
Corresponding Author of This Article
Michael Tanzer, FRCS, MD, Doctor, Full Professor, Surgeon, Department of Orthopaedic Surgery, McGill University, 1650 Cedar Ave B5159, Montreal H3G 1A4, QC, Canada. michael.tanzer@mcgill.ca
Research Domain of This Article
Surgery
Article-Type of This Article
Systematic Reviews
Open-Access Policy of This Article
This article is an open-access article which was selected by an in-house editor and fully peer-reviewed by external reviewers. It is distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Author contributions: Both authors made substantial contributions to conception and design of the study, acquisition of data, or analysis and interpretation of data.
Conflict-of-interest statement: There is no conflict to disclose.
PRISMA 2009 Checklist statement: The authors have read the PRISMA 2009 Checklist, and the manuscript was prepared and revised according to the PRISMA 2009 Checklist.
Open-Access: This article is an open-access article that was selected by an in-house editor and fully peer-reviewed by external reviewers. It is distributed in accordance with the Creative Commons Attribution NonCommercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/Licenses/by-nc/4.0/
Corresponding author: Michael Tanzer, FRCS, MD, Doctor, Full Professor, Surgeon, Department of Orthopaedic Surgery, McGill University, 1650 Cedar Ave B5159, Montreal H3G 1A4, QC, Canada. michael.tanzer@mcgill.ca
Received: December 24, 2020 Peer-review started: December 25, 2020 First decision: January 18, 2021 Revised: January 28, 2021 Accepted: April 5, 2021 Article in press: April 5, 2021 Published online: April 18, 2021 Processing time: 108 Days and 18.5 Hours
Abstract
BACKGROUND
As the average age of surgeons continues to rise, determining when a surgeon should retire is an important public safety concern.
AIM
To investigate strategies used to determine competency in the industrial workplace that could be transferrable in the assessment of aging surgeons and to identify existing competency assessments of practicing surgeons.
METHODS
We searched websites describing non-medical professions within the United States where cognitive and physical competency are necessary for public safety. The mandatory age and certification process, including cognitive and physical requirements, were reported for each profession. Methods for determining surgical competency currently in use, and those existing in the literature, were also identified.
RESULTS
Four non-medical professions requiring mental and physical aptitude that involve public safety and have mandatory testing and/or retirement were identified: Airline pilots, air traffic controllers, firefighters, and United States State Judges. Nine late career practitioner policies designed to evaluate the ageing physician, including surgeons, were described. Six of these policies included subjective performance testing, 4 using peer assessment and 2 using dexterity testing. Six objective testing methods for evaluation of surgeon technical skill were identified in the literature. All were validated for surgical trainees. Only Objective Structured Assessment of Technical Skills (OSATS) was capable of distinguishing between surgeons of different skill level and showing a relationship between skill level and post-operative outcomes.
CONCLUSION
A surgeon should not be forced to hang up his/her surgical cap at a predetermined age, but should be able to practice for as long as his/her surgical skills are objectively maintained at the appropriate level of competency. The strategy of using skill-based simulations in evaluating non-medical professionals can be similarly used as part of the assessment of the ageing surgeons’ surgical competency, showing who may require remediation or retirement.
Core Tip: A surgeon should not be forced to hang up his/her surgical cap at a predetermined age, but should be able to practice for as long as his/her surgical skills are objectively maintained at the appropriate level of competency. The strategy of using skill-based simulations in evaluating non-medical professionals can be similarly used as part of the assessment of the ageing surgeons’ surgical competency, showing who may require remediation or retirement.
Citation: Frazer A, Tanzer M. Hanging up the surgical cap: Assessing the competence of aging surgeons. World J Orthop 2021; 12(4): 234-245
The aging surgeon has remained a contentious patient safety issue, as the average age of surgeons continues to rise. The number of physicians exceeding the age of 65 has more than quadrupled in almost 40 years, and surgeons are no exception[1]. Currently, 38% of surgeons working in Canada and over 46% of the surgeons in the United States are above the age of 55 and 16% of Canadian surgeons are over the age of 65[2-4]. When should an older surgeon stop operating? This becomes an important issue when we consider that the surgical profession is highly dependent on memory, sensory acuity, clinical decisiveness, technical skills and physical stamina; skills and abilities that may decrease with age.
Like all professionals, surgeons are fallible and vulnerable to the natural process of aging in which cognitive and physical skills experience a decline. The relationship between surgeon age and operative risk is controversial and uncertain, with some studies showing worse outcomes and some showing reasonable, or even better outcomes[3,5-7]. Even so, there are accounts of prominent older surgeons struggling with simple fine motor skills and clinical decision making; while various studies have demonstrated that senior surgeons have worse outcomes than their younger colleagues, suggesting surgeon age is an operative risk factor[2,8-12]. In the past two decades, more research has been dedicated to studying the aging surgeon and trying to find more effective ways to help surgeons experiencing age-related decline[8,13-15].
Although age has been identified as a possible occupation hazard in the surgical profession[9,10], other professions within the public sector have maintained a more proactive approach with their aging workforce. These strategies include enforcing strict certification and mandatory retirement. Currently, retaining certification in a surgical field is not as rigorous a process and retirement of surgeons remains largely at their discretion, relying on the assumption that they are fully capable of identifying their own cognitive and physical regression[1,2]. Both the American College of Surgeons (ACS) and American Medical Association (AMA) recognize the safety implications of the aging physician and surgeon; and have recommended practitioners aged 65 to 70 undergo voluntary physical examination and visual testing, in addition to peer-reviewed performance evaluation for re-credentialing[1,2]. Programs have been developed to assess the competence of aging surgeons and guide them depending on their incapacities or capacities[16,17]. However, no standardized approach to address age-related deterioration in surgeons exists at this time.
In order to address this issue, we performed a scoping review of well-established and accepted policies of testing for competency of aging non-medical professionals from the industrial workplace. In particular we sought to identify strategies used to determine competency that could be transferrable in the assessment of aging surgeons. In addition, we reviewed the present testing of senior surgeons by hospitals to determine if the testing specifically addresses the skills, other than knowledge, required to maintain their competency to perform surgery.
MATERIALS AND METHODS
In order to identify mandatory testing of non-medical professions, where cognitive and physical competency are necessary for public safety, we conducted scoping review of the literature in PubMed, EMBASE, Google Scholar and Google using the terms mandatory retirement, mandatory retirement testing, retirement, mandatory retirement ages, surgeon retirement. The Google search engine was used to locate websites describing these professions within the United States. The mandatory age and certification process, including cognitive and physical requirements, were reported for each profession.
Google Scholar and the Google search engines were used to search for current policies, at health institutions in the United States, that are being used to evaluate older physicians and surgeons with regard to their ability to provide safe and high-quality care. The search queries used were: Late career practitioner policy, late career practitioner, assessing late career practitioners, aging physician policy, and aging surgeon policy. These policies were described based on their objectives, candidates, examination procedures, and modification of privileges.
Methods for determining surgical competency currently in use or practice as well as those existing in the literature were identified. Licensing bodies and subspecialist certification boards were described, along with their use of Continuing Medical Education (CME) and Maintenance of Certification (MOC) as a means of assessment. These agencies were selected as they are invested in ensuring the clinical competency of practicing physicians for the welfare of the general public[18,19]. The published literature was reviewed for objective assessments of surgical technical skills that were shown to be valid and reliable, as well as predictive of surgeon competency and improved patient outcomes for surgeons. Studies were identified through a search of databases, including PubMed, EMBASE, and Google Scholar. Search queries were developed from the following keywords: Technical skill, surgery, assessment, surgical training, surgical skill. Objective assessment tools identified in the database search were described if studies evaluating these tools were written in English, evaluated participants performing surgical tasks, and analyzed the validity and reliability of the objective assessment. Review articles found were used to identify studies assessing the reliability and validity of the objective assessment tools. Virtual reality simulators were excluded as assessment tools as they are often used primarily to determine the validity and reliability of the performance tools in mock surgical procedures, rather than for assessment of surgical skills. All search queries were completed by May 20, 2020.
RESULTS
Non-medical professionals
Four non-medical professions requiring mental and physical aptitude that involve public safety and have mandatory testing and/or retirement were identified: Airline pilots, air traffic controllers, firefighters, and United States State Judges (Table 1). The nature of their work is such that any condition which compromises their competency, such as knowledge; cognition; illness or physical stamina; can put many members of the public at risk. As a result, these professions have mandatory retirement at specific ages, as well as strict recertification and licensing programs. A mandatory retirement age, which is different for each occupation, is enforced in all 4 professions. An annual medical exam is required for 3 of these professions; a skills refresher course and assessment in two of the professions; and a physical ability test in one to maintain the professional’s ability to continue working. These skill assessments, knowledge or physical, are designed to reflect the competency and proficiency standards of their profession. They entail simulated situations one may expect to encounter as part of their profession.
Table 1 Surgeon certification compared to non-medical professions.
Occupation
Mandatory retirement age (yr)
Certification based on cognitive and/or physical requirements
Complete residency and board examination. Continuous self-learning or medical education (CME) for credit with periodic examination for certification. May have case list peer-reviewed and be evaluated by in-hospital staff. No physical exam or performance-based skills currently defined
Every 24 mo, complete flight review with instructor: Ground/flight training. Must demonstrate proficiency, competency, and sound judgement within approved standards during training. Medical certificate every 12 mo or every 6 mo if age > 40 yr
Apply at < 31 yr. Medical exam. Pass biographical assessment, pass the ATSA, and pass training course at FAA academy. The ATSA is designed to measure the cognitive, visuospatial reasoning, and psychomotor abilities of candidates. Must submit to yearly physical exam, and job performance twice a year, with periodic drug screenings. Certificate valid until surrendered, suspended, or revoked
Complete CPAT. Complete annual medical exam and physical testing, depending on department discretion. Physical testing consists of exercises related to firefighting such as step tests, and leg lifts
Obtained law degree and passed Bar examination. State court often requires mandatory retirement for state court judges
Late career practitioner policies
There are currently no mandatory retirement criteria for all surgeons in North America. We identified 9 late career practitioner policies designed to evaluate the ageing physician, including surgeons. One of the policies, the Aging Surgeon Program at Sinai Hospital, specifically targets the aging surgeons, but is identical to the Hospital’s evaluation of medical doctors, and has no specific testing of surgical skills (Table 2). All nine of these policies vary according to their trigger for assessment, the assessments being utilized (medical exam and performance testing), and their influence on hospital privilege modification. Eight of the policies use age to identify physicians that require cognitive and physical testing, starting typically at the age of 70 years, with repeated testing required every 1 or 2 years. All 9 physician monitoring programs have a medical exam component, focusing on general health, cognition, vision, and hearing testing, although the evaluation process does vary from program to program. Performance testing is part of the evaluation in 6 of the health systems and is done by peer assessment in 4, and only done by dexterity testing in 2. In the peer review assessment, two policies address the technical and procedural competencies required by surgeons to safely and effectively perform surgery. The evaluation of these technical competencies is subjective in nature, where candidates are rated on scale from “significant concern” to “outstanding” clinical competence. Observed performance or behavior that influenced the peer assessment must be commented if labeled “significant concern” or “minor concern”. The results of the testing could be used to modify the physician’s privileges in 7 of the 9 health systems identified.
Table 2 Late career practitioner policies adopted at health institutions.
To ensure high quality care for patients and protect them harm and identify health concerns of practitioners
74.5 years old and every 2 yr thereafter
Comprehensive history and physical examination including vision, hearing, neurological, and cognitive testing
Peer assessment by hospital staff of technical and procedural competencies, relative to Stanford expectations
Yes, if non-compliant or unsafe practice patterns
Aging Surgeon Program at Sinai Hospital, Baltimore, MD[43]
Designed to protect patients from unsafe surgeons and guard surgeons from arbitrary or unreliable methods of assessing competence or cognitive capacity. The program can identify potentially treatable or reversible disorders that, if properly treated, could restore or improve functional capacity
No mandatory age. Requested from surgeons, hospitals or licensing bodies of all surgical sub-specialties
Evaluation of general health, vision, hearing, neurocognition, visual-spatial and fine motor capability
To assess physicians to ensure patient safety and physician wellness
70 years old and above, and every 2 yr thereafter
Physical capacity by occupational therapy and neuropsychological testing
Peer review assessment
Yes, determined by Credentials Committee, if health problems interfere with safe practice
Driscoll’s Children’s Hospital, Corpus Christi, TX[17]
To assure that patient safety and quality are adequately supported by carefully assessing the capabilities, competencies and health status of each practitioner
70 years old and above
Comprehensive examination addressing physical and mental capacity by a physician
Peer review assessment may be required. Must meet technical and procedural competencies
Yes, determined by Credentials committee, if practice unsafe or incompetent
To fairly and accurately evaluate physician performance and capabilities
70 years old and above. Required to partake in LCHS
See LCHS
See LCHS
Yes, after consultation with department Chair if adjustment is required
Competency assessments in the workplace
Licensing bodies and certification boards in the United States and Canada use CME and MOC as assessment methods to ensure continued clinical competency in the surgical workplace of board-certified surgeons (Table 3). However, these competency evaluations only address knowledge and do not take into account surgical skills, and to some degree judgment. State and provincial licensing bodies can initiate a competency evaluation by independent evaluator if a surgeon has been identified to the licensing body as performing below the accepted level of competency. Within the hospital setting, the surgeon-in-chief is primarily responsible to the community for the safety of the operating room and the competence of surgeons on the staff.
Table 3 Existing methods for determining surgical competency.
Mandatory to practice. Required to demonstrate competency through CME. However, states individually may evaluate professional conduct when a physician fails to provide appropriate quality of care
Must regularly participate in CME activities and may require board certification. May have competency evaluation by independent evaluator or approved assessment program if signs of dyscompetence
Voluntary certification to show knowledge of standards of practice. Rigorous process of evaluation every 10 yr with MOC
MOC consists of 4-part assessment: Licensure/professional standing, participation in CME programs, cognitive expertise through examination, and documentation of quality of care and/or audits or peer review
Mandatory to practice. Required to demonstrate competency through CME. Provincial licencing bodies identify those with deficiencies in competence, requiring peer review
Must regularly participate in CME activities. If evidence of dyscompetence, rigorous individualized assessment of the surgeon’s practice is performed, with emphasis on quality of care
Video recorded surgical performance and evaluated with validated with rating scale
Crowds of anonymous and independent reviewers, including those nonmedically trained, evaluate surgical skill with validated performance tools such as OSATS
Several tools using surgical simulation or direct observation are currently being used on practicing surgeons to evaluate performance and technical skills needed for a surgical specialist (Table 3). Six of these tools reliably and accurately detect different level of technical skills among study participants, consisting of surgical trainees, fellows, and staff. All of these methods were validated for surgical trainees. Objective Structured Assessment of Technical Skills (OSATS) is the only method capable of distinguishing between surgeons of different skill level and showing a relationship between skill level and post-operative outcomes.
Surgical procedures and/or skills are evaluated by an expert, or non-medically trained reviewers with regards to Crowd-Sourced Assessment of Technical Skills (C-SATS), using a paper-based tool in six of the object assessments, blinded to the post-graduate year of the participant. The objective assessments were designed for common surgical procedures extending over multiple specialties, including laparoscopic cholecystectomy, open reduction and internal fixation (hip, wrist, or ankle), arthroplasty, and robotic prostatectomy. Global Operative Assessment of Laparoscopic Skills (GOALS) and Global Evaluative Assessment of Robotic Skills (GEARS), however, target laparoscopic and robotic surgery only. The last tool, Direct Objective Metric Measures, uses measurable metrics to determine skill level instead of technique. Direct Objective Metric Measures investigate the stiffness and failure load of the final surgical product, which are critical within the orthopaedic field.
DISCUSSION
For almost all of history, people worked until they died. Retirement is a recent phenomenon, starting during the Great Depression when governments, unions, and employers, desperate to make room in the workforce for young workers institutionalized retirement programs as we know them today, complete with social security and pension plans. Initially, the designated retirement age of 65 was longer than the life expectancy, but as life expectancy has increased, retirement age in certain professions has become more arbitrary. Surgeons may continue to work longer than other professionals because of their satisfaction and gratification in treating patients, because their work connects the surgeon to an identity or for financial reasons. Surgeon’s retirement age remains a contentious issue and presently there is no mandatory retirement age for surgeons. Furthermore, we could not find any universal, well-established and accepted policies of testing for competency of aging surgeons.
Although a review of 65 studies of physicians’ retirement planning found that most physicians retire between ages 60 and 69 years, some surgeons delay retirement because of financial insecurity, lack of other interests or fear of change in their personal lives and identity[20]. Despite the well-recognized decline of cognitive and physical skills with ageing, most surgeons only require verification of their CME to maintain their medical license to practice, in the absence of skill-based simulations that regulate non-medical professionals. Airline pilots, air traffic controllers, firefighters, and United States State Judges were found to have thorough medical examinations and skill assessments during their practice. Regardless of inherent ability, they are further subjected to mandatory retirement ages. Although the United States Age Discrimination and Employment Act of 1967 protects individuals at or above the age of 40 from mandatory retirement ages, employers have established a legitimate age-based criterion referred to as a bona fide occupational qualification (BFOQ), allowing them to justify an age-based BFOQ for certain professions.
Despite age being a variable process, the potential association within the surgical profession between increasing age and poorer operative outcomes has been reported in some studies[9-12]. The mandatory late career practitioner policies identified in this study begin to address the competency of the surgical workforce, through cognitive and physical testing. Performance testing consisted of peer-review assessment and varied across the different health institutions. In the absence of standardized technical skill assessments and continued legal challenges[21], age may not be the most reliable and objective performance indicator.
Surgeons, like other medical specialties, need to regularly engage in activities that keep them up to date with standards of care. However, the requirements to be certified or licensed are highly dependent on the standards developed by individual states, provinces or specialty boards. Furthermore, the validity of CME and MOC, defined as the ability to distinguish expertise level and effects on patient outcomes, was noted as unclear based on available literature. CME, such as didactic educational meetings, were shown to have only a small effect on clinical practice and patient outcomes[22,23]. Completion of MOC is not associated with differences in complication rates in specific surgical subspecialties[24]. In fact, surveyed surgeons were in favor of improving MOC with additional testing such as cognitive assessments or review of cases for older surgeons, in place of a mandatory retirement age[25]. Meeting CME or MOC requirements alone does not guarantee a successful surgical practice, despite being used to certify surgeon competency.
Similar to the testing of pilots and air traffic controllers, there are surgical simulators designed to provide an objective assessment of surgical technical skills. Although these simulators have been used primarily for surgical residents to gauge their level of training, surgical simulators can equally be used to evaluate practicing surgeons by their peers or recognized experts[26,27]. These simulators may then be used to assess surgical competency among, not only older surgeons, but all surgeons as part of their medical education, showing who may require remediation or retirement. However, one of the most significant limitations in the use of simulation for assessing competence for aging surgeons–namely the ability to evaluate the most important skill that dictates surgical expertise and competence: Intra-operative cognitive skills. To date, very few if any surgical simulations have demonstrated the ability to assess advanced mental processes such as decision-making and judgment; and pattern recognition. These cognitive behaviors (or “thinking skills”) are some of the most important aptitudes that dictate performance in the operating room and better methodologies are required to measure them[28]. In addition, the use of simulators would need to be specialty specific and the specialty societies have not yet built these programs.
While this study demonstrated that current competency assessment methods for older surgeons require improvement, the study was limited by various factors. This study is a scoping review, thereby providing an overview of an important topic, without describing every possible assessment tool. Information regarding non-medical specialties was constrained to predefined professions, weakening the comparative analysis between these professions and surgeons. Other non-medical professions were not investigated, although the ones presented are commonly known. The number of late practitioner policies identified was also small, as they needed to be accessible online. It is clear that other policies exist, some of which have been referenced to online, such as Sharp Rees-Stealy Medical Group, Intermountain Healthcare in Utah, Scripps Health network in San Diego, Arkansas Children’s Hospital, Cooper University Hospital, University of Pittsburgh Medical Center, and Virtua Health[29,30]. However, their policies are not explicitly described so that it is unclear whether or not these policies address older surgeon competence in a similar manner to the policies discussed in this study, or in a manner separate from cognitive testing or peer-assessment. As well, there are likely other late practitioner policies that exist but are not present online. In addition, the non-licensing body surgical simulators have been studied across different procedures within different specialties, but the generalizability remains in question. For example, Crowd Sourced Assessment of Technical skills (C-SAT) was shown to be valid for robotic skills amongst urology residents but whether it is valid in orthopaedics or other specialties is not known[26,31]. And given that these surgical simulators have largely been validated for distinguishing skill among surgical residents, more studies need to evaluate the validity and reliability of these simulators for staff surgeons before even considering implementation. In addition, even when there is a valid technique for objective assessment of competence in the execution of particular operations by surgeons, such as the assessment of intraoperative videos, there has been low utilization due to its labor-intensive nature involving human factors (cognitive engineering) expertise[32].
CONCLUSION
Surgery is a profession that requires good surgical judgment, as well as manual dexterity and physical skills for performing an operation. Age alone is not an indication of surgical competence, so testing of these attributes is necessary to ensure that the ageing surgeon remains competent. This requires regular periodic review of the surgeon’s outcomes and skills to ensure that the ageing surgeon has the competency to meet the standards of the profession. The strategy of using skill-based simulations in evaluating non-medical professionals can be similarly used as part of the assessment of the aging surgeons’ surgical competency. While more studies investigating the validity of these simulators is needed, future implementation of these simulators may ensure all aging surgeons maintain an appropriate professional standard for patient safety. A surgeon should not be forced to hang up his/her surgical cap at a predetermined age, but should be able to practice for as long as his/her surgical skills are objectively maintained at the appropriate level of competency. For those aging surgeons with a diminishing skillset, there other potential options to integrate these surgeons into important aspects of surgical care such as assisting younger surgeons for more complex cases, teaching and training the next generation of surgeons, coaching surgeons in practice, being involved in quality-improvement and leadership roles.
ARTICLE HIGHLIGHTS
Research background
The aging surgeon has remained a contentious patient safety issue, as the average age of surgeons continues to rise.
Research motivation
When should an older surgeon stop operating? This becomes an important issue when we consider that the surgical profession is highly dependent on memory, sensory acuity, clinical decisiveness, technical skills and physical stamina; skills and abilities that may decrease with age.
Research objectives
The aim of this scoping review study was to investigate strategies used to determine competency in the industrial workplace that could be transferrable in the assessment of aging surgeons and to identify existing competency.
Research methods
Scoping review.
Research results
Surgeon’s retirement age remains a contentious issue and presently there is no mandatory retirement age for surgeons. Furthermore, we could not find any universal, well-established and accepted policies of testing for competency of aging surgeons.
Research conclusions
A surgeon should not be forced to hang up his/her surgical cap at a predetermined age, but should be able to practice for as long as his/her surgical skills are objectively maintained at the appropriate level of competency.
Research perspectives
More studies need to evaluate the validity and reliability of these simulators for staff surgeons before even considering implementation.
American Medical Association.
Competency and the aging physician. Report 5 of the Council on Medical Education (A-15). In: Chicago, IL, 2015: 2-19.
[PubMed] [DOI][Cited in This Article: ]
Neumayer LA, Gawande AA, Wang J, Giobbie-Hurder A, Itani KM, Fitzgibbons RJ Jr, Reda D, Jonasson O; CSP #456 Investigators. Proficiency of surgeons in inguinal hernia repair: effect of experience and age.Ann Surg. 2005;242:344-8; discussion 348.
[PubMed] [DOI][Cited in This Article: ][Cited by in Crossref: 118][Cited by in F6Publishing: 132][Article Influence: 6.6][Reference Citation Analysis (0)]
Davis D, O'Brien MA, Freemantle N, Wolf FM, Mazmanian P, Taylor-Vaisey A. Impact of formal continuing medical education: do conferences, workshops, rounds, and other traditional continuing education activities change physician behavior or health care outcomes?JAMA. 1999;282:867-874.
[PubMed] [DOI][Cited in This Article: ][Cited by in Crossref: 1549][Cited by in F6Publishing: 1451][Article Influence: 55.8][Reference Citation Analysis (0)]
Madani A, Watanabe Y, Bilgic E, Pucher PH, Vassiliou MC, Aggarwal R, Fried GM, Mitmaker EJ, Feldman LS. Measuring intra-operative decision-making during laparoscopic cholecystectomy: validity evidence for a novel interactive Web-based assessment tool.Surg Endosc. 2017;31:1203-1212.
[PubMed] [DOI][Cited in This Article: ][Cited by in Crossref: 21][Cited by in F6Publishing: 21][Article Influence: 2.3][Reference Citation Analysis (0)]
Holst D, Kowalewski TM, White LW, Brand TC, Harper JD, Sorensen MD, Truong M, Simpson K, Tanaka A, Smith R, Lendvay TS. Crowd-Sourced Assessment of Technical Skills: Differentiating Animate Surgical Skill Through the Wisdom of Crowds.J Endourol. 2015;29:1183-1188.
[PubMed] [DOI][Cited in This Article: ][Cited by in Crossref: 64][Cited by in F6Publishing: 64][Article Influence: 6.4][Reference Citation Analysis (0)]
American Board of Orthopaedic Surgery.
Maintenance of Certification. [cited 30 January 2020]. Available from: https://www.abos.org/moc/.
[PubMed] [DOI][Cited in This Article: ]
Hardison CM, Lim N, Keller KM, Jefferson MP, Payne LA, Bozick R, Mariano LT, Mauro JA, Miyashiro L, Oak G, Saum-Manning L.
The 2013 Firefighter Selection Process. In: Recommendations for Improving the Recruiting and Hiring of Los Angeles Firefighters. Santa Monica, California: Rand Corporation; 2015: 21-34.
[PubMed] [DOI][Cited in This Article: ]
UC San Diego Physician Assessment and Clinical Education (PACE) program.
Late Career Health Screening for Physicians and Healthcare Professionals (LCHS). 2020. [cited 1 April 2020]. Available from: http://www.paceprogram.ucsd.edu/Assessment/LCHS/LCHS_Main.aspx.
[PubMed] [DOI][Cited in This Article: ]
Norcross WA, Henzel TR, Freeman K, Milner-Mares J, Hawkins RE. Toward meeting the challenge of physician competence assessment: The University of California, San Diego Physician Assessment and Clinical Education (PACE) Program.Acad Med. 2009;84:1008-1014.
[PubMed] [DOI][Cited in This Article: ][Cited by in Crossref: 24][Cited by in F6Publishing: 17][Article Influence: 1.1][Reference Citation Analysis (0)]
Page GG, Bates J, Dyer SM, Vincent DR, Bordage G, Jacques A, Sindon A, Kaigas T, Norman GR, Kopelow M. Physician-assessment and physician-enhancement programs in Canada.CMAJ. 1995;153:1723-1728.
[PubMed] [DOI][Cited in This Article: ]
Zevin B, Bonrath EM, Aggarwal R, Dedy NJ, Ahmed N, Grantcharov TP; ATLAS group. Development, feasibility, validity, and reliability of a scale for objective assessment of operative performance in laparoscopic gastric bypass surgery. J Am Coll Surg 2013; 216: 955-965.
quiz 1029-31, 1033.
[PubMed] [DOI][Cited in This Article: ][Cited by in Crossref: 72][Cited by in F6Publishing: 75][Article Influence: 6.3][Reference Citation Analysis (0)]
Dwyer T, Slade Shantz J, Kulasegaram KM, Chahal J, Wasserstein D, Schachar R, Devitt B, Theodoropoulos J, Hodges B, Ogilvie-Harris D. Use of an Objective Structured Assessment of Technical Skill After a Sports Medicine Rotation. Arthroscopy 2016; 32: 2572-2581.
e3.
[PubMed] [DOI][Cited in This Article: ][Cited by in Crossref: 6][Cited by in F6Publishing: 8][Article Influence: 0.9][Reference Citation Analysis (0)]
Ghaderi I, Vaillancourt M, Sroka G, Kaneva PA, Vassiliou MC, Choy I, Okrainec A, Seagull FJ, Sutton E, George I, Park A, Brintzenhoff R, Stefanidis D, Fried GM, Feldman LS. Evaluation of surgical performance during laparoscopic incisional hernia repair: a multicenter study.Surg Endosc. 2011;25:2555-2563.
[PubMed] [DOI][Cited in This Article: ][Cited by in Crossref: 18][Cited by in F6Publishing: 15][Article Influence: 1.1][Reference Citation Analysis (0)]