Implementación de instrumentos de evaluación objetiva de competencias técnicas en cirugía

Abstract

La adquisición de habilidades quirúrgicas constituye un factor central en la formación de todo cirujano. Sin embargo, la evaluación de las habilidades técnicas es uno de los factores más débiles y menos desarrollados. En la actualidad los recursos para evaluar las competencias técnicas adolecen de subjetividad, falta de confiabilidad y validez. La observación directa, método de evaluación más frecuentemente utilizado en nuestro medio, presenta inconvenientes instrumentales y está fuertemente influenciada por las relaciones intersubjetivas y los rasgos de personalidad. El objetivo de esta investigación es proponer la creación y el uso de un instrumento objetivo para evaluar el desempeño técnico, y determinar su confiabilidad y validez. Materiales y métodos: se seleccionaron dos procedimientos: la colecistectomía laparoscópica y la hernioplastia inguinal (técnica de Lichtenstein). Se constituyeron tres grupos de comparación según la experiencia quirúrgica: inicial, intermedio, y expertos. Se filmaron las cirugías en tiempo real, sin identificación del paciente ni del cirujano. Las filmaciones sin edición fueron asignadas a dos cirujanos expertos en una secuencia aleatoria por sorteo y con sobres sin identificación. Para la evaluación se propuso el uso de un instrumento objetivo (explicitación de pasos a evaluar y cuantificación mediante escala de Likert) y específico para cada procedimiento. Así mismo se utilizó la escala global OSATS (R. Reznick). Se aplicó un análisis de varianza no paramétrico para determinar la validez. Valores de p menores a 0.05 fueron considerados estadísticamente significativos. Valores superiores a 0,80 del Coeficiente alfa de Cronbach aseguraron confiabilidad.

Introduction: technical skills acquisition is considered to be of paramount importance in surgical training. Yet, formal assessment of technical skills is the weakest and less developed area. Currently available resources to evaluate technical skills are largely subjective, and lack of validity and reliability. Direct observation, one of the most frequently used methods, is largely biased by interpersonal subjectivity and personality traits. We propose the creation and use of a new procedure-specific tool for objective assessment of technical skills in surgery to evaluate validity and reliability. Materials and Method: laparoscopic cholecystectomy and Lichstenstein’s inguinal hernia repair were the chosen procedures. Three groups of comparison were defined according to surgical expertise: initial, intermediate, and experts. Surgeries were videorecorded in real time without identification of the patient or the surgeon. Tapes without any posterior edition were assigned to two expert surgeons in a blind and randomized sequence. A newly proposed procedure-specific rating scale was used for evaluation, as well as Reznick’s OSATS global scale. Kruskal-Wallis non-parametric test was used to assess validity. p < 0,05 was considered statistically significant. Chronbach’s α values > 0.8 granted reliability. Results: from April 2010 to December 2012 36 laparoscopic cholecystectomies and 31 inguinal hernia repairs were recorded. Significant difference was found among groups of comparison for every item (p<0.05). Crhonbach’s α was largely over 0,80 for both procedures. There was no significant difference between the assessments of the two raters. There was no significant difference between our newly proposed procedure-specific instrument and Reznick’s OSTATS. Discussion: the objective assessment of technical skills in surgery is feasible and useful. The tool we proposed showed construct validity and reliability. Video recording of surgical procedures grants durability over time to an ephemeral phenomenon. The objectivity is based on the explicit statements and quantification of every step to be evaluated, and the blind randomization and anonymous treatment of the sample. Sharing the same quality criteria between evaluators is of paramount importance to reach satisfactory results. The process of evaluation always implies a shortened view of the reality.