Source: Anesth 2002 96(1): 5-9

Reviewer:  R. Prasad, MD

Summary:

• Review of videotapes assoc with more improvement in overall rating, and specific performance criteria

• What a great, and simple idea! We've talked about it, but haven't done it (yet).
• Probably also useful for fiberoptic intubations, central lines blocks, and other techniques.
• All residents knew they were being taped ... key was to review tapes. I'd think that more immediate review would be better, but even twice/week seems adequate.
• Observer bias possible: tapes judged on day of taping, so attendings knew to expect some improvement. May have been better to judge all tapes in random order at end of study. Also, hard to believe that judges couldn't identify the resident being judged (despite faces not being included).

• 22 CA-2s beginning first OB rotation assigned to Video or Non-Video groups
• All: filmed daily during epidural placement
• V: reviewed tapes twice/week with attending
• NV: never saw their films
• 4 attendings independently judged videos
• days 1, 15, and 30
• "overall" skill = subjective score (0-10, summed 4 judges = 0-40 range)
• criteria skill = scored 0-2 (major error=0, minor error=1, no error=2) on 13 predefined criteria (0-26 x 4 = 0-104 range)
• criteria: positioning, aseptic technique, no spillage iodine into tray, iodine allowed to dry, draping, needle control, midline location, cephalad needle direction, air amount not >5ml, catheter insertion, needle withdrawl, catheter securing, gentle patient treatment
• an increase of 3 pts considered necessary to detect "notable improvement" for any skill
• 3 pts = 3 our of 4 judges finding a 1-point (major error=>minor error, or minor error=>no error) improvement, or 2 judges finding improvements of 1 and 2 pts.
• looked at # of residents who improved from low score at start to higher at end

• interrater reliability high (kappa - 0.7-0.8)
• excellent correlation (0-933, 0.846, 0.799 on day 1, 15, 20) between criteria scores and overall scores
• V improved more, and continued to improve

Median Criteria Scores

Day	V	NV
1	54 (0-99)	46 (1-103)
15	100 (33-104)	81 (34-104)
30	104 (75-104)	82 (46-104)

• at least 3-pt improvement in skills between days 1 and 30
• V: almost all residents in all 13 skills (11 skills: 100% residents improved; 2 skills: 86%)
• NV: improvement rate >85% in only 4 skills (in other 9 skills, 43-83% of residents improved)