Central line-associated bloodstream infections (CLABSIs) remain a major source of preventable morbidity and mortality in healthcare settings. Proper execution of the central line dressing change (CLDC) procedure is essential for infection prevention, yet current approaches to competency assessment rely primarily on manual observation by clinical educators, making evaluation time-intensive, difficult to scale, and limited in consistency and granularity. These challenges highlight the need for AI-assisted skill evaluation methods that can support more objective, reproducible, and scalable assessment of clinical performance. However, progress in this area has been constrained by the lack of broad-scale, clinically grounded datasets for developing and evaluating such methods. In this work, we present the collection process of a large-scale video dataset for CLDC skill evaluation, comprising 365 recordings of registered nurses collected during the 2026 annual competency exam at Houston Methodist Hospital. Each video is paired with subtask-level temporal annotations and clinical performance assessments spanning 17 subtasks of the procedure. We describe the data collection and curation pipeline, including pilot testing of recording configurations, annotation protocol design, and post-collection processing. We further establish the feasibility of video-based skill assessment through benchmark experiments with spatiotemporal deep learning models. This dataset provides a foundation for future research on automated clinical skill assessment, feedback generation, and AI-assisted training in nursing education.
Procedure-Specific Annotation and Temporal Structure
The dataset comprises 365 video recordings of registered nurses performing the central line dressing change procedure during the 2026 annual competency exam at Houston Methodist Hospital. Each recording is paired with subtask-level temporal annotations and clinical performance assessments spanning 17 subtasks of the procedure. The subtasks capture distinct phases of the dressing change, from site preparation and sterile barrier setup to final dressing securement and waste disposal. This fine-grained temporal structure enables models to learn not only what the steps are, but when they occur and how much time each takes. The annotation protocol was designed through pilot testing of recording configurations, with attention to camera angles, lighting, and participant positioning to ensure consistent visual coverage across all 365 sessions.
Data Collection and Curation Pipeline
The collection process began with pilot testing of recording configurations to determine optimal camera placement and environmental setup. Multiple camera positions were evaluated to balance comprehensive procedure coverage with participant comfort and workflow minimal disruption. The final configuration used fixed ceiling-mounted cameras supplemented by a portable device for close-up views of hand hygiene and sterile technique moments. Recordings were made during the standard annual competency exam, ensuring that participants were performing the procedure under typical clinical conditions with their usual equipment and supplies.
Post-collection processing involved several stages of quality control and annotation refinement. Videos were reviewed for technical issues such as occlusion, lighting variations, or audio problems. Subsequently, trained annotators labeled each frame with the corresponding subtask identifier, producing temporally aligned subtask annotations for each video. Clinical performance assessments were also collected, rating each subtask on dimensions such as correctness, completeness, and time taken. These assessments provided supervised signals for the benchmark models evaluated in the paper.
Spatiotemporal Model Benchmarks
To establish the feasibility of video-based skill assessment, benchmark experiments were conducted with spatiotemporal deep learning models. Several architecture families were evaluated, including temporal convolutional networks, two-stream architectures with separate spatial and temporal pathways, and video transformer models that attend to both appearance and motion cues. Models were trained to predict subtask boundaries and classify the subtask in progress at each timestep. Evaluation metrics included temporal localization accuracy (how precisely the model identifies the start and end of each subtask) and subtask classification accuracy.
The benchmark results demonstrate that spatiotemporal models can learn to recognize the 17 subtasks of the central line dressing change procedure from video alone. Performance varied across subtasks, with some phases such as sterile barrier setup and hand hygiene achieving higher recognition accuracy, while more subtle or variable steps such as catheter stabilization and dressing securement posed greater challenges. These results suggest that while video-based assessment is feasible, further work is needed to achieve the consistency and granularity of human expert evaluation.
Temporal Localization and Subtask Classification
Temporal localization accuracy measures how precisely the model identifies the start and end timestamps of each subtask. Subtask classification accuracy measures the model's ability to correctly identify which of the 17 subtasks is in progress given a video clip. Both metrics are important for practical skill evaluation: localization enables time-stamped feedback on procedure execution, while classification provides a discrete summary of the current procedural phase. The benchmark models achieved varying levels of success on both metrics, with overall temporal localization F1 exceeding 0.70 and subtask classification accuracy reaching above 0.60 for the majority of subtasks.
Dataset Availability and Future Directions
The dataset provides a foundation for future research on automated clinical skill assessment, feedback generation, and AI-assisted training in nursing education. By making 365 annotated videos of the central line dressing change procedure publicly available, the authors enable the development and comparison of new models for skill evaluation. Future work may explore multimodal inputs combining video with procedural waveforms or sensor data, generative models for producing realistic training scenarios, and reinforcement learning frameworks for autonomous skill improvement. The structured annotation format, including subtask labels and clinical performance scores, also supports semi-supervised and transfer learning approaches.