Checklists can effectively assess work-related risk of musculoskeletal injuries

Previous scholarship regarding the reliability and validity of checklists is limited and sometimes contradictory. Some critics suggest that checklists may overestimate the presence of risk factors and others have questioned their reliability. In this paper, Albin presents a well-founded approach that an ergonomics practitioner can use to dynamically measure the reliability and validity of checklists that he or she uses in order to be sure that they are effective tools.

Traditionally the reliability of a checklist is based on correlating two ratings of the same job performed by a single evaluator or by assessing the agreement of the ratings given to a job by a group of evaluators. Albin argues that alternate methods of assessing consistency are equally appropriate, for example, the use statistical process control tools such as control charts.

Albin also questions how well un-validated checklists serve as a tool to put resources against an at-risk job. "If you have to persuade a manager to release resources to remediate a high-risk job based on checklist findings, what assurance can you give that you are committing limited resources to the appropriate job or jobs?" Albin said.

Albin recommends a thorough yet concise approach for assessing a checklist's predictive quality. Using the basic information gathered from analyzing jobs with a checklist, the practitioner can construct a 2 x 2 statistical table that serves as the basis for probability calculations, which ultimately yield the probability that at-risk jobs are correctly identified. He also discusses testing the significance of differences between alternate forms of a checklist, fine-tuning checklists, generalizing findings to new situations, and assessing the relative importance of risk factors to the identification of the problem job.