Construction Worker Images with Ergonomic Risk Captions

Description

It is noteworthy to emphasize the scarcity of publicly available datasets explicitly tailored for training or evaluating VLMs with construction ergonomic risk knowledge. This dataset was created for the study titled 'Interactive VLM-Based Visual Querying for Construction Ergonomic Risk Reasoning'. The study focuses on the construction industry, which is highly sensitive to WMSDs. To achieve high accuracy in postural ERA, the study adopts an image-based approach and uses the vision Transformer (ViT) to process images. This choice stems from recognition of the need for image input in postural ergonomic risk assessment, while relying on Transformer-based large language models (LLMs) to improve assessment accuracy. This approach enables distinct functionalities beyond those offered by traditional ML-based ERA methods. These functions include: (1) answering visual questions about postural risk exposure; and (2) generating captions that describe workers’ actions and associated postural risks.

Institutions

• University of Alberta

  Alberta, Edmonton

Categories

Licence