Civil Cockpit Multi-View-To-Depth Transfer
Description
Eight typical manipulation actions were defined in this dataset: A1: pulling the left and right thrust levers back at the same time, reducing the thrust of both engines simultaneously. It is commonly used at descent transition and in the landing flare. A2: pushing the left and right thrust levers forward at the same time, increasing the thrust of both engines simultaneously. It is commonly used at takeoff thrust set and during go-around. A3: controlling the left and right thrust levers separately with the left and right sides of hand, cross alternating both levers to reduce the thrust of the two engines in a staggered manner. A4: cross alternating both thrust levers to increase the thrust of the two engines in a staggered manner. Both A3 and A4 are commonly used to match engine indications. A5: pushing the air speed brake lever out to stow the speed brake. It is performed once the target speed or altitude is captured. A6: pulling the air speed brake lever back to deploy the speed brake. It is performed when a higher descent rate or rapid deceleration is required. A7: turning the target-airspeed knob on the MCP, changing the selected target airspeed. It is used whenever air traffic control assigns a speed. A8: pressing the VNAV button on the MCP. It is used when vertical navigation is engaged after takeoff. Four subsets are included. D1 (source domain) was recorded in the laboratory engineering simulator. Eight RGB cameras were fixed around the seat and calibrated beforehand. During collection the operator's right-hand actions were synchronously recorded by all cameras. AlphaPose was run online on every view to localize the two-dimensional joints of the Halpe_coco_136 whole-body skeleton. The detections were triangulated immediately into three-dimensional coordinates in the rig frame. D2 (target domain) was recorded in the 737 MAX Level-D simulator with a single depth camera. Three-dimensional joint coordinates were derived directly from the depth stream, expressed relative to the camera frame. D3 (paired set) was recorded in the laboratory, adding the depth camera to the D1 rig so that the same action was captured simultaneously by both. In actual service, the mounting position of camera and each pilot's seat adjustment may both vary. Also, camera installed in cockpit cannot avoid vibration as the airframe moves. Both factors make the relative position between pilot and camera can be hardly determined. Accordingly, the spatial relationship between the depth camera and the eight source-domain cameras during the D3 recordings was left not recorded intentionally. D4 (sensor characterization set) was recorded in the laboratory and contains raw depth frames of body motions and of the calibration board. It was recorded at stand-off distances of roughly 0.3 m to 2.2 m, with the camera intrinsics and depth scale. It characterizes the depth sensor itself: range-dependent noise, quantization step, and projection geometry.
Files
Institutions
- Shanghai Jiao Tong UniversityShanghai, Shanghai