Hand Gesture and Arm Movement Recognitionfor Multimodal Control of a 3-DOF Helicopter

Abstract: In this project we present the application of hand gestures and arm movements to control a dual rotor testbench. A multimodal control method is developed for a 3-degrees-of-freedom (DOF) tandem helicopter based on surface electromyography sensors and an inertial measurement unit (IMU) included in the Myo Armband sensor.

The implemented architecture consists of three basic blocks: the first is data acquisition which is achieved via a Myo Armband sensor that allows the user to interact with the system in a natural way through hand gestures and arm movements. The second part is the processing and HMI that deals with receiving the information from the sensor and generating a setpoint command for the 3-DOF helicopter accordingly, which is the third and last part of the system.

The recognition system can classify five different hand gestures which are used for switching between flight modes and generating set point values for the helicopter. Flight commands:

Demonstration video:

The experimental results shown in the following video demonstrate the effectiveness of using hand gesture recognition and arm movement for controlling an aerial test bench with a fast and accurate response.

Source code:



  • Ladrón de Guevara E11-253, Quito – Ecuador
  • Campus Politécnico “José Rubén Orellana”
    Facultad de Ingeniería de Sistemas
    Cuarto Piso