Home ARDUINO Control your PC with hand Gesture using Arduino and Ultrasonic Sensors

Control your PC with hand Gesture using Arduino and Ultrasonic Sensors

4

Today hand Gesture controlled Mobile Computers and other Portable gadgets are getting very famous so in this article we are going to make a device which gives any computer or laptop hand gesture feature. So basically this technique is called Leap motion which actually enables us to control certain functions on our computer/Laptop like a keyboard or mouse do for the laptop and computer and in keyboard and mouse we press the buttons of keyboard and move the mouse for make communication between the computer and human so in this simply waving our hand in front of it. Yes, with the gesture of hands and this will be a very cool and fun project to do it, but these laptops are really priced very high which is actually supporting these features. So in this project, we will try building our own Gesture Control Laptop/Computer by combining the Power of Arduino and Python programing language.

In this project, we will use two Ultrasonic sensors to control the position of our hand and control a media player such VLC based on the position. I have used this for a demo, but once you have understood the project, you can do anything by just changing few lines of code such change the gesture into keyboard words or other stuff and control your favourite application in your favourite way.

Concept of this project

The concept behind this project is very simple as we discuss the things above. We will place two ultrasonic sensors on the top of the display and use the Arduino to read the distance between the display and with our hand which will be used as input gesture that we will perform some operations using VLC. So Perform an Arduino as input keyboard for a computer we will use some operation in our computer, and for this, we will use the Python pyautogui library. Arduino commands are sent to the computer through the serial port (USB). The data will be read in python running on your computer and will take an action based on the data read.

So let’s get Started

STEP 1:- Collect the hardware

  • Jumper wires

Step 2:- Wire up hardware

Wire up the hardware, connect the Arduino UNO with HSR04 sensors as the mentioned image right below!

Step 3:- Program Desktop with Python

DOWNLOAD Python 2.7

Installing pyautogui module for windows:

Follow the below steps to install a pyautogui library in windows. If you are using other platforms the steps will also be more or less similar. Make sure computer/Laptop is connected to the internet and follow the steps below.

Step 1: Open Windows Command prompt by Press”window+ R” and type in Run Application as CMD  and change the directory to the folder where you have installed python. By default, the command should be

cd C:\Python27

Step 2: Inside your python directory use the command python –m pip install –upgrade pip to upgrade your pip. Pip is a tool in python which helps us to install python modules easily. Once this module is upgraded (as shown in picture below) proceed to next step.

python –m pip install –upgrade pip

Step 3: Use the command “python –m pip install pyautogui” to install the pyautogui module. Once the process is successful you should see a screen something similar to this below.

python –m pip install –upgrade pip

 

Now that the pyautogui module and pyserial module  is successful installed with the python, we can proceed with the python program. The complete python code is given at the end of the tutorial but the explanation for the same is as follows.

Let us import all the three required modules for this project. They are pyautogui, serial python and time.

import serial #Serial imported for Serial communication

import time #Required to use delay functions

import pyautogui

 

Next we establish connection with the Arduino through COM port. In my computer the Arduino is connected to COM 18. Use device manager to find to which COM port your Arduino is connected to and correct the following line accordingly.

ArduinoSerial = serial.Serial(‘com18’,9600) #Create Serial port object called arduinoSerialData

time.sleep(2) #wait for 2 seconds for the communication to get established

Here Is Python Code run the code in python as the image shown below

Step 4 Compile Python Code:

import serial #Serial imported for Serial communication

import time #Required to use delay functions

import pyautogui

ArduinoSerial = serial.Serial(‘com18’,9600) #Create Serial port object called arduinoSerialData

time.sleep(2) #wait for 2 seconds for the communication to get established

while 1:

incoming = str (ArduinoSerial.readline()) #read the serial data and print it as line

print incoming

 

if ‘Play/Pause’ in incoming:

pyautogui.typewrite([‘space’], 0.2)

if ‘Rewind’ in incoming:

pyautogui.hotkey(‘ctrl’, ‘left’)

if ‘Forward’ in incoming:

pyautogui.hotkey(‘ctrl’, ‘right’)

if ‘Vup’ in incoming:

pyautogui.hotkey(‘ctrl’, ‘down’)

 

if ‘Vdown’ in incoming:

pyautogui.hotkey(‘ctrl’, ‘up’)

incoming = “”;

Step 5 Upload Code

Note if you are beginner with Arduino UNO then please have a look on it How to Program Arduino

Arduino Code

 

/*

  • Program for gesture control VLC Player
  • Controlled using Python

*/

const int trigger1 = 2; //Trigger pin of 1st Sesnor

const int echo1 = 3; //Echo pin of 1st Sesnor

const int trigger2 = 4; //Trigger pin of 2nd Sesnor

const int echo2 = 5;//Echo pin of 2nd Sesnor

long time_taken;

int dist,distL,distR;

 

void setup() {

Serial.begin(9600);

 

pinMode(trigger1, OUTPUT);

pinMode(echo1, INPUT);

pinMode(trigger2, OUTPUT);

pinMode(echo2, INPUT);

}

/###Function to calculate distance###/

void calculate_distance(int trigger, int echo)

{

digitalWrite(trigger, LOW);

delayMicroseconds(2);

digitalWrite(trigger, HIGH);

delayMicroseconds(10);

digitalWrite(trigger, LOW);

time_taken = pulseIn(echo, HIGH);

dist= time_taken*0.034/2;

if (dist>50)

dist = 50;

}

void loop() { //infinite loopy

calculate_distance(trigger1,echo1);

distL =dist; //get distance of left sensor

calculate_distance(trigger2,echo2);

distR =dist; //get distance of right sensor

//Uncomment for debudding

/*Serial.print(“L=”);

Serial.println(distL);

Serial.print(“R=”);

Serial.println(distR);

*/

//Pause Modes -Hold

if ((distL >40 && distR>40) && (distL <50 && distR<50)) //Detect both hands

{Serial.println(“Play/Pause”); delay (500);}

calculate_distance(trigger1,echo1);

distL =dist;

calculate_distance(trigger2,echo2);

distR =dist;

 

//Control Modes

//Lock Left – Control Mode

if (distL>=13 && distL<=17)

{

delay(100); //Hand Hold Time

calculate_distance(trigger1,echo1);

distL =dist;

if (distL>=13 && distL<=17)

{

Serial.println(“Left Locked”);

while(distL<=40)

{

calculate_distance(trigger1,echo1);

distL =dist;

if (distL<10) //Hand pushed in

{Serial.println (“Vup”); delay (300);}

if (distL>20) //Hand pulled out

{Serial.println (“Vdown”); delay (300);}

}

}

}

//Lock Right – Control Mode

if (distR>=13 && distR<=17)

{

delay(100); //Hand Hold Time

calculate_distance(trigger2,echo2);

distR =dist;

if (distR>=13 && distR<=17)

{

Serial.println(“Right Locked”);

while(distR<=40)

{

calculate_distance(trigger2,echo2);

distR =dist;

if (distR<10) //Right hand pushed in

{Serial.println (“Rewind”); delay (300);}

if (distR>20) //Right hand pulled out

{Serial.println (“Forward”); delay (300);}

}

}

}

delay(200);

}

Done!

Here is the Test video

You may also like to read these articles

How to Make Radar Detector with Arduino and Ultra Sonic Sensor

Build Your Own Vehicle Tracking System using Arduino (Start with Sim808 )

How to Program ESp8266 Esp01 with Arduino UNO

Hope my article “Control your PC with hand Gesture using Arduino and Ultrasonic Sensors” helps you to Control your PC with hand Gesture using Arduino. If you have any query, feel free to comment.

4 COMMENTS

  1. You just have to upload the source code and as i mention in my blog that The concept behind this project is very simple as we discuss the things above. We will place two ultrasonic sensors on the top of the display and use the Arduino to read the distance between the display and with our hand which will be used as input gesture that we will perform some operations using VLC. So Perform an Arduino as input keyboard for a computer we will use some operation in our computer, and for this, we will use the Python pyautogui library.

  2. The project can be done with one ultrasonic sensor also know???
    Can you explain the phython code as i don’t know nothing about phython, i know c and c++,MATLAB can we do with that
    Thank you

    • yes it can be done via single ultrasonic sensor and also 2 or more !!
      so basically in the python script, i am using 4 type of sensing trigger method
      const int trigger1 = 2; //Trigger pin of 1st Sensor

      const int echo1 = 3; //Echo pin of 1st Sesnor

      const int trigger2 = 4; //Trigger pin of 2nd Sesnor

      const int echo2 = 5;//Echo pin of 2nd Sensor
      So basically my Arduino working as a keyboard for the laptop,
      In my computer the Arduino is connected to COM 18. Use device manager to find to which COM port your Arduino is connected to and correct the following line accordingly.
      So as i mention above, we will place two ultrasonic sensors on the top of the display and use the Arduino to read the distance between the display and with our hand which will be used as input gesture that we will perform some operations using VLC. So Perform an Arduino as input keyboard for a computer we will use some operation in our computer, and for this, we will use the Python pyautogui library. Arduino commands are sent to the computer through the serial port (USB). The data will be read in python running on your computer and will take an action based on the data read.

LEAVE A REPLY

Please enter your comment!
Please enter your name here

sixteen − 1 =