Touch gesture recognition using body capacitance

For my final year project, I tried to replicate and possibly improve the Disney's touche device using much cheaper components than they did. It was a partial success, but a great learning experience nontheless. These days most touch sensitive devices are designed either to recognize where it has been touched or whether is touched or not. Many objects around us in everyday life can be potentially used as touch interactive surface providing they are made out of conductive material. With the conventional way it would only be possible to detect whether the object has been touched or not. However, by exciting the object with different frequencies it is possible to detect how much skin is touching it. Essentially the system could recognize whether the object is grabbed, pinched, touched by one or more fingers or any other gesture which have different amount of skin touching it. There are many surfaces, objects and liquids which can be transformed into touch sensitive devices without additional buttons or touchscreens like door handles, mobile phones, lamps, desks, walls etc. 


1.1       Outcomes and goals

Main goal of the project was to develop and design a device which would be able to do GD (gesture detection) on the objects connected to it. This would include designing and making the main PCB and writing software for the PC and Android smart phones. However, due to lack of time and unforeseen challenges it was infeasible.

As mentioned above one of the goals was designing and making the main PCB for learning purposes. This process includes:

  1. Designing the PCB in Eagle CAD software
  2. Etching the PCB in home environment
  3. Applying solder mask
  4. Soldering

For the software side of the project, the main goals were as following:

  1. Design PC side application in C++/Qt
  2. Learn LibSVM and write a wrapper to simplify user interactions with it
  3. Port the application mentioned in 1st point for smart phones

 

Most of the goals were successfully achieved, except for increasing the device sensitivity. One of the requirements was to develop a sensor which can be interfaced with human skin directly and used as touch interface. Unfortunately, the current design wasn’t able to excite the human body in different frequencies enough to detect change from the input signal. On the other hand, the GD system is able to sense the presence of human body and differentiate from multiple gestures when touching a conductive object, including liquids. As of now GD system could be used for detecting gestures on objects only, with some modification it can be potentially used as smart light switch, where different gestures enable different lighting in the room or by manufacturing a copper frame in a smart phone, the GD system could sense how is the phone being held in hand.

1.2       GD system architecture

Standard way of doing capacitive touch sensing is by using one fixed frequency. However GD system is using a technique called frequency sweep, which enables to do capacitive touch sensing in multiple frequencies. For example, in frequencies ranging from 100 kHz up to 50 Mhz.

For frequency sweep the micro-processor will generate AC signal in specific frequency range to excite the electrode. The electrode can be any surface connected to the ADC of the micro-processer. The micro-processor measures the amplitude of the specific frequency the electrode was excited. Essentially, constructing spectrum for different levels of excitement. By varying the capacitance the spectrum should in theory change. A human touch basically changes, in almost all cases increases, capacitance. The way human is touching the electrode, or the amount of skin contacted with the electrode will produce a different output.  In other words, electrical properties of human touch will change the excitement spectrum.  Then by using simple machine learning algorithms like SVM or neural network, for pattern recognition, the way of human touching the electrode can be detected.  In short, with frequency sweep technique it is not only possible to detect a touch, but as well how it was touched.  Interestingly this is all achieved through single electrode.  Please see Figure A‑1 for the basic architecture of the touch sensing technique

Figure A‑1 Touch sensing architecture

The technique of frequency sweep has already been used for various medical applications. For example, already in year 1979 it was proposed to be used for analysis of neuromuscular junction continuity[1].  As well it has been used for wireless communication and different kinds of proximity sensors.

The project is highly inspired by Disney’s research called touché[2]. More or less they are using exactly the same approach GD system is using.

There are two more articles (projects) which are using frequency sweep technique to do touch recognition.  Both of them have been highly inspired by touché and they constructed much cheaper version of the touché. Sprites [3]method involves using a standard AVR micro-controller to do frequency sweep from 0-3.5MHz. For recognition, just like touché Sprite is using SVM algorithm. And the other one by DZL [4] reconstructed touché using arduino, however for gesture recognition he is measuring the highest point of the frequency sweep. Which is very unreliable compared to pattern recognition using machine learning.

Even though both of the “cheap” implementations are using PWM for exciting the electrode, the GD system is using a proper sine wave generator, just like touché. More precisely it uses AD9850 function generator. This unit can generate up to 50MHz pure sine wave. The decision was made to use a proper sine wave generator because of the electrical properties of the human body. More than 99% of the human body’s resistance is at the skin on average which is about 1MΩ[5]. On the other hand AC signal can pass through the human skin and go through the least impedance in the corresponding phase and amplitude of AC signal. Internal human resistance is around 100Ω.

 

Files

Download: final_year_project_report2.pdf (5.48M)

Download: final_year_all_files.zip (19.01M)


[1] http://informahealthcare.com/doi/abs/10.3109/03091907909160663

[2] http://www.disneyresearch.com/wp-content/uploads/touchechi2012.pdf

[3] http://spritesmods.com/?art=engarde&page=1

[4] http://dzlsevilgeniuslair.blogspot.se/2012/05/arduino-do-touche-dance.html

[5] http://www.meo.etc.upt.ro/materii/cursuri/IBM/1.pdf




ADVERTISEMENT

  • James J

    James J in Simple library for driving 20x4 LCD with 4bits

    Very good writeup. I thought had been fighting my code for a week or two and confirmed what I was dooing was right. This overview is the best I have seen for doing the code. You may wish to add one thing though that I have found on another website. Urge people to check the contrast dial on...

    Helpful friend

    Helpful friend in MEMS (Part 1) - Guide to using accelerometer ADXL345

    Start with reading about C language, proceed to reading about Arduino platform/ide proceed to reading about i2c bus proceed to reading about adxl345. Rather not the other way round, although that's probably also possible, but it will be more demanding, as you have already...

    Chris Sparks

    Chris Sparks in Simple TLS/SSL SMTP client for Qt5

    qt.network.ssl: QSslSocket::connectToHostEncrypted: TLS initialization failed

    Edward Martin

    Edward Martin in Introduction to data encryption

    The introduction to the data encryption is very much helpful as you will get to know about the data encryption procedure which might help you to protect the files and folders. You can check the 

    bean

    bean in DIY 3D printer, AKA RepRap

    The 3D printer creates the dimensional object and with the features of the 3D printer, it executes the process if need further query then visit fix error code 0xc004f074...

    preter jack

    preter jack in Single Cycle MIPS CPU in VHDL

    Single-cycle CPU is more improved than the normal and usual CPUs and it has a lot of advantages and all you can get from fix

    henrydevid

    henrydevid in nRF51 Makefile with Qt Creator

    It is great for all, I have some different tricks if you want to know it then for that I have some different tricks, for that, you want to learn to code. For that, if you want to know it then for that you just visit some tutorial. And from there it is great for all. If you faced any type...