Mastering Gesture Recognition with Raspberry Pi and APDS9960 Low-Cost Sensor

In the world of electronics and IoT, gesture recognition has emerged as a fascinating and practical technology. Whether it’s controlling your favorite gadgets with the wave of a hand or adding a touch of magic to your projects, gesture recognition sensors play a pivotal role. Among the numerous sensors available, the APDS9960 stands out as a versatile and widely used I2C sensor at a very affordable price. In this blog post, we will take you on a journey through the fascinating world of gesture recognition using the APDS9960 sensor on a Raspberry Pi and any of our popular add-on boards: ANAVI Infrared pHAT, ANAVI Light pHAT, ANAVI Gardening uHAT, ANAVI Info uHAT as well as ANAVI Miracle uHAT and ANAVI CO2 uHAT (both of which are in final development).

What is APDS9960?

APDS9960 is an I2C (Inter-Integrated Circuit) sensor produced by Broadcom (formerly Avago Technologies). It is known for its versatility and is commonly used for gesture recognition, proximity sensing, ambient light sensing, and color sensing applications. Gestures are detected at a distance of 10 to 20 cm. The sensor has built-in UV and IR filters for better recognition. Its wide range of applications makes it a favorite choice among electronics enthusiasts, engineers, and hobbyists. Ovewr the years APDS9960 has been integrated in many popular consumer electronic devices, including Samsung Galaxy S5 smartphone.

In this tutorial, we will explore how to set up gesture detection on a Raspberry Pi running the Raspberry Pi OS Linux distribution. Specifically, we will use the APDS9960 I2C sensor connected to a Raspberry Pi uHAT (add-on board) and a mini OLED SSD1306 I2C yellow-blue display.

BH1750 I2C light sensor attached to ANAVI CO2 uHAT and a Raspberry Pi single board computer running GNU Linux distribution

🔌 Hardware Setup

Both APDS9960 sensor and the mini yellow-blue SSD1306 OLED display rely on the serial communication protocol I2C that allows multiple electronic devices to communicate with each other using only two wires: serial data (SDA) and serial clock (SCL). Raspberry Pi single board computers have two different I2C busses. The primary I2C bus is at GPIO 2 (physical pin 3 for SDA) and GPIO3 (physical pin 6 for SCL). These pins should be used to attach APDS9960 and the mini OLED display to the Raspberry Pi. Also APDS9960 should be connected to 3.3V and GND pins of the Raspberry Pi to be powered. This makes 4 wires in total to attach the sensor. ANAVI HATs (Hardware Attached on Top) for Raspberry Pi offer dedicated slots for I2C sensors. On ANAVI Info uHAT and ANAVI CO2 uHAT there are even dedicated slots for the mini OLED display.

Of course the OLED display is optional and as an alternative we offer a simple command-line Python3 example for APDS9960 which can function without the OLED display.

🖥️ Software Setup

The software setup is straight forward: install Raspberry Pi OS on microSD card, boot your Raspberry Pi and enable I2C using raspi-config. More details are available in the user’s manual for our HATs.

🐍 Python3 Scripts

The heart of our demonstration are scripts written in the Python 3 programming language. We’ve tailored the script to work seamlessly with Raspberry Pi OS, but it should also run smoothly on any other GNU Linux distribution. These scripts rely on popular Python3 libraries like PIL and Luma OLED. You can find the source code on GitHub for reference and further experimentation:

There are two different Python 3 scripts to demonstrate APDS9960 gesture detection:

  • for detecting gestures and showing them on the mini OLED display as in the video
  • for detecting gesture and printing them in the command-line interface (use this one if don’t have a suitable mini OLED display)

👁️ Real-Time Gesture Detection

Experience the magic as the APDS9960 sensor detects your hand movements, including swipes, taps, and more. Witness how your Raspberry Pi interprets and responds to these gestures in real-time, opening up a world of interactive possibilities. The video provides insights into potential issues you might face during setup and offers practical tips to ensure a seamless experience.

🎓 Conclusion

APDS9960 sensor and Raspberry Pi combination offer a compelling glimpse into the exciting world of gesture-based interaction. Instead of using a breadboard and multiple wires it is always easier to rely on one of our add-on boards like ANAVI Infrared pHAT, ANAVI Light pHAT, ANAVI Gardening uHAT, ANAVI Info uHAT, ANAVI Miracle uHAT and ANAVI CO2 uHAT.

So, if you are ready to embark on this exciting journey into gesture recognition technology, grab your Raspberry Pi, APDS9960 sensor, and let’s get started!

You may also like

Understanding I2C: Exploring OLED Displays and Peripherals on Futuristic Mechanical Keyboards

I2C (Inter-Integrated Circuit) is a popular serial communication protocol that allows multiple integrated circuits to communicate with each other over a short distance, typically limited to a few meters. Each device on the bus has a unique address, identifying it and allowing it to communicate individually. The protocol was developed by Philips (now NXP Semiconductors) in the 1980s. Over the years it has become a standard for communication between various electronic components in embedded devices. I2C can be used to connect various peripherals, such as sensors, displays and EEPROMs. The mini OLED yellw-blue displays on our compact mechanical keyboards ANAVI Macro Pad 12 and ANAVI Arrows are connected to the Raspberry Pi RP2040 microcontrollers over I2C.

ANAVI Arrows and ANAVI Macro Pad 12 are open source mechanical keyboards with mini OLED I2C displays

I2C is easy to use becase it requires only two wires for communication:

  • SDA (Serial Data) for transmitting and receiving data between devices
  • SCL (Serial Clock) for a clock signal to synchronizes the data transfer between the devices

The core component of our hot-swappable mechanical keyboards ANAVI Macro Pad 12 and ANAVI Arrows is Seeed Studio XIAO RP2040 module. This is actually a tiny development board suitable for surface-mount technology (SMT) assembly and equipped with a Raspberry Pi RP2040 32-bit dual-core ARM Cortex M0+ MCU, 264 KB SRAM, 2 MB Flash memory, 11 GPIO pins and USB-C connector. The I2C interface is located on pins D4 (for SDA) and D5 (for SCL) of XIAO RP2040.

Seeed Studio XIAO RP2040 module on ANAVI Macro Pad 12 mechanical keyboard

A yellow-blue mini OLED I2C display is included in all kits with ANAVI Macro Pad 12 and ANAVI Arrows. It has 4 pins: GND (ground), VCC (supply voltage), SCL, and SDA. This display relies on SSD1306, a single-chip CMOS OLED/PLED driver with controller for organic / polymer light emitting diode dot-matrix graphic display system which consists of 128 segments and 64 commons. It is the same display we include in our other mechanical keyboards like the ANAVI Macro Pad 10 and ANAVI Macro Pad 8, Internet of Things devices like the ANAVI Thermometer and ANAVI Gas Detector, tools like ANAVI Fume Extractor, and Raspberry Pi add-on boards like ANAVI Info uHAT. This versatile mini OLED display is a great fit for many projects, You can pick one up at Mouser if you need a spare.

Mini yellow-blue 0.96″ OLED display attached to ANAVI Macro Pad 12 with QMK firmware for mechanical keyboards

There is a dedicated slot for the display on the printed circuit board. Just plug the mini OLED display into it and then connect the mechanical keyboard to a computer. In every kit with ANAVI Macro Pad 12 and ANAVI Arrows, you will find four additional male-to-female jumper wires included, providing you with an exciting opportunity to get creative with your project. If you decide to design your own 3D printed case for the keyboard, these jumper wires may become handy. They grant you the flexibility to reposition the mini OLED display to a location of your choice within the case.

ANAVI Macro Pad 12 is compatible the two most popular open source firmwares for mechanical keyboards: KMK and QMK. KMK is written in CircuitPython and QMK in the C programming language. Both support OLED displays over I2C.

Out of the box ANAVI Macro Pad 12 and ANAVI Arrows come with the KMK firmware. It uses the extension Peg Oled Display based on the open source CircuitPython libraries Adafruit_CircuitPython_DisplayIO_SSD1306 and Adafruit_CircuitPython_Display_Text. This extension allows your keyboard to display images or text and even to react to the currently selected keyboard layer.

ANAVI Macro Pad 12 with a breadboard and additional I2C peripherals

Makers with advanced skills have a fantastic opportunity to extend the capabilities of the keyboard project by adding more I2C peripherals. Using a breadboard and without any soldering, makers can easily connect additional I2C peripherals, such as sensors or other modules that communicate via I2C, to the existing setup. This allows them to expand the project’s functionalities and explore various creative ideas. Those seeking a more permanent and tailored extension can even design their own custom I2C add-on printed circuit board. The mini OLED displays included in the kits work out of the box, but it is important to be aware that incorporating any other I2C devices into the keyboard will require adjusting the KMK firmware to support the additions.

Support our crowdfunding campaign and get the open source mechanical keyboards ANAVI Macro Pad 12 and ANAVI Arrows with a mini OLED display for real-time notifications and customizable graphics at your fingertips. Learn how to use I2C and unleash your creativity by extending the keyboard with additional peripherals!

You may also like

ANAVI Gas Detector Assembly Guide

ANAVI Gas Detector is an ESP8266-powered, open source, Wi-Fi development board for monitoring air quality and detecting dangerous gases. After a very successful crowdfunding campaign at Crowd Supply we are manufacturing a batch of units in Plovdiv, Bulgaria. All kits of ANAVI Gas Detector also include MQ-135 sensor module, USB to UART debug cable, an acrylic stand and a mini OLED I2C display. In this article you will learn how to get started and assemble all parts in about 10 minutes.

The printed circuit board of ANAVI Gas Detector has the same dimensions as the one for my other open source product ANAVI Thermometer so actually both use the same laser cut acrylic stand. Both ANAVI Gas Detector and ANAVI Thermometer have been designed using the open source software KiCAD. The open source software OpenSCAD was used to design the case.

ANAVI Gas Detector Developer Kit

In this article you will learn the exact steps how to assembly ANAVI Gas Detector Starter Kit. The procedure is the pretty much the same for Advanced and Developer kits, however they including additional I2C sensor modules. Appropriate nuts, screws and washers are included in all kits.

Getting started with ANAVI Gas Detector Starter Kit

Step 1

Peel off the protective film from the acrylic stand. Also remove the protective film from the display.

Step 2

Mounting the mini OLED display to the acrylic stand

Using a screwdriver, gently attach the mini OLED display to the acrylic stand with the M2 screws and nuts as shown on the photo. The mini OLED display is fragile so please be very careful. Do NOT fasten it too tight because the torque may break it.

Step 3

Add four M2.5 screws and nuts to the case of the acrylic stand. In the next step we will put ANAVI Gas Detector on top of the nuts.

Step 4

Add ANAVI Gas Detector on top of the nuts and fasten it with the additional M2.5 nuts. At the end you will have a spare M2.5 screw and nut to optionally attach an additional sensor to the acrylic enclosure.

Step 5

Connect the mini I2c OLED display to the dedicated slot for it on ANAVI Gas Detector

Gently connect the mini OLED display to the dedicated slot using the male to female jumper wires. The colors of the wires do not matter. Just have a look at the labels on the top of the OLED display and connect each of the four pins to a pin with the same label on the dedicated slot for the display on ANAVI Gas Detector.

Add the MQ-135 sensor module for monitoring air quality

Plug the MQ-135 sensor module in slot labeled as MQ. Owners of Advanced or Developer kit should also plug the additional I2C sensor modules.

Finally, you are ready to turn on and enjoy ANAVI Gas Detector!

On first boot you need to calibrate MQ-135 and configure and connect ANAVI Gas Detector to your WiFi network.

The so called “burn-in” procedure for initial calibration of MQ-135 is important to be done on first boot. Despite its fancy name it is actually quite easy. Just place ANAVI Gas Detector with MQ-135 sensor module in a room with clean air and leave it running for at least 24 hours. This has to be done only once when MQ-135 sensor module is used for the first time. After doing this procedure, on every next boot ANAVI Gas Detector and MQ-135 will do a quick calibration in couple of minutes and start working properly.

We have also published this tutorial as an update at Crowd Supply to notify all backers of ANAVI Gas Detector. The whole user’s manual for ANAVI Gas Detector is available at GitHub. This is an entirely open source project so as usual ideas for improvements and bug fixes are always welcome!

You may also like

How to Use Mini Monochrome OLED I2C Displays (SSD1306)?

Mini monochrome OLED I2C displays are cheap, reliable and easy to use by makers. They come in a huge variety of sizes. Our open source hardware Internet of Things, like ANAVI Thermometer and ANAVI Gas Detector, support this type of display and all kits include a particular yellow-blue model that is a bit below 1”. The display has 4 mounting holes. The screen resolution is 128×64 pixels. The driver is SSD1306. You can find exactly the same type of display with only white OLED pixels. The usage is the same no matter what is the color of the OLED pixels.

ANAVI Thermomter with yellow-blue mini OLED I2C display

In this video tutorial you will learn the exact steps how to get started with I2C OLED mini display with about 1” diagonal on Arduino compatible board such as our ANAVI Thermometer. ANAVI Thermomter is an ESP8266-powered, open source, Wi-Fi dev board with temperature and humidity sensors.

Continue reading “How to Use Mini Monochrome OLED I2C Displays (SSD1306)?”

You may also like