SOAPI was inspired by the latest COVID-19 pandemic and the multiple advisements from health organizations and the media that everyone should remember to wash their hands frequently and with proper technique. While hand washing might be seen as common sense among most adults, children, especially those who are grade school, often forget the basic procedural methods of how to wash their hands without the assistance of someone older. Thus, in production of creating SOAPI, I wanted to create a positive interactive experience for children that teaches hand-washing with the assistance of a colorful game built onto a household soap dispensor.
I believe that SOAPI is a wonderful addition to help families who are worried for the safety of their younger children in their household as:
- the price to build and install is reasonably cheaper and more functional than an average automatic soap dispensor
- it has cute and colorful design elements in order to appeal to a younger demographic (credit to @yasmineleeart for art created for SOAPI)
What it does
SOAPI has multiple functions in order to earn its name as an interactive soap dispenser.
- Automatically dispenses soap
- Tracks hands of participants using machine learning
- Plays interactive game utilizing machine learning in order to teach children how to wash their hands properly
In addition, other functions can ideally be added upon in order to further the capabilities of SOAPI. However, these are the main tasks it is able to perform currently.
How I built it
For the Soap Dispensor (hardware)
I heated kydex plastic and molded it around a common soap dispensor.I attached a high torque servo motor with an extended lever that presses down on the pump when getting an arduino serial command.
The command is triggered from a py game based library that is running the yolov3 and sorting the neural networks for object tracking and detection.
For SOAPI (software)
When two tracked objects (hands) are within a certain pixel distance, SOAPI knows that you are washing your hands and launches a happy animation. When two tracked objects are far apart, SOAPI knows you are not washing your hands and launches a sad animation.
Demonstration of tracking individual persons in preparation of separating right and left hands apart
If it is even further, pixel distance wise, SOAPI is frustrated. In terms of speed, the program is multi-threaded to run the neural network and animation handler/arduino controller in parallel.
Demonstration of animation for py-game
Challenges I ran into
- Getting Cuda Nvidia GPU to work with the repository for the yolov3 algorithm.
- Converting neural network weights to the right format.
- Learning py-game.
Accomplishments that I'm proud of
- Managing the run the yolov3 algorithm.
- Training my own neural network weights to subsitute for yolov3.
- Interfacing the hardware, neural network, and software to work together.
What I learned
I learned machine learning and computer vision in my first attempt. This will open up future opportunities for custom image detection in later projects that I hope to venture into in order to improve current health problems in the world.
What's next for Soapi - The Interactive Soap Dispenser
- To make the image detection use the trained weights I created for hands.
- Depth sensing to auto dispense soap.
- Vibration for more interaction and personality.
- Gearing the pump down so that it takes less force to push down soap.
arduino, machine-learning, python