Authors

Inspiration

The new COVID-19 virus is testing our countries, our societies and all of us human beings. 3 billion people are now living under lockdown with no clarity on when we can bring normalcy to our lives. Our lives are on hold, our livelihood threatened and our mental health under stress. In this critical moment, our question was how can we work together to solve one of the biggest crises of our lifetime? We started a few weeks ago to gather different expertise people from around the world around a common cause to use technology and big data to improve diagnosis. As some of  our team members are highly skilled in voice recognition, we came up with a concept for voice diagnostics: just provide your voice and cough samples by speaking into your smartphone microphone or computer microphone, and have the software instantly give you a pre-diagnosis!

Our vision is to become the most convenient and effective way for diagnostics and healthcare!

What it does

We want to provide an easy-to-use and highly accessible solution for ANYONE to check how probable they are infected, simply by providing voice and cough samples using the microphones on the phone or laptop.

Our AI-trained voice recognition software compares your voice and cough samples with that of known COVID-19 infected people to give you immediate results: your chance of being infected or not.

Currently the detection accuracy levels of our technology vary between 81% and 97% in comparison to traditional doctors' diagnosis. Some medical research papers on using voice diagnostics to detect other respiratory diseases indicate it is possible to achieve a high level of detection accuracy, e.g. this research paper presents findings of detecting pneumonia in children using only 5-10 cough samples per child.

The envisioned end solution will be easy to use for anyone with a device that has an Internet connection and microphone. With a simple intuitive interface, it will be easy for non-tech savvy people to use. The healthcare systems, local authorities and government could perform massive amounts of tests without physical material and healthcare intervention and in a very safe way.

As a pre-diagnosis tool, it can help to reduce pressure on the healthcare hotline, giving the hotline staff more time and more information to help callers who really need medical help.  Given the pre-diagnosis of the patient we empower physicians/doctors/nurses to prepare for receiving the patient, thus, reducing the risk of infection and load when the patient arrives at the hospital.

We also plan to make our solution available as a SaaS and API for anyone willing to integrate VoiceMed into their apps, phone lines, call centers, medical services etc.

In the future, our technology could be used to detect other diseases, and provide the most convenient and effective way for diagnostics and healthcare.

How we built it

We worked closely with pulmonologists and virologists to create a survey of questions that we need to ask to identify COVID-19 patients and we end up with a final survey. We started collecting data of COVID-19 patients as well as healthy people to train the machine. We built: 1) A website voicemed.io 2) A simple questionnaire with 3 steps to gather voice and cough samples 3) A Machine Learning / Artificial intelligence technology that analyzes cough sounds (one of the COVID-19 symptoms) and breath rate through voice recording samples

Challenges we ran into

1) Collecting a sufficient amount of training data to generate models that will give highly accurate inference results 2) Creating the right instructions for recording voice data. Many patients were recording unclear sounds, and our team spent many hours cleaning the data (the initial voice samples). 3) Challenges in voice recognition pattern

Accomplishments that we're proud of

1) Winner - Hack for Crisis Italy  2) Winner - Hack for Crisis Sweden  3) One of top 10 entries - Hack for Crisis Norway 4) Have inspired many people to continue working in the project after their initial hackathon participation.

What we learned

We read medical research papers on the possibility of using voice diagnostics to detect COVID-19 virus infection as well as how voice diagnostics has been used to detect other respiratory diseases.  We found strong scientific support for the effectiveness of voice diagnostics in detecting respiratory conditions for Parkinson’s disease and respiratory illnesses in children.  We think there is a possibility to use voice diagnostics for detecting potential COVID-19 infection cases.

What's next for VoiceMed

1) Marketing plan to get more voice samples from known COVID-19 infected persons using crowdfunding campaigns, partnerships with private healthcare, massive promotions. 2) Make our voice sample survey more accurate and tech implementation of it 3) Fundraise to make the team 100% operational. Currently people are working in this project on a volunteer basis. 4) Create a data-privacy-by-design structure, in compliance with GDPR and global laws. 5) Improving feasibility document using scientific researches, medical doctors’ knowledge and healthcare experts.

Try It out

Hackathons

Technologies

api, cpus, firebase, github, gpus, javascript, jupyter-notebooks, keras, librosa, lottie, numpy, opencv, pandas, pycharm, pydub, pysptk, python, python-speech-features, r, react, recordrtc, scipy, scss, sklearn, tensorflow, wordpress

Devpost Software Identifier

258712