- Shresth Agrawal
- Shizhe He
- Mohamed Amr
- Stephana Müller
- Muhammad Zeeshan Karamat
- Pia Eggert
- Simon Hofer
- Melia Fleischmann
- Gabriel Brandle
- Vlada Petrusenko
- Muhammad Zeshan Karamat
- Actual testing methods are cumbersome and require trained staff to be carried out. Moreover, they have a high-risk for virus aerosolization and put healthcare workers at risk of contagion
- We want that everybody can get screened at zero costs and directly from home
What it does
- We want to set a proof of concept that cough types may be detected and categorized by an AI, which via deep learning process can filter out the cough produced by Covid19.
- Everybody who is coughing shall be able to record their coughing sound through a simple web interface and learn if they likely have COVID-19 or not.
- DISCLAIMER: The diagnostics function is not live yet, as this will require a medical trial first
Limitations and Challenges
- Coughing may be one of the symptoms of COVID-19, but not the only one. It is currently unclear how accurately we can detect COVID-19 from coughing samples alone.
- The model is dependent on the data that the users are inputting. We can not guarantee that the data is error-free. * However, we have ideas for cleaning and verification.
- It is hard to collect verified data from COVID-19 positive tested people, as this has to be done in a clinical trial.
- It needs to be clear that this can not be used to replace a medical examination but instead shall assist the national health care systems.
- We are a group of machine learning experts, doctors, and entrepreneurs from Switzerland, Egypt, Germany, China, Ukraine, India, Pakistan
- We initially found together through a Slack group during the #codevscovid19 challenge and are working completely remotely
How we built it
- As we didn’t have the data of COVID samples from the start, we trained the deep learning model using data of cough sounds from https://osf.io/4pt2s/ . We tested the audio classification with multiple algorithms and realised that Random Forest gives the best result. We plan to retrain the model as we collect more data of the actual Samples.
- The trained model is exposed through simple flask endpoints. That can be accessed by the frontend.
- Frontend is built using bootstrap, Jquery and integrated with the backend via SSL secure http protocol to the flask server. WebRCT combined with native JS API are used to cover a broad range of devices for recording media. Web app is deployed on scalable Azure webapp with minor hacks including kubernetes runtime initialization to include custom libraries for AI.
- Azure bot framework with LUIS is used for interactive user experience for COVID19 QNA to enhance model confidence about disease prediction.
- Azure blob storage is used as a data lake for scaling data at store and mongoDB is used for maintaining meta-deta about blobs, which is connected with azure web app.
- Moreover, the model takes into account other factors which can affect the voice like age, country, gender to improve the result, this data is included in backend and considered for the result.
What we are proud of
- Working on a common mission with a team of machine learning engineers, doctors, and entrepreneurs
- Having built a functional prototype over the weekend: https://www.detect-now.org/
What we learned
- Working remotely can be very efficient
- Building a diagnostic tool requires a clinical trial
What's next for Detect Now
- This approach to cough analysis might provide a foundation towards support both clinical research on pulmonary disease at and for capturing patient outcomes
- Intriguing potential for early-warning outbreak detection in public areas
- Collect more coughing sound samples to improve the algorithm
- Look for scientists who will create a test design for clinical trial
Try It out
- COVIDathon - Decentralized AI Hackathon
- COVID-19 Global Hackathon 1.0
- Tech Takes On COVID Hackathon
bootstrap, flask, html, pyaudioanalysis, python, sklearn