Authors

Inspiration

After seeing news reports of overcrowded hospitals in various countries and trying to understand the risk that healthcare professionals are exposed to, we felt it was necessary to control the crowd at the hospitals and only let the critical patients be administered.

For the efficient management of this COVID situation, it is necessary to prioritize who needs care and who does not. Today, it’s very difficult to do this because there are a lot of patients and not enough doctors. And this leads to overcrowding of hospitals and clinics and brings about an imbalance in the healthcare system. Therefore, it is necessary to handle this situation by analyzing the health of the patient at their homes itself. It also prevents healthcare professionals from being directly exposed to the virus, which is a huge advantage. Because of the doctors and nurses get sick, then our first line of defense is lost

What it does

TeleVital screens the possibilities of having Covid-19 through chatbot support, measuring vitals (heart rate, respiratory rate, and SpO2 levels) remotely, prioritizing the patients’ hospital admission based on their health and other factors, COVID X-Ray analyzer for quick and cost-effective analysis and a digital prescription for a contactless prescription. Components of TeleVital Platform

1. AI Chatbot:

Our bot is helping the society as it includes a COVID-19 assessment test which is a guiding tool for people to go over their coronavirus assessment all by themselves. This is a great way for users to understand if they need to visit a COVID-19 testing center based on their symptoms right at the comfort of their homes. It categorizes users into low, moderate, and high-risk zones and prompts them to take the vitals test if they lie in a moderate or high-risk zone.

Preventive Measures: In addition to providing the Covid-19 assessment service, this bot also includes the safety measures and procedures to help people during this pandemic.

Mental Therapy: Today, as the need for mental health services continues to surpass availability, people in distress can reach out online to mental health chatbots and one such as our TeleVital bot. What makes it unique is it’s ability to listen and chat anytime, anywhere, and recommend activities that improve the users’ wellbeing, yet providing privacy. This bot is an automated conversational agent that is specialized in monitoring moods of the user.

Integrations: TeleVital Bot is also integrated with WhatsApp and Facebook to help and reach out to people on a larger scale during this pandemic and reduce the load on the health systems.

2. Vitals Test:

SpO2 : Peripheral capillary oxygen saturation (SpO2), which is an estimate of the amount of oxygen in the blood is one of the key vitals to be monitored. For SpO2 calculation, the behaviour of components of white light is monitored when a finger is placed between a source of white light and the camera. The captured frame is pre-processed by splitting it into red and blue colour channels. Then specific numerical computations involving the mean and standard deviations are carried out on these channels. These give out the concentration of oxygen in blood in percentage.

Respiratory Rate: The number of breaths taken by a person in a minute is termed as the respiratory rate for the person. For the calculation of respiratory rate, the oscillations of the face while inhaling and exhaling have to be studied for relative displacement. The first step in this process is capturing frames of the face of the person to be tested. Then the frames are sampled to get a clear picture of the change in pixel intensities. Then the frames are pre-processed to find the required template which has the least noise and sharper features. The region of interest (ROI) is determined from the template. Then basic signal processing techniques are applied to the ROI to extract the frequency of the change in intensity of the pixel. The most dominant frequency is selected as the respiratory rate and is displayed in frequency per minute.

Heart Rate: Heart rate is the number of a person’s heartbeats per minute. To determine the heart rate, image frames containing the person’s face are read. Then a monitoring algorithm for face detection is applied to select the region of interest (ROI). Classifiers are used on the frame to detect the coordinates of boundary for ROI selection and hence the region of interest is acquired. Then RGB signal extraction is carried out on the frame for accurate computations. Then the ROI is put under an algorithm of signal detrending to remove drifts and noise. On the resulting frame, some filtering and normalization are carried out along with matrix calculations. These calculations result in heart rate in beats per minute (bpm).

3. COVID X-Ray Analyser:

Covid-19 X-Ray Analyzer is an AI-based analysis platform to bring down the cost of Covid-19 testing. This analyzer requires only the patient's chest x-ray which would cost about £50 against the standard cost of £249. We have trained a deep learning model using Keras and TensorFlow to predict COVID-19 in our image dataset.

4. Digital Prescription:

To ease doctors with providing patients, hand-written prescriptions, the digital prescription portal enables the doctors to voice record prescriptions for patients. When the doctor visits the website, they can record their voice by clicking on the record button and record all necessary details like name, age, the gender of the patient, their symptoms, the diagnosis, the prescription, some medical advice, etc. Then this audio file is converted into text, which is filtered and displayed in text boxes namely age, name, sex, symptoms, diagnosis, prescription, and advice. If the doctor needs to edit/add something, they can do it here manually, and then finally they can generate the pdf of the report by clicking on the generate PDF button which can be viewed, downloaded, or mailed to the patient's mail ID.

5. Patient Prioritizer:

Understanding the rapid increase in cases of Covid-19, the patient prioritizer portal enables hospitals in identifying the high-risk patients and giving them the care accordingly. The comparison between the patients is done based on 14 parameters namely, age, gender, height, weight, heart rate, respiratory rate, SPO2 levels, body temperature, cough, sore throat, breathing difficulty, tiredness, pre-existing medical conditions like asthma and pregnancy. Weights are assigned to every parameter (Adhering to the WHO guidelines) which are used to produce the final score of the patient. The doctor can view all the patient's name and their corresponding scores. The list is arranged in descending order. They can also search for the patient by their name. The patient with the highest score requires the maximum amount of medical attention. The doctor can edit the already registered patients’ details. They can search their patients by their name and age, and all of his/her previous records will be retrieved and displayed automatically on the screen which can be changed or deleted by the doctor.

There are three options present on the patient prioritizer webpage. First is that the doctor can add the patient's details and generate their score. Second is where the doctor can view all the patient's name and their corresponding score. The list is arranged in descending order. They can also search for the patient by their name. The patient with the highest score requires the maximum amount of medical attention. The third is to edit the already registered patients’ details. The doctor can search the patient by their name and age, and all of his/her previous records will be retrieved automatically doctor can either change it or delete it.

6.Video Conferencing:

Online Consultation from TeleVital is a service that allows users to connect with doctors through video calling facility. This facilitates the users with consultation from doctors without having to leave their homes. This also provides an easy yet reliable way for those people who have limitations on their movements due to their health condition. It ensures the safety of the doctors and also provides them with more time to devote to their duty by cutting down their travel delays.

This technology is based on WebRTC and is designed for one to one private communications. The user is required to schedule an appointment with the doctor and a report of the same is generated and sent to the doctor via email. The report holds details of the patient as well as of the scheduled appointment.

For the meeting to begin the doctor can send the details of the meeting to the user through the "share" button. This sends an autogenerated email to the patient which has the URL for the consultation website as well as the meeting id. Upon reaching the website the user receives a unique ID. In the path for the doctor's ID they have to mention the ID they received in the email and click on the call icon. They can choose either voice-only or a video call. This provides an end to end secure communication line between the user and the doctor.

How we built it

Since the issue is with monitoring the patients at the hospital, we decided to do preliminary analysis on various parameters. Therefore, we consulted a few medical professionals who are treating COVID patients and tried to understand the problem they currently face. After having a conversation with them, we felt vitals play a major role in determining if a patient needs hospitalization or not. The doctors recommended capturing vitals such as heart rate, respiratory rate, spo2, and body temperature without any contact-based measurement tools such as pulse oximeter as that would potentially increase the spread of contact-based infections/diseases.

So, the final plan we made as a team was to capture vitals, body temperature, and also try to check if the person is at possible risk of being infected / a carrier. We also planned to design an Xray analyzer and make prescription and consultation contactless so as to avoid any direct contact between the healthcare professionals and patients.

Since mobiles/laptops are readily available at most of the homes, we decided to use this as a tool to capture vitals and calculate other parameters via a web browser and establish a communication between the patient and the doctor via the internet. We referred to a couple of research papers to monitor vitals via web camera, analyze X-rays to get started, and ended up implementing a better algorithm than the existing ones out there.

Challenges we ran into

Initially, we had issues connecting the python script which we wrote with the web-browser, and later we managed to run it locally. Post local deployment when we tried hosting it, we had issues in streaming data via the web browser. After multiple attempts, we successfully managed to implement all this in less than 48hours. The idea of a Chatbot is an obvious one. But it’s not innovation until you actually do it. And the challenges which arise from this principle are exactly the ones which we decided to take on. The main challenge we faced was acquiring a clean, annotated COVID-19 X-ray dataset for our analysis.

Accomplishments that we're proud of

When we started this project one fine evening, we could not have foreseen just how much spirit, commitment and vigor the team would put into tackling this project. Without the unbelievable motivation of our team, we would not have been able to implement this complex project in one weekend. We are particularly proud of the technical organization of the project, with individual parts being almost seamlessly integrated into a complete prototype during development. We were initially hesitant about the idea to measure the vitals of the patients remotely and were not sure how to approach this problem. Nevertheless, we sat down and worked hard on building our product for the benefit of society. We referred to some research papers and implemented them successfully for measuring the Heart Rate, Respiratory Rate, and SpO2 concentration via a web camera. We came up with a lot of ideas to make the entire testing procedure remotely handled. With our Patient Prioritizer and Digital Prescription, we could finally achieve this. We believe these features enhance our product thereby moving the entire testing phase at the comfort of the patients.

For the hackathon, a feasible and completely normal approach would have been only to hint at large parts of the prototype and focus more on the pitch. However, we unanimously decided to lay a solid foundation for the sustainability of the project now, and especially, if possible, to put into practice those parts that were difficult to implement, at least as proof of concept. An important prerequisite for a stable system, later on, is that it should have a modular setup.

What we learned

We learned that a problem well-identified is a problem half solved and that team diversity is of utmost importance in building a foolproof product to align with the increasingly competitive market.

From a technological point of view,we learnt the following: We learned a lot about designing for scalability and also how to store and retrieve information efficiently from a relational database. It took a lot of patience and perseverance to develop this project in its entirety. Not only did we gain a lot of insights from developing the app components, but we also developed our soft-skills. With a great team effort, we were able to jot down the important sections of the project and implement them within the weekend. We individually took initiatives on completing different tasks and in the process, we were able to come up with a great prototype to help our society.

What's next for TeleVital

After the hackathon is over, we seriously look forward to making the vitals portion more accurate through clinical testing, publish a paper regarding the same, and converting it into a product. This is not only for the momentary crisis but also for a long term basis and is definitely worth the efforts. We would look forward to integrating the patient prioritizer and the digital prescription with the hospital’s existing system to help the professionals determine whom to admit in case of resource shortages. We plan to integrate facilities for measurement of different parameters like height, weight, etc. [through augmented reality] so that the reach of this application goes beyond COVID.

Try It out

Hackathons

Technologies

bootstrap, css, firebase, flask, gcp, google-cloud, heroku, html5, javascript, keras, machine-learning, opencv, python, react-native, tensorflow

Devpost Software Identifier

267009