After successfully working on CovidAPI.info, a set of APIs related to Covid-19, empowering developers, hackers and makers across the globe, handling over 3.5 million API calls in a short span of 3 weeks, we were looking for a way to integrate a chatbot which replies to user's queries regarding Covid-19
The main motive behind this site was to try to help curb the spread of rumors and unverified data circulated through many social media sites and applications. The tragic death of over 600 people in Iran after following instructions from a baseless rumor to drink industrial alcohol to help protect from the virus was a turning point for us. That moment onwards we dedicate our time, energy and thoughts on this project with our heart and soul.
Being from a country with over 1.3 Billion people, in general, there used to be a spread of fake news and information which has increased with the unknown of the pandemic around us. This is what we wanted to reduce by empowering people with the right factual information to fight it.
As independent developers, we faced two main challenges in building a chatbot:
1) Sustain the costs of running a chatbot for a few months at scale 2) Scattered data regarding Covid-19 across the internet
We brainstormed the architecture for the chatbot and soon realized our monetary constraints in scaling it would be a bottleneck.
We didn't give up here, the fight had to go on. There was a larger problem to solve.
We soon realized that other developers wanting to build something like this too would be facing a similar problem. That's what also went through our minds during our previous project - CovidAPI.
We decided to tackle this issue by building a FAQ based dataset on Covid, sourced only from reliable health organizations and government advisories.
One of the main reasons for us to continue until now and even further is that even though we couldn't build our dream application, some developer, somewhere around the globe with enough resources could build using our data. This is what makes us tick
What it does
In technical terms, it's a search index on COVID-19 related FAQs to enable users to distinguish between fake rumors from real information.
We fetch data from government advisory bodies and medical institutes like CDC, WHO, UN, Harvard, etc. A full list is available here on our site.
Once the FAQ is added, a user could simply put in keywords of their query and immediately would be able to see matching results.
Once on the FAQ page, they can see the source of that information and a link to it. Also, they could easily share the page on social media platforms to help factually fight misconceptions about the virus.
Also, recently we have partnered with a bunch of doctors who are collaborating with us in this endeavor. As soon as we get any questions from our users, we pass it on to our doctors and they respond to the query.
We built the site keeping in mind the goal of making it as a PWA. As a mobile user, you would be asked to add the site to the home screen if your device is supported and you would be able to access the site as any other app.
Along with this, we have built it in a cache-first way thereby enabling users to still visit the previously accessed pages even without the internet.
How we built it
We started with the thought of building this application from scratch in a web framework in Python. We had an ambitious goal of serving millions of users across the globe.
To achieve this on our custom application we would've had to create an infrastructure that is heavily cached, serving static assets via CDN, invalidates cache on new builds and so on.
Now, the challenge is not in building this as we have done it in the past. Challenge was in making it cost-effective as we had to invest in it ourselves.
We Indians, have a way of doing things called Jugaad which means a flexible approach to problem-solving that innovatively uses limited resources.
We took the "serverless" approach here. We thought of using a static site generator to simply generate a bunch of HTML pages pieced together. Along with this we also wanted to generate an index of our pages during the build and later use it using a library like [lunr.js] on the frontend and provide an instant result for any query.
We had thought of using a service like Netlify or Now.sh which would help us setting up Continuous integration in our workflow easily, power our application with globally distributed CDN and allow us to focus on building our application while letting it scale easily.
We took on the task and decided to use Jekyll along with base a theme made with UIKit and chose Netlify as our platform.
We had our site up and running soon and then we started with putting in our FAQ.
Challenges we ran into
Oh boy! Our project itself started with challenges.
Not having resources to build a scalable application was our main challenge but we successfully pivoted the application into something which provides similar functionality.
Not having enough dataset which we solved by uh.... building this application
We had always wanted to build a proper PWA application but never had a chance to do so. This was a challenge that took some time to overcome and fine-tune.
Accomplishments that we're proud of
Privacy being paramount to us, we wanted to enable proper privacy protection to our users. Keeping this in mind, we ensured that none of our tracking scripts would be loaded unless a user explicitly allows cookies. Also, we took the time to explain what our cookies do and what cookies are in our privacy page
We did it! Finally, we were able to successfully implement PWA in our application. This enabled us to give our users a native-like experience on almost any platform. Also, we were able to successfully able to convert our PWA application to an APK which could be installable via Play Store.
However, due to restrictions on apps around Covid, we weren't able to get it up and hope to have it up running on a play store near you pretty soon!
Remote Working was another thing that we were very proud of. Having not worked remotely on such applications it was a wonderful experience and we are happy to have collaborated successfully! A quick shoutout to Google Hangouts!
The website structure as been kept simple scrapers in mind. The infrastructure we've chosen enables us to serve at scale thus we don't mind scrapers and have explicitly encouraged them in the disclaimer page
What we learned
Not getting bogged down by the challenges that life throws your way and having the mindset to adapt your goals according to resources available is one of the major things we learned.
You don't need to reinvent the wheel, but you have to know how it works.
Build solutions if you don't find any.
We hate working remotely, yet we collaborated successfully
What's next for CovidFAQ
This is just the start of the road for us. There's still a lot more features to come, a lot more over-caffeinated sleepless nights to go and a lot of users to reach and empower.
One of the main things for us is a section called Myth Busters where we debunk myths floating around Covid all while being powered by data from reliable sources. Here's an exclusive sneak-peek from our secret development systems
P.S: There are a few CSS tweaks left here and there
Another major feature in the pipeline is making this truly multi-lingual. Not just adding a script to enable Google Translation, we want the user to even search in any supported language.
Here's a sneak-peak into this feature available on port 4000 of any local system around you by following the instructions here
The language shown here is Hindi
Coming from a nation with 20+ official languages and a country where English is not as known as regional languages, this was a must-have feature for us. We are planning to go multi-lingual with two languages to start with.
A major bottleneck that we are currently solving is sourcing volunteers who can help translate these FAQ into multiple languages. We have set up a workflow powered by Google Translate to help our volunteers with the translation.
Having a few doctors onboard, we are planning to global with this initiative by partnering with doctors from around the world. We are in talks with a few doctors and looking forward to working with them very soon in solving the queries of users.
We are actively looking for developers as well as volunteers to translate this from English to their respective languages.
DATA - The reason why this whole project started. This website serves information to normal users directly. However, to enable developers we'll be opening up our APIs soon so that they could have direct access to globally sourced FAQ.
Try It out