Authors

Inspiration

Originally, XSight idea is inspired by following. Time is our most precious commodity. It doesn’t matter where you are, who you are or how rich you are. Everyone has the same fixed supply of time: 24 hours. And we believe that XSight major value is that: it’s going to save us time. Like, where is your friend in the people crowd? Or who is this pretty person on the street? Or which sedan on the street is your Uber cab?

It is almost impossible right now to interact online based on what you see in the real world.

And this is what we are trying to solve at XSight.  Our mission is to merge virtual and real worlds.

During pandemic times one thing that drives people crazy - is when they see other people not respecting social distancing. I think some stores are doing a great job with "social distancing" themed stickers on the floor and on the walls, but some other stores and places haven't done much. So why don't we show the distance to every person in a frame and the distance between people in a frame using smartphone camera in realtime? This will save time to our users to identify, whether it is safe to stand where they stand now, and to understand if the path they are walking is safe in terms of respecting social distancing. For example, if you can see two persons standing on the street and talking, and it's less than 2 meters distance between them - you'd better walk on another side of this street to minimise your chances of getting infected. In other words, we want to save people time by showing them distances between people, so people can just use XSight Social Distancing Camera instead of calculating approximate sitances in their heads on the fly.

What it does

XSight is a hyper-local social app that lets you connect with people and places around you - using just your camera. Point your camera to a person on the street, and you'll see his/her social profile hologram floating on top of his/her head in real-time, click this hologram and message this person instantly.

While XSight itself can help connecting people without close contact, we want to make a step further and introduce a new mode for our app - SocialDistancing Camera. We will add two features:

  1. Displaying distance from current user to every person in the frame.
  2. Displaying distance between people in the frame. If this distance is less than recommended for social distancing - this information will be highlighted with red colour.

How we built it

XSight is a native mobile app available for Android and iOS. We use variety of different tools and techniques to achieve 2 cm accuracy.  Our key advantage - we can display information not only mapped to static objects like cafes or museums, but also mapped to dynamic real-time moving objects in the camera viewfinder. We use Sensor Fusion, Machine Learning and Computer Vision algorithms to achieve this.

Challenges we ran into

Smartphone hardware performance was a real challenge. We had to improve our algorithms performance 20 times, compared to where we've started. Currently we can achieve close to 15 frames per second.

Accomplishments that we're proud of

We talked to many prospective users and found out that 95% of our targeted users are looking forward to use an app like XSight - according to our research.

We have 50.000 lines of code, hundreds of design drafts, one interactive prototype, one website and video demo of the product. Looking forward to our first release Android and iOS apps any time soon!

What we learned

We learned that users are always concerned about privacy. Every second prospective user asks us - "what if I don't want to be recognizable now?". And the answer to this - of course we will respect privacy settings. As a user you can choose to be publicly recognizable by anyone or only by your friends.

What's next for XSight Social Distancing Camera

Next step for XSight Social Distancing Camera will be to use this information about distance to and between people in the frame and display even more relevant data on top of the real world - display a safe way. So the user can follow our hints to walk his path in a way that reduces his/her chances to be infected.

Try It out

Hackathons

Technologies

android, c#, computer-vision, firebase, ios, java, kotlin, ml, swift, tensorflow, unity

Devpost Software Identifier

259557