Welcome to SideTracked!
github: https://github.com/Maderlime/lahacks_2020 (see branches)
Los Angeles and many cities have pothole inconveniences that cost billions of dollars in damage and repair. We aim to utilize crowd sourcing to help cities and individuals locate potholes easier and for free.
What it does
SideTrack is an ios app that manually or autonomously reports potholes to our Oracle Database. It uses user-reports, accelerometer and gyroscope tracking, and video/photo input that is transferred into a model to determine the intensity of the pothole.
Potholes are stored on a database and available to users in a google map interface available to all users. Each pothole has individual statistics corresponding to several attributes: GPS location, Photo(s), Spatial location i.e. 2 right most lane. Ratings for various statistics: depth, size/avoidability, hiddenness, probability that a pothole is in image capture
How we built it
The app has the following components:
- Backend Receive coordinate, user data for pothole location / details Assign pothole unique ID Send pothole data w/i X mile region
- App frontend: Intro page > Driving Mode > Map interface for potholes, click on pothole > go to details page
- Used the Oracle autonomous database to store our database, providing a REST API with node.j to interact with the front end.
- Created accelerometer filter which detects potholes based on 3-dimensional change in phone accelerometer to automatically detect potholes
- Front end was developed with Xcode and Swift, used google API for map tracking. Developed API endpoints to link front and back-end
Challenges we ran into
- Machine learning: Working with big data was new to us, and we had to work with video feed rather than images. We attempted to add it but loading a pre-trained tflite model into the app and adapting it to video proved difficult (see pics). Training a model in tensorflow also proved difficult so we decided to drop ML
- Backend: Difficulty setting up Oracle cloud and with SSH, spun up autonomous database however couldn't use
- Application: Experienced crashes when opening map interface when using apple API, switched to google API.
Accomplishments that we're proud of
We completed a functional app, with landing website, minor bugs that can be fixed with more time.
What we learned
There is a lot more to image processing with Tensorflow than we thought. We were able to practice Tensorflow/CNNs for multiple object detection. For the Front end development we learned how to implement the google maps API in Xcode. Back end development we learned how to set up and host a server using Oracle Cloud.
What's next for SideTrack
- Add more community-oriented features such as a social aspect
- Implement pothole alerts for users i.e. Level 3 Pothole in highway on-ramp Auto detected potholes Potholes from map on same road/predicted path
- Network Analysis, for example: Most driven over potholes, Worst roads, central/hard to avoid location on road, Roads that receive potholes quickest after repair
- Provide information that aids pothole repair: Potholes most valuable to fix based on metrics, Most traffic vs most likely to damage vehicle if hit (severity x avoidability)
- Scheduling: Fix most potholes with least distance traveled, construction delay
- Predict upcoming regions with potholes
Applications With enough users we will be able to develop detailed pothole maps describing where and how to fix most urgent potholes based on construction delay, probability of pothole caving again, potential damage
Try It out
adobe-illustrator, cocoapods, gcp, oracle, python, sketch, sql, swift