Emerging from a senior thesis project by Thomas Bao '12, CarSafe, an Android application that promotes driver safety, has since grown into a collaborative international project that represents the first application to use a smartphone's dual camera data to analyze a person's driving habits, according to computer science professor Andrew Campbell, faculty advisor to the department's Smartphone Sensing Group.
The application analyzes drivers' physical motions, such as head turning and blinking rates, as well as the safety of their driving, monitoring their following distance and lane changing, Campbell said.
"The goal of this project is to detect dangerous driving conditions using dual camera phones," Bing You, a visiting researcher from Taiwan's Academia Sinica, said. "The application detects both road-side and driver-side information. For the driver-side, we measure the blink duration of the driver to see if he is tired."
Although the project is now mainly run by You, Bao's thesis idea was the impetus behind the application.
"Thomas really did the first breakthrough part of this application," Campbell said. "He was definitely the one who pushed ahead with it and did the early hacking."
Campbell thought that driver safety would be an interesting topic to explore since he had an idea that a phone should be a "safety blanket" around its user. Bao did some research and decided to focus on driver drowsiness, which can be determined by measuring long blinks, or periods of time when the driver falls into "micro-sleep," Campbell said.
"I mainly focused on head direction and detecting blinks," Bao said. "There's quite a bit of work on finding a person's face and locating their eyes."
Bao began the project knowing little about machine learning, computer vision or Java, which is the basis of Android programming, he said. He studied machine learning and computer vision using a library of computer algorithms called OpenCV but had to learn how to use Java and phone programming as the project moved forward.
"There's usually some solution or tutorial on the web, and if you can't find one, you have to work around it," he said. "It's all very hacky not well-structured or beautiful."
Computer science professor and faculty director of the department's Visual Learning Group Lorenzo Torresani advised Bao and currently advises You on analyzing the visual patterns of the smartphone video, Campbell said.
"It's a very challenging project because the quality of the images taken from the phone camera is not great," Torresani said. "There are continuous changes in illumination, so being able to design something that is robust, that is able to function continuously in different environmental settings, is very challenging."
You has integrated Bao's component of measuring driver-side data to the larger CarSafe application, which also uses road-side inputs to alert drivers about potentially dangerous situations, such as unsafe following distances, according to Torresani.
The project team ran into issues with using smartphones' front and back cameras simultaneously, since the phones do not have platforms to support this function, according to Campbell.
"We have to come up with new techniques to switch from one camera to another camera, and there's a cost of doing that because if you switch to the front camera, you have to close down the back camera pipeline of data," he said.
The application was developed with virtually no outside funding, according to Bao. Bao said that Campbell's smartphone lab had all of the equipment and technology needed to develop his component of the application.
CarSafe will be released on the Android market next year, and the team is currently working on streamlining the system to improve its accuracy and efficiency, according to You.
Torresani said that the application has high potential because luxury car brands are already trying to incorporate similar systems into their vehicles at much higher costs.
"They have come up with very expensive solutions, which can only be afforded by people who can buy luxury cars," he said. "But now, we have a solution that essentially reuses what most people already have."
Although undergraduates have limited time to work on a research project, they bring great interest and energy to their projects, Torresani said.
"In my experience, in order to be successful in projects involving undergraduates, you need to make sure the problem is very well-defined from the beginning," he said. "In this case, we were able to identify a problem that was narrow enough and interesting enough that [Bao] could make a contribution in a limited time."
Bao is now working at Evolution Capital Management, a Hawaii hedge fund. Although he is not currently working in programming, he said his computer science experience has improved his critical thinking skills.
"The way you learn to think is helpful problem solving, logical reasoning and the ability to drill down and really understand something complicated," he said.
