Beeps from the left, beeps from the right: a Harvard phone app is tested as a walking guide for blind people

Problem

Most people walk to the bus stop or a friend’s house without planning it. For someone who is blind, the same trip means knowing which way to turn, staying on the sidewalk and getting past whatever is in the way.

About 340 million people around the world live with blindness or moderate-to-severe visual impairment, which limits how freely they can get around outdoors.

Before building anything, a team at the Harvard John A. Paulson School of Engineering and Applied Sciences (SEAS) asked people what they needed. They surveyed 112 blind and low-vision people. The answer came down to three things: turn-by-turn directions, steady guidance along sidewalks and paths, and help detecting and avoiding obstacles.

Existing tools each cover part of that list. White canes, guide dogs and electronic travel aids all help, but they rarely do all three, and some are expensive or hard to get. Map apps give turn-by-turn directions, but satellite positioning (GPS) is not accurate enough for a blind person to rely on alone, the Harvard team says.

Most people, though, already carry a smartphone, and the team set out to build on that.

AI solution

To use Mobilio, the walker wears the phone on their chest, puts on open-ear headphones, which leave the ears uncovered, and keeps using their white cane.

The app combines the phone’s camera, its GPS, its motion sensors and, on phones that have one, a laser depth sensor. All of the processing happens on the phone itself.

The AI at the center is a computer-vision model that looks at the live camera image and labels what it sees: sidewalk, crosswalk, road and other surfaces. This is called semantic segmentation. Raymond Liu, the Ph.D. student who led the work, trained the model on pictures taken from a walker’s point of view. Most public street-image collections are shot from cars, so models trained on them often misread the things that matter to someone on foot.

“There’s a need for more data that comes from the perspective of humans walking along sidewalks, or walking outdoors,” Liu told Harvard.

Patrick Slade, the assistant professor of bioengineering who leads the lab, compares the system to a self-driving car: “Our system uses the sensors in a smartphone, GPS information, and the actual motion of the person to create what’s essentially a small autonomous vehicle plan on how to navigate them from where they are to where they want to go.”

Instead of only speaking instructions, Mobilio plays a steady stream of beeps that seem to come from the left or the right, to show which way to steer. As the person walks, the app checks how closely they follow the beeps and adjusts the cues and pitch patterns to suit them. Researchers call this “human-in-the-loop” tuning.

The team has published the navigation code and the study data on GitHub, the code-sharing site, under an open MIT license. It is research code built with the Unity game engine, not an app people can download.

Impact

Mobilio is a pilot. Its main evidence is one peer-reviewed study, published in Nature Biomedical Engineering on August 24, 2026 and run by the people who built the app.

Fourteen volunteers took part: nine who are blind and five with a visual impairment, recruited through the Carroll Center for the Blind in Newton, Massachusetts. Each person did the tasks twice, once with Google Maps directions plus a white cane and once with Mobilio plus a cane.

On an outdoor community path, people finished 13% faster with Mobilio, on average. They also needed about 72% fewer verbal corrections along the way, according to the journal’s press release.

On an obstacle course, participants touched obstacles and other things around them 41% less often, on average, but took about 17% longer to finish. Harvard describes this course as indoors. The wording of the paper’s summary leaves some doubt about which task the 41% belongs to, and the full paper is behind a paywall.

The study also found that Mobilio guided people to their destination about as reliably as a human guide did. In surveys, participants said the app was easy to use, did not demand much effort and gave audio cues that made sense to them.

The authors are open about what remains to be tested: more kinds of tasks, more places, different weather and longer use in daily life. Next, the team plans tests in a wider range of real settings around Boston.

People

Liu has an older brother who is blind, and helping his brother get around on his own strongly shaped the goals of the project, Harvard reports.

“Having actual people test out your device is absolutely crucial because it’s impossible to predict how somebody is going to react to your device. Especially within blindness and visual impairment, which is a huge, diverse set of impairments,” Liu said.

The 14 volunteers from the Carroll Center for the Blind walked the routes and the obstacle course and let the app tune its sounds to each of them. We found no public account from any of them.

The work has support from the Harvard Grid Accelerator, which helps turn research into products. Harvard’s Office of Technology Development has protected the inventions and is looking for ways to commercialize them.

Why it matters

A navigation aid that runs on an ordinary phone, with no extra hardware, could reach far more people than costly specialist devices. Mobilio’s first test is small, but it shows that guidance by sound can help people walk with fewer bumps, and the team has shared its code and data for others to build on. Whether it works in daily life, and when people can get it, is still to be seen.

What we could not confirm (7)

• Exactly which task the 41% drop in obstacle contacts comes from. The Harvard and journal press releases put it on the obstacle course (Harvard calls it the indoor course), while the wording of the paper’s abstract could be read as covering the outdoor path. The full paper is behind a paywall.
• Who gave the “verbal corrections” counted on the outdoor path, and how they were counted. The figure comes from the journal’s press release.
• Whether the obstacle-course result would hold outdoors, in bad weather or over weeks of daily use. The authors say this still needs testing.
• When, or whether, Mobilio will be available to download, what it will cost, which phones it will run on, and which company will bring it to market.
• Any participant’s own account of using the app. We found no public statement from a volunteer or from the Carroll Center for the Blind.
• Any independent test. The only study so far is by the app’s developers, with 14 people in the Boston area.
• There is no image of Mobilio in this story: we have no licensed photo of it and no public app to screenshot, so both pictures show the setting only.