Apple has packed an interesting new accessibility feature into the latest beta of iOS: a system that detects the presence of and distance to people in the view of the iPhone\u2019s camera, so blind users can social distance effectively, among many other things.
\nThe feature emerged from Apple\u2019s ARKit, for which the company developed \u201cpeople occlusion,\u201d which detects people\u2019s shapes and lets virtual items pass in front of and behind them. The accessibility team realized that this, combined with the accurate distance measurements provided by the lidar units on the iPhone 12 Pro and Pro Max, could be an extremely useful tool for anyone with a visual impairment.
\nOf course during the pandemic one immediately thinks of the idea of keeping six feet away from other people. But knowing where others are and how far away is a basic visual task that we use all the time to plan where we walk, which line we get in at the store, whether to cross the street and so on.<\/p>\n
The new feature, which will be part of the Magnifier app, uses the lidar and wide-angle camera of the Pro and Pro Max, giving feedback to the user in a variety of ways.<\/p>\n
The lidar in the iPhone 12 Pro shows up in this infrared video. Each dot reports back the precise distance of what it reflects off of.<\/p>\n
First, it tells the user whether there are people in view at all. If someone is there, it will then say how far away the closest person is in feet or meters, updating regularly as they approach or move further away. The sound corresponds in stereo to the direction the person is in the camera\u2019s view.
\nSecond, it allows the user to set tones corresponding to certain distances. For example, if they set the distance at six feet, they\u2019ll hear one tone if a person is more than six feet away, another if they\u2019re inside that range. After all, not everyone wants a constant feed of exact distances if all they care about is staying two paces away.
\nThe third feature, perhaps extra useful for folks who have both visual and hearing impairments, is a haptic pulse that goes faster as a person gets closer.
\nLast is a visual feature for people who need a little help discerning the world around them, an arrow that points to the detected person on the screen. Blindness is a spectrum, after all, and any number of vision problems could make a person want a bit of help in that regard.<\/p>\n
As ADA turns 30, tech is just getting started helping people with disabilities<\/p>\n
The system requires a decent image on the wide-angle camera, so it won\u2019t work in pitch darkness. And while the restriction of the feature to the high end of the iPhone line reduces the reach somewhat, the constantly increasing utility of such a device as a sort of vision prosthetic likely makes the investment in the hardware more palatable to people who need it.
\nHere\u2019s how it works so far:<\/p>\n
Here\u2019s how people detection works in iOS 14.2 beta \u2013 the voiceover support is a tiny bit buggy but still super cool https:\/\/t.co\/vCyX2wYfx3 pic.twitter.com\/e8V4zMeC5C
\n\u2014 Matthew Panzarino (@panzer) October 31, 2020<\/p>\n
This is far from the first tool like this \u2014 many phones and dedicated devices have features for finding objects and people, but it\u2019s not often that it comes baked in as a standard feature.
\nPeople detection should be available to iPhone 12 Pro and Pro Max running the iOS 14.2 release candidate that was just made available today. Details will presumably appear soon on Apple\u2019s dedicated iPhone accessibility site.<\/p>\n
Microsoft Soundscape helps the visually impaired navigate cities<\/p>\n