Posted by Rich Sutton
We process a lot of data. Since 2008 when Skyhook provided the first positioning system on the first generation iPhone, we have generated hundreds of billions of high precision device locations from every place on earth where people use smartphones.
Our system for delivering these locations requires there to be at least one Wi-Fi access point or a couple of cell antennas in the vicinity of the mobile phone when it goes looking for it’s current position. And we need to be very familiar with the details of these signal sources. Then we can match the identifiers seen to what our database knows and add in signal strengths and other tidbits to send precise latitude, longitude coordinates back to the requesting device.
So many of those times you think you are using “GPS” from “satellites” to deliver a location to your phone (like whenever you are indoors or out of sight of open sky) you are actually checking Wi-Fi and cell ids against a database that finds you in a totally different way that has nothing to do with satellites. It’s an elegantly simple process but it happens fast and it happens a lot. Millions of times a day we get little packages from phones with some signals that we recognize and some we don’t, and the new ones get plastered into our database and the old ones get adjusted and the database heals and grows like some sort of fantastic evolving amoeba.
Eric Fischer at Mapbox does a beautiful job capturing this here.
Using just an unscrubbed snapshot of Skyhook’s access point and cell antenna databases, Eric color-grades the time since last observation of each of the nearly one billion (count them!) communication beacons and creates a sort of magical earth-at-night composition.
And it’s way more than just a pretty map; there is a ton of analytical value in here. The yellow content is freshest and most of those points have been seen in the wild within the past 30 days. The age of the data then passes down through orange, red, green and magenta to get to the oldest inventory displaying in the blue range. You can search around and find all sorts of things that look amazing (like schools and hospitals and campuses and stadiums and other places that flare and glow hot from having lots of beacons that get used a lot). But you can also find some (like points in the water or clumped in strange, incongruous locales) that make you scratch your head and say: Skyhook gives me precise location from that?
And we do. Because visualizations like these help us to analyze and identify errors and pathologies in this vast and fiendishly dynamic data that creep in relentlessly as people move and businesses relocate and everyone replaces their APs. And when cell carriers upgrade and re-number their antennas. And when mobile devices pretend to be static. And on and on.
It’s sweet data to dig into if you can get past its basic hugeness. Thanks for doing such a great job with that, Eric.