How Not to Lie with Big Data
Waterfront of Portland, Oregon Photo by Sean Benesh on Unsplash
Readers of the previous newsletter on two ways that transportation planning misses the mark might enjoy this one about how to tell truths in transportation. Data platforms make big data for transportation decisions easy. Users can create a login, perhaps pay a fee, and download what would have been very difficult, if not impossible to get only a few years ago. Those of us who work in transportation planning, academia, or are just interested in how good decisions get made, are often told opposing versions of the same story:
Big data helps us answer previously unknowable problems, fast, and with little cost; and
Transportation data platforms are biased, over-representing high-income earners in wealthy countries, and are more often male.
Over a decade of research supports both the advantages and problems of big data to be true (e.g. danah boyd & Kate Crawford, 2012; Jeremy Crampton et al., 2013; Geoff Boeing, 2021) yet transportation planners and others can find themselves caught between proponents of each claim. Big data for transportation planning is too often presented as a one-sided argument either way, offered by major data companies and even critical scholars. Thinking about how best to plan for bicycling and other sustainable transportation solutions has spanned my career, beginning in Central Texas a quarter-century ago. Vignettes from my experience and recent research helps illustrate these seemingly conflicting approaches to big data.
Platforms Fill Gaps for Decisions
Working as a planner at Capital Area Metropolitan Planning Organization (CAMPO) over fifteen years ago, we were responsible for creating a process to allocate grants from federal Surface Transportation Program funding, including bicycle and pedestrian projects. The application for general roadway projects included safety and demand factors, including existing traffic levels. The bike/ped version addressed proximity to schools, but there was no data on existing traffic the same way local and state governments collected it for motor vehicles. Hence, some projects were funded in far-flung locations that might not get as much use if we included existing or forecasted travel demand. That's when I started experimenting with using American Community Survey commute data to develop Simple Techniques for Forecasting Bicycle and Pedestrian Demand (2009), and created the first active travel monitoring plan for the region (2011). Neither effort resulted in a proper 'count' of bicycle traffic for every road in the region, but I knew that emerging platforms like Strava could provide some data for every segment, even if it would not represent much of the actual traffic totals. Would some data be better than no data? I had recognized that my practical research could have as much or more impact than in strict planning practice, and so joined Texas A&M Transportation Institute (TTI).
As a sometimes-competitive cyclist, I used Strava to track rides, and my TTI colleague Shawn Turner inspired me to ask the provider for data to evaluate. Some calls with Brian Riordan and others at Strava at the time turned into a working approach to share data that I used to calculate the proportions of trail users in Austin using Strava against a full count (2015), and then to describe the routing choices of fitness-oriented bicyclists (2015). These two peer-reviewed publications were among the first using Strava data to support bicycle planning, and showed value for this type of emerging big data. My dissertation project included data from a then-emerging provider at the time Ride Report, and interviews with thirty-three informants from around the globe, tracing the social construction of crowdsourcing technologies for bicycle planning.
Strava went on to develop the Metro platform to sell and later give away data, and just yesterday announced an approach to integrate stationary counting stations with Metro to soon provide what they'll call "Total Trip Estimates." I'm excited to see that work finally deployed after lots of great research with Shawn Turner, Joan German Hudson and others in 2018, Ipek Nese Sener, and Trisalyn Nelson and her teams often working parallel with ours. All of these studies showed the usefulness of the data, along with potential for the younger-male-skewed Strava demographic to distort bicycle traffic data if not balanced in some way.
Algorithmic Bias Blurs Reality
Critical geographers and media studies scholars have led thinking on how pervasive algorithms and hyper-automated decision processes can impact cities with real-world consequences. The fundamental problem is that big data doesn't track people, it tracks the high-tech devices that some of us carry or use. I think Richard Shearmur said it best in 2015 (p. 967): "however big the data, Big Data is not about society, but about users and markets." Just as Strava data only reflects the people with spare time, money, and interest in self-tracking, logged trips miss the vast majority of bicycling, including those for commutes, shopping, and any by the vast majority of people who don't conflate bicycling with a social media opportunity.
Other platforms like StreetLight take a different approach to monitoring bicycle traffic, though their methods are proprietary and not open to external analysis. By using triangulated cell-service 'pings' between three or more cell phone towers, data providers can approximate locations and speeds similar to GPS, mash that data into a behavioral model, and make (educated?) guesses about which travel modes people use. Sharing personal trip data has become an industry default, rather than an opt-in service.
A new study reminded me of how important telling, and re-telling these truths are. Drs. Shaun Williams and Frauke Behrendt just published 'Data visibility of cyclists: social justice implications of Strava Metro data in transport planning' in the peer-reviewed journal Mobilities. Interviews with seven informants reveal a "tension between data validity and data bias" that reinforce most of my previous findings, while suggesting a new stream of research on how this data could be fed into AI platforms, potentially obscuring and scaling non-representative data for making real-world decisions. As a researcher and editor of the journal Mobilites that published the study, transportation geographer Julie Cidell continues to move the field toward more equitable and just futures.
Other than spending the time and money to develop robust active travel monitoring programs with individual count stations, we now we have two major options out there for solving my original problem for where to prioritize limited funding for bicycling infrastructure: balance trip-logging platforms with other data to reduce inherent biases, or use a black-box platform that uses seemingly more representative data. Whichever approach practitioners and researchers choose, reality remains buried beneath the zeros and ones—which might still be better than having no information at all.
Telling Truths with Big Transportation Data
Users of any big data platform should engage with the difference between what knowledge the data provides and reality, and include that context with any communication of findings or decisions. Telling these truths requires recording and describing what we do and don't know about the validity, reliability, and appropriateness of big data for transportation decisions.
The most direct way to validate big data traffic counts is to compare results against a small count. For bicycling, that might include using an app like CounterPoint to record and share field-based observations with counts from a data platform on the same segment. Qualitative validation could be just as important. Ask users of the platform how they record trips or opt-out of recording. Talk to data scientists about the real process of translating raw data into the product. Digging further, we often find that most 'Big Data' services are really highly-processed information that involved many human decisions along the way. Perhaps a stretched analogy is that corn flakes cereal is not corn, but there's some in there! Let's read the ingredients of our big data before we serve it.
If we validate a dataset in one place and time, does that transfer to others? Reliability deals with whether we can use the same approach in different contexts to find correct answers. I expect that Strava's implementation of "Total Trip Estimates" will only use local bicycle traffic counts to scale Strava counts accurately, but how will they determine how many local counts to use, and how far away this is appropriate to scale Strava counts across a city network? Can scaling values from Minneapolis be used in Saint Paul? If so, are daytime trends similar to night, or between summer and winter? Again, both quantitative data and interviews with experts would be useful. The end of all of these questions has to arrive with whether an approach is accurate and practical.
An appropriate application of big data is fundamentally more about professional ethics, rather than data availability. Just because we can buy a service, does not mean we should use it to decide which community gets a safe shared-use path and which does not. In the second chapter of the book Transport Truths, I compare how the codes of ethics vary between planners, engineers, and public administrators, showing that what one field finds correct and appropriate may not match our colleagues'. Communities, taxpayers, and elected officials deserve to know what they're getting from transportation planners, engineers, and builders.
So, jumping on the bandwagon of big data and AI might solve problems, but it could worsen inequities if taken without addressing validity, reliability and appropriateness for a given context. Prof. Lisa K. Bates's recent editorial reminds us that "A Computer Must Never Make a Planning Decision" (2024). If you're a researcher or practitioner ready to dig in and find answers, great! But fear not if these issues have you second-guessing how you or your organization take the next step. Rather than framing every decision as yes or no, ask these questions of your data providers, the likely end users, and the people that could be impacted by the transportation decisions. This mixed-methods approach is at the heart of examples I use in Transport Truths that show the difference between a simple answer and complex transport realities. Present and future generations deserve truths in transportation.