The Blinking Eye. Big Data – the comprehensive collection and analysis of data – and ML offer a powerful combination to sustain the effective and efficient management of modern urban societies .Employed well, it facilitates nowcasting and forecasting, greasing the city machinery. This improves decision-making by all relevant urban stakeholder, from policy makers to businesses to city dwellers. The “eye of the city” is both promise and realization of stability, predictability and evidence-based rationality in the practical governance of daily urban life – a continuation of a dream of enlightenment that originated in European cities some three hundred years ago and that inspired generations of urban policy.
The implicit assumption of data-driven algorithmic decision-making, however, is that the past reflects the present and informs the future. At best, this notion applies in times of incremental changes and clearly defined overall objectives: if the goal is clear and reality changes slowly, data can help decision-makers optimize their choices. It breaks down, though, when changes are rapid and the overall trajectory not only fuzzy, but substantially unpredictable. In such phases of radical change and unclear direction, conventional data-driven planning breaks down, and so does data-driven ML. There simply isn’t enough training data available to gain sufficient clarity of how the future will look.
Ironically, the very tools that enable data-driven predictions and rational decision-making and promise stability through predictability have been fueling a series of fundamental and rapid disruptions of both economy and society: electronic commerce through data-rich markets that far exceed the matching and transaction efficiency of conventional markets; social networking platforms that seamlessly connect people with each other at scale; increased automation of routine human decision-making tasks that endanger what had been perceived to be “safe” human jobs, such as mid-level office work. Experts disagree on the size of the impact this has on labor markets and employment, with displacement figures of about fifty percent being touted . But there is wide-spread agreement that the skills needs and job demands over the next five to ten years will be very different from the supply available. This may exacerbate tensions within society, undermining stability and prompting social unrest.
With such radical disruptions in the air, data-driven predictions will have limited value to stabilize human societies, especially when many humans have to live together in tight spaces, such as in the urban context. Because times of rapid change aren’t predictable, and because the interventions the predictions suggest may prove to be ineffective. Therefore, there is an urgent need for an alternative planning strategy for an age of urban disruption.
Fortunately, such a strategy is available, and we can glean the core determinants of it by looking at how nature reacts to radical shocks. When life is challenged in this way, evolution switches from a relatively linear process of steady progress to an all-out dynamic of massive experimentation, creating an enormous richness of variation aimed at maximizing the chances that at least one variant may do well under the changed circumstances. The rapid, almost explosive blooming of many different species in times of disruption may look enormously costly – after all, many such trials will end up being errors, many new variations of species will not survive – but is incredible efficient for life overall: its goal is to ensure the survival of life itself.