Urban visual intelligence: Uncovering hidden city profiles with street view images.

A longstanding line of research in urban studies explores how cities can be understood through their appearance. However, what remains unclear is to what extent urban dwellers' everyday life can be explained by the visual clues of the urban environment. In this paper, we address this question by applying a computer vision model to 27 million street view images across 80 counties in the United States. Then, we use the spatial distribution of notable urban features identified through the street view images, such as street furniture, sidewalks, building façades, and vegetation, to predict the socioeconomic profiles of their immediate neighborhood. Our results show that these urban features alone can account for up to 83% of the variance in people's travel behavior, 62% in poverty status, 64% in crime, and 68% in health behaviors. The results outperform models based on points of interest (POI), population, and other demographic data alone. Moreover, incorporating urban features captured from street view images can improve the explanatory power of these other methods by 5% to 25%. We propose "urban visual intelligence" as a process to uncover hidden city profiles, infer, and synthesize urban information with computer vision and street view images. This study serves as a foundation for future urban research interested in this process and understanding the role of visual aspects of the city.

Diversity beyond density: Experienced social mixing of urban streets.

Urban density, in the form of residents' and visitors' concentration, is long considered to foster diverse exchanges of interpersonal knowledge and skills, which are intrinsic to sustainable human settlements. However, with current urban studies primarily devoted to city- and district-level analyses, we cannot unveil the elemental connection between urban density and diversity. Here we use an anonymized and privacy-enhanced mobile dataset of 0.5 million opted-in users from three metropolitan areas in the United States to show that at the scale of urban streets, density is not the only path to diversity. We represent the diversity of each street with the experienced social mixing (ESM), which describes the chances of people meeting diverse income groups throughout their daily experience. We conduct multiple experiments and show that the concentration of visitors only explains 26% of street-level ESM. However, adjacent amenities, residential diversity, and income level account for 44% of the ESM. Moreover, using longitudinal business data, we show that streets with an increased number of food businesses have seen an increased ESM from 2016 to 2018. Lastly, although streets with more visitors are more likely to have crime, diverse streets tend to have fewer crimes. These findings suggest that cities can leverage many tools beyond density to curate a diverse and safe street experience for people.

Using computer vision and machine learning to identify bus safety risk factors.

In road safety research, bus crashes are particularly noteworthy because of the large number of bus passengers involved and the challenge that it puts to the road network (with the closure of multiple lanes or entire roads for hours) and the public health care system (with multiple injuries that need to be dispatched to public hospitals within a short time). The significance of improving bus safety is high in cities heavily relying on buses as a major means of public transport. The recent paradigm shifts of road design from primarily vehicle-oriented to people-oriented urge us to examine street and pedestrian behavioural factors more closely. Notably, the street environment is highly dynamic, corresponding to different times of the day. To fill this research gap, this study leverages a rich dataset - video data from bus dashcam footage - to identify some high-risk factors for estimating the frequency of bus crashes. This research applies deep learning models and computer vision techniques and constructs a series of behavioural and street factors: pedestrian exposure factors, pedestrian jaywalking, bus stop crowding, sidewalk railing, and sharp turning locations. Important risk factors are identified, and future planning interventions are suggested. In particular, road safety administrations need to devote more efforts to improve bus safety along streets with a high volume of pedestrians, recognise the importance of protection railing in protecting pedestrians during serious bus crashes, and take measures to ease bus stop crowding to prevent slight bus injuries.

Rhythm of the streets: a street classification framework based on street activity patterns.

As the living tissue connecting urban places, streets play significant roles in driving city development, providing essential access, and promoting human interactions. Understanding street activities and how these activities vary across different streets is critical for designing both efficient and livable streets. However, current street classification frameworks primarily focus on either streets' functions in transportation networks or their adjacent land uses rather than actual activity patterns, resulting in coarse classifications. This research proposes an activity-based street classification framework to categorize street segments based on their temporal street activity patterns, which is derived from high-resolution de-identified and privacy-enhanced mobility data. We then apply the proposed framework to 18,023 street segments in the City of Boston and reveal 10 distinct activity-based street types (ASTs). These ASTs highlight dynamic street activities on streets, which complements existing street classification frameworks, which focus on the static or transportation characteristics of the street segments. Our results show that a street classification framework based on temporal street activity patterns can identify street categories at a finer granularity than current methods, which can offer useful implications for state-of-the-art urban management and planning. In particular, we find that our classification distinguishes better those streets where crime is more prevalent than current functional or contextual classifications of streets.

Street life and pedestrian activities in smart cities: opportunities and challenges for computational urban science.

Ongoing efforts among cities to reinvigorate streets have encouraged innovations in using smart data to understand pedestrian activities. Empowered by advanced algorithms and computation power, data from smartphone applications, GPS devices, video cameras, and other forms of sensors can help better understand and promote street life and pedestrian activities. Through adopting a pedestrian-oriented and place-based approach, this paper reviews the major environmental components, pedestrian behavior, and sources of smart data in advancing this field of computational urban science. Responding to the identified research gap, a case study that hybridizes different smart data to understand pedestrian jaywalking as a reflection of urban spaces that need further improvement is presented. Finally, some major research challenges and directions are also highlighted.