From seeing to hearing: A feasibility study on utilizing regenerated sounds from street view images to assess place perceptions

The impact of sound integration on Human Perception of Place (HPP) measurement remains unexplored due to the challenge of real-sound integration, as large-scale sound data acquisition is time-consuming. Additionally, the underdevelopment of HPP measurement through visual-aural integration stems from the widespread acceptance of vision as the most objective sense, despite the significance of auditory perception. This restricts the advancement of HPP measurements, which are restricted by the limitations of visual-based approaches to delivering real-life experiences and intangible factors. Incorporating pseudo-sounds, generated from street view images, could be a feasible alternative for real sounds to overcome current challenges. Thus, this study investigates the efficiency of regenerated sounds using the cross-modal technique in HPP measurements and evaluates its effects on six perceptual indicators. Through experiments in various settings, including visual-only, visual-sound integration, and field experience, we found that regenerated sounds can be a feasible substitute for real sound sources to produce more realistic outcomes. This study contributes to the urban planning and design body of knowledge by exploring the potential of utilizing regenerated sound from street-view images to enhance the visual-aural experience and mitigate visual bias in acquiring human perception of the built environment. The findings of this study promote multisensory-based assessment, which allows for a comprehensive investigation of HPP. Accordingly, decision-makers can gain a better idea of how individuals perceive their surroundings, thereby creating spaces that are sustainable and better aligned with the needs of the community.