What is sonification

When we make a sound to inform about something we are applying a sonification system. We represent data in the auditory field. We turn data into sounds, these data usually can be representing anything that can be expressed in numbers: a physical measurement, a notion, an action or the vectorial tracking of a sequence of values from a sensor. Many definitions were created for this process called sonification: from “subtype of auditory displays that use non-speech audio to represent information”, to “transformation of data relations into perceived relations in an acoustic signal for the purposes of facilitating communication or interpretation” and, in a more definitive and precise way, “data-dependent generation of sound, if the transformation is systematic, objective and reproducible” , and finally “technique of transforming non-audible data into sound that can be perceived by human hearing” . To make it simple in the context of this manual we can state briefly that “sonification is the process of generating sound from any sort of data to represent their information as audio”. In even more simple terms we can say to a student that sonification describes data with sound as visualization does with graphs, flow charts, histograms etc.

So basically we want to combine data (Input) and sounds (Output), and decide the way these two are related (mapping or protocol). So a sonification system is defined by these 3 parts:

1 - Input data 2 - Output sounds 3 - Mapping or protocol

Type of Data and Sonification use

Sonification is increasingly used as a scientific tool to analyze and monitor data of several phenomena, and it evolved especially in the astronomical community due to the large amounts of data produced from observing the cosmos, but also as an artistic tool, and educational complement to other disciplines like medicine, mathematics, physics, chemistry but also geography, economy or even literature. For example in medicine, doctors monitor patients’ bio-metric reactions in real time without having to look at a screen. In literature an audio representation can be created "a posteriori" (in post-time) using the number of adjectives in a book, the number of times a certain word appears in an article. Any kind of data is made of numbers. And numbers can trigger audio because music and sound are fundamentally resumed to numbers, in the sense that we can describe those using numbers.

Sonification uses

The purpose of sonification is representing, displaying and sharing data. Using the auditory field the data can be more accessible and understandable to as many users as possible, especially for people who have difficulty understanding visual representations of data and it can also be used to make data more engaging and memorable for everyone. Sonification can be used in a variety of applications, such as visualizing scientific data, monitoring environmental conditions, and creating interactive multimedia experiences but also in education when engaging students in the conception of a scientific notion using audio instead of visual stimuli. Here are some examples of how sonification is used in the real world: Analyzing scientific data: Sonification can be used to analyze data that is too complex or abstract to be represented visually. For example, scientists have used sonification to analyze the behavior of atoms (The Sounds of Atoms), the activity of neurons in the brain (Interactive software for the sonification of neuronal activity | HAL) , and the evolution of galaxies (https://chandra.si.edu/sound/gcenter.html). Sonification can also be applied when data is recorded in a too dense sequence and therefore time manipulation allows audible up-scaling or sound transformations in larger or shorter duration, such as when transforming the seismograph of an earthquake into sound. Monitoring environmental conditions: Sonification can be used to monitor environmental conditions in real time, for example, to monitor the sound of the ocean to track changes in water temperature and pollution levels Creating interactive multimedia experiences: Sonification can be used to create interactive multimedia experiences that are more immersive and engaging than traditional visual interfaces. For example, sonification has been used to create interactive maps , educational games (CosmoBally - Sonokids), and virtual reality experiences.

Real-time sonification vs 'a posteriori'

According to the use of the sonification system (to analyze or to monitor a certain phenomena) we distinguish two “modes”: 1) in real-time (to monitor) - a stream of data is sonifed instantly and a sound is produced to display the value and behavior of the data in that particular moment; 2) “ a posteriori” (to analyze) - time-series sonification of a set of pre-recorded data is converted into an audio file that displays the values and behavior of the data over the period of time covered by the time-series. These two methods are not mutually exclusive and can eventually display the same sounds.The difference is that in “a posteriori”, as the sound is produced after the events that originated the data happened, the parameters of the final piece can be adapted, i.e.the total duration. In a real-time case, you can control the time resolution: that is the time interval at which the sound can change and is played.

Acoustic ecology

The aesthetic is important. A sound can be mapped very precisely but sounds “awful” to the user. This could be considered as a defect and therefore it could limit the efficacy of the system because the user will not bear listening to it. On the other side (i.e. in alarms) the sound can be intentionally noisy and aggressive. The choice of the output sound is in some way artistic in a sense that it must take into consideration the type of audience and its taste. It does not mean that we are obliged to play something that the user will like, but at least be aware of what type of sound is familiar to him/her. Even if the taste is subjective we would like to recall the work done in the field of acoustic ecology. There are some common factors indicated by psychology studies and also cultural models of “beauty”. In the present project, as the name of the project suggests, we reference the work and vision of the Canadian composer Murray Schafer, who popularized the term “soundscape” in the book “The Tuning Of The World” in 1977. Soundscapes can be simply considered as a composition of the anthrophony, geophony and biophony of a particular environment. The author argues that we've become desensitized to the rich sounds of our environment, which he calls the "soundscape." This soundscape encompasses all the natural and human-made sounds that surround us, and Schafer believes we should learn to appreciate and manage it for a better world.His work generated the “acoustic ecology movement” which aims to study the relationship between humans, animals and nature, in terms of sound and soundscapes. The Acoustic Ecology Institute was founded to raise consciousness of the effect of noisy acoustic environments, proven to be harmful for increasing stress levels on individuals when immersed in these.

Examples

SonarX is a software designed to transform images and video into meaningful sound for blind individuals and all. It runs on Pure Data and can be downloaded at this github repository.