What is sonification

When we make a sound to inform about something we are applying a sonification system. We represent data in the auditory field. We turn data into sounds, these data usually can be representing anything that can be expressed in numbers: a physical measurement, a notion, an action or the vectorial tracking of a sequence of values from a sensor. Many definitions were created for this process called sonification: from “subtype of auditory displays that use non-speech audio to represent information”, to “transformation of data relations into perceived relations in an acoustic signal for the purposes of facilitating communication or interpretation” (Kramer et al., 1999) and, in a more definitive and precise way, “data-dependent generation of sound, if the transformation is systematic, objective and reproducible” (Hermann et al., 2011), and finally “technique of transforming non-audible data into sound that can be perceived by human hearing” (wikipedia on 9th of April 2024). To make it simple in the context of this manual we can state briefly that “sonification is the process of generating sound from any sort of data to represent their information as audio”. In even more simple terms we can say to a student that sonification describes data with sound as visualization does with graphs, flow charts, histograms etc.

So basically we want to combine data (Input) and sounds (Output), and decide the way these two are related (mapping or protocol). So a sonification system is defined by these 3 parts:

1 - Input data 2 - Output sounds 3 - Mapping or protocol

Type of Data and Sonification use

Sonification is increasingly used as a scientific tool to analyze and monitor data of several phenomena, and it evolved especially in the astronomical community due to the large amounts of data produced from observing the cosmos, but also as an artistic tool, and educational complement to other disciplines like medicine, mathematics, physics, chemistry but also geography, economy or even literature. For example in medicine, doctors monitor patients’ biometric reactions in real time without having to look at a screen. In literature an audio representation can be created a posteriori (in post-time) using the number of adjectives in a book, the number of times a certain word appears in an article. Any kind of data is made of numbers. And numbers can trigger audio because music and sound are fundamentally resumed to numbers, in the sense that we can describe those using numbers.

Sonification uses

The purpose of sonification is representing, displaying and sharing data. Using the auditory field the data can be more accessible and understandable to as many users as possible, especially for people who have difficulty understanding visual representations of data and it can also be used to make data more engaging and memorable for everyone. Sonification can be used in a variety of applications, such as visualizing scientific data, monitoring environmental conditions, and creating interactive multimedia experiences but also in education when engaging students in the conception of a scientific notion using audio instead of visual stimuli. Here are some examples of how sonification is used in the real world: Analyzing scientific data: Sonification can be used to analyze data that is too complex or abstract to be represented visually. For example, scientists have used sonification to analyze the behavior of atoms (The Sounds of Atoms) and molecules (Molecular sonification for molecule to music information transfer - Digital Discovery (RSC Publishing)), the activity of neurons in the brain (Interactive software for the sonification of neuronal activity | HAL) , and the evolution of galaxies (https://chandra.si.edu/sound/gcenter.html).collision of particles (http://quantizer.media.mit.edu/). Sonification can also be applied when data is recorded in a too dense sequence and therefore time manipulation allows audible up-scaling or sound transformations in larger or shorter durations, such as when transforming the seismogram of an earthquake into sound. Monitoring environmental conditions: Sonification can be used to monitor environmental conditions in real time, for example, to monitor the sound of the ocean to track changes in water temperature and pollution levels (Data Sonification: Acclaimed Musician Transforms Ocean Data into Music). . Creating interactive multimedia experiences: Sonification can be used to create interactive multimedia experiences that are more immersive and engaging than traditional visual interfaces. For example, sonification has been used to create interactive maps (Interactive 3D sonification for the exploration of city maps | Proceedings of the 4th Nordic conference on Human-computer interaction: changing roles), educational games (CosmoBally - Sonokids), and virtual reality experiences.

Real-time sonification vs 'a posteriori'

According to the use of the sonification system (to analyze or to monitor a certain phenomena) we distinguish two “modes”: 1) in real-time (to monitor) - a stream of data is sonifed instantly and a sound is produced to display the value and behavior of the data in that particular moment; 2) “ a posteriori” (to analyze) - time-series sonification of a set of pre-recorded data is converted into an audio file that displays the values and behavior of the data over the period of time covered by the time-series. These two methods are not mutually exclusive and can eventually display the same sounds.The difference is that in “a posteriori”, as the sound is produced after the events that originated the data happened, the parameters of the final piece can be adapted, i.e.the total duration. In a real-time case, you can control the time resolution: that is the time interval at which the sound can change and is played.