Magnitude estimation of conceptual data dimensions for use in sonification

Abstract

Most data exploration tools are exclusively visual, failing to exploit the advantages of the human auditory system, and excluding students and researchers with visual disabilities. Sonification uses non-speech audio to create auditory graphs, which may address some limitations of visual graphs. However, almost no research has addressed how to create optimal sonifications. Three key research questions are: (1) What is the best sound parameter to use to represent a given data type? (2) Should an increase in the sound dimension (e.g., rising frequency) represent an increase or a decrease in the data dimension? (3) How much change in the sound dimension will represent a given change in the data dimension? Experiment 1 simply asked listeners which of two sounds represented something that was hotter, faster, etc. However, participants seemed not to make cognitive assessments of the sounds. I therefore proposed magnitude estimation (ME) as an alternative, less transparent, paradigm. Experiment 2 used ME with visual stimuli (lines and filled circles), replicating previous findings for perceptual judgments (length of lines, size of circles). However, judgments of conceptual data dimensions (i.e., the temperature, pressure, or velocity a given stimulus would represent) yielded slopes different from the perceptual judgments, indicating that the type of data being represented influences value estimation. Experiment 3 found similar results with auditory stimuli differing in frequency or tempo. Estimations of what temperature, pressure, velocity, size, or number of dollars a sound represented differed, indicating that both visual and auditory displays should be scaled according to the type of data being displayed. Experiment 4 presented auditory graphs and asked which of two data descriptions the sounds represented. Data sets based on the equations determined in Experiment 3 were preferred, providing validation of those slope values. Results also supported the use of the unanimity of mapping polarities as a measure of a mapping's effectiveness. Replication with different users and sounds is required to assess the reliability of the slopes. However, ME provides an excellent way to obtain a function relating conceptual data dimensions to display dimensions, which can be used to create more effective, appropriately scaled sonifications.