Abstract

This report deals with the use of sonification in the presentation of a set of meteorological data.

Sonification is the method of combining the use of sound and visual means to give further insights, and better understanding of a scientific data set. Furthermore sonification gives the opportunity of presenting more variables, or dimensions, than when solely visual means are employed.

A multidimensional visualisation was produced, and the possible gains, abilities, and limitations of combining it with sonification were reviewed.

Sonification of the atmospheric temperature in a three-dimensional space was implemented in AVS.

The model constructed contains only one variable in a three-dimensional space, but is yet so general that additional variables easily can be added

Abstract_____________________________________________________________________ 1

Introduction__________________________________________________________________ 3

Sonification in general__________________________________________________________ 3

Purpose____________________________________________________________________ 3

Limitations_________________________________________________________________ 3

The data set__________________________________________________________________ 4

Sonification in this project_______________________________________________________ 5

Methods_____________________________________________________________________ 5

The Field Files______________________________________________________________ 5

Topography and Atmospheric levels______________________________________________ 6

The Coastal Outline__________________________________________________________ 7

Sonification_________________________________________________________________ 7

Cameras___________________________________________________________________ 9

Potential Extensions_________________________________________________________ 10

Outcomes___________________________________________________________________ 11

Acknowledgements___________________________________________________________ 11

Literature___________________________________________________________________ 11

Internet:__________________________________________________________________ 11

Appendices_________________________________________________________________ 12

temperature.fld_____________________________________________________________ 12

topog.fld__________________________________________________________________ 12

Introduction

This report deals with the use of sonification in the presentation of a set of meteorological data. Visualising a multidimensional data set and investigating the abilities, limitations and the possible gain of involving sonification.

The data used contains only four dimensions: the three space co-ordinates and the temperature measurements from the atmosphere above Australia. However the model constructed is fairly general and prepared for further variables and dimensions.

Sonification in general

What is sonification?

- Sonification is the method of using sound, in combination with a possibly multidimensional visualisation, to give further insight and understanding into a scientific data set. Sonification could be described as the equivalent of a colour-map, but with sound instead of colour. That is, relating a sound to the variable data at each point in the data set.

Purpose

There are limits to the number of variables that are possible to represent by merely visual means. The gain when using sonification is that the use of sound in addition to visual methods creates the possibility of handling more variables, or dimensions, than when solely visual techniques are implemented. In addition to that, a redundancy between audio and visualisation methods often simplifies the interpretation and understanding of the presented problem and makes the conclusions clearer.

Also, in some cases the sound itself offers a better, more logical and natural way, of representing a variable.

Limitations

Although sonification is useful, the connection between the variable and the sound used can often be rather vague. For the method to be successful a recognisable link between the variable and the applied sound is often needed. This makes the method quite a delicate one that requires thought and consideration, and should probably be used in moderation.

When relating a sound to a point in the data field many limitations and dilemmas confront the user. In a representation of the values of a few variables at one point and how they change over time, the sound can simply just be played as time lapses. An example of this kind of problem could be a visualisation of the temperature, the number of ice creams sold, and the amount of traffic at Bondi Beach over a specified time. In that case colour would be suitable for representing the temperature, a simple graph could represent the number of sold ice creams, and sound could preferably be used for the amount of traffic. Together that would result in a coloured graph that changes over time, representing the temperature and the ice cream sales, complemented by a traffic-specific sound whose amplitude, for example, changes as time go by.

With the previous example in mind when considering two- or three-dimensional visualisations the problem of choosing a point to listen to occurs, especially if the data is time dependent. Using a probe or a cursor to select a point is one way of dealing with the problem, but it also clearly reduces the overview of the data, and besides that requires interactivity for the presentation to be meaningful.

Also important to mention is the reality that human hearing capabilities often vary, and the fact that sound is a fairly limited media for representing numerical values. Small changes in the attributes of a sound are usually hard to detect for the unaided ear.

Nevertheless the relative change in a sound can be a very effective media for understanding trends and overall attributes of the data.

The data set

The data set used in this project is a set of meteorological data concerning atmospheric temperature measurements above Australia. The data is divided into two data files; xyz.dat that contains the space co-ordinates of the measurements, and temperature.dat that include the actual temperature measurements at each of the previously specified points.

The temperature measurements are carried out in 16 different amplitude layers, starting on ground level, and ranging upwards. These layers are not evenly spaced; they are spaced in a logarithmic fashion from the ground and upward. That is, the altitude distance between two measurements is small close to the ground, and gets bigger and bigger as altitude increases.

At the lowest layer the measurements are done at ground level, which implies that the co-ordinate data there represents the topology of Australia.

The temperatures are ranging from 189.9K to 301.2K, which corresponds to a range from –83.1°C to 28.2°C.

The data itself is stored as unformatted Fortran binary files, which require some special attention when they are used.

Sonification in this project

As previously mentioned the data set contains of temperature values in a three-dimensional space. The intention in this project was to, at first, use colour to view the temperature values at each point, but also to produce a sound related to that temperature value. Just one point at a time could be listened to, and due to the three-dimensional nature of the data a probe had to be employed. This probe selected the point of interest.

Methods

AVS was chosen for this project as it was necessary to use the AVS Sonify module. This module was written by Brian Kaplan from the Center for Innovative Computer Applications at Indiana University and was the only sonification module for either AVS or OpenDX that could be publicly obtained.

The Field Files

The temperature.fld file (See Appendix) reads in the x, y and z coordinates from the xyz.dat file and the temperature as a variable from the temperature.dat file. An irregular field of “floats” is created. The x-y dimensions are 82x60 with 16 z-direction levels. The temperatures were read straight from temperature.dat, while the skip and stride commands needed to be used to read the correct coordinates. Stride had a value of 3 and skip had values of 0, 4 and 8 for the x, y and z coordinates respectively. As the format of the data was Fortran binary, the file type was set to “unformatted”.

The topog.fld file (See Appendix) was used to create a sea level outline of Australia. The difference between this and temperature.fld is that the temperature is no longer the variable. Data is now read only from the xyz.dat file and the z-coordinate is set as the variable. This allowed us to highlight (using the tube module) the points at which the z-coordinate was zero and so highlight the coastline.

Figure 1. AVS network of the final audiovisual presentation.

Topography and Atmospheric levels

Using AVS, the above network was created (See Figure 1.).

Orthogonal slices of the data were first constructed, using the orthogonal slicer module. A single z-plane level of the 16 in the dataset was fed in to the field to mesh module and then into geometry viewer. The topography of Australia could roughly be seen on the lowest level (15, which corresponded to ground level), but when viewing only one slice at a time there was no frame of reference for the atmospheric levels. Therefore, a second slice (using another orthogonal slicer/field to mesh set) was added, which was left at ground level. The relative heights of the atmospheric levels could then be judged. The modules generate colormap and color range were used for both slices to indicate the temperature. The colour map used ranges from red (hottest) to blue (coolest).

To see all atmospheric levels at once, the bubbleviz and scatter dots modules were initially used to place dots across the surfaces of the levels. Viewing 16 opaque levels would mean that all but the uppermost level would be obscured, while the dots allowed the viewer to see through the volume of data. However, while the dots did give an indication of the positions of the atmospheric levels, it was decided that they confused the visualisation and cluttered the image. It was found that displaying one atmospheric level at a time was adequate and that by making that level transparent no information was obscured. In addition, by using the animate integer module an animation could be created that scrolled through the levels and enabled the viewer to quickly see the rough temperature at each level in the atmosphere. Experiments were carried out towards the end of the project concerning transparency when displaying all levels simultaneously. One way of doing this would be to use a macro-module of 15 orthogonal slicer/field to mesh sets. However, it was found that each level’s transparency would need to be adjusted individually and at that stage of the project it was decided this method would be an unnecessary use of time and would not produce the desired image.

Other modules were used in the underlying visualisation. These included label (to create the main and colour bar labels) and color legend (which created the colour bar). Also used was volume bounds, which created a three dimensional outline of the extents of the data set. This was useful in determining the position of and moving the sonification probe. Statistics and print field were used to view the data numerically and were particularly useful in initially troubleshooting the field files.

The Coastal Outline

A second field file was added to read the z-coordinate as the dependent variable, in order to create an outline of the coast of Australia. The vague shape of Australia was visible on the bottom slice, but needed to be more clearly defined. Contour to geom was used to select the z = 0 contour and the tube module was applied to create a thick line on this contour. The result was overlaid on the bottom slice in the geometry viewer.

Sonification

Once the basic dataset was visualized, the Sonify module was added. This module had been obtained online and had to be compiled before being imported into AVS. The “makefile” included didn’t initially compile the module and the C code contained needed to be modified. The coding task proved to be beyond our ability, but with help from Daniel Mitchell (Vislab, University of Sydney) the Sonify module was compiled successfully and was ready for use. Sonify is a module that accepts values for various parameters and outputs a waveform according to the value of those parameters and so a sound is produced. The parameters include, among others, things like pitch, volume, attack and decays rates. The waveform is played for a certain short time period and can be played continuously or only when the input changes.

The main challenge, once Sonify had been imported, was to extract values from the topography and temperature visualisation for use in controlling the parameters of the output waveform. AVS contains a number of modules that can probe a visualisation. Probe path is one that was relevant in this case. It can display the position of the probe and the value of the variable at that point. However, the module had no output port corresponding to the variable read. Therefore Probe path was modified into probe value (thanks again to Daniel Mitchell) which contains the required output port. This port was connected to Sonify to control that module’s parameters. The temperature values were fed into the pitch parameter in Sonify, as that variable was one of the easiest to hear changes in. There was a final problem at this stage. The temperature range of the data set was around 189 to 301 Kelvin. However Sonify requires its input parameter to have a value between 1 and 100. Field math modules were used to adjust the data read by the probe to a value between 1 and 100. The equation used to do this was

Finally, field legend was used simply to change the temperature from a “field” variable into the required “real” form. It was found that by linking the variation in temperature to the volume of the output sound, as well as the pitch, the effect of the sonification could be accentuated. The final result: a high frequency, loud tone indicates high temperature and low frequency, soft tones indicate low.

The final outcome of the visualisation is displayed below (See Figure 2.).

Figure 2. Final visualisation with temperature layer at ground level and

highest altitude displayed. The white grid shows the position of the rest of

the 16 layers. Furthermore, the probe used for sonification can be viewed over

the centre of Australia.

Cameras

To aid the manipulation of the probe, three extra camera angles were used with the geometry viewer. These views were top-down, from the east and from the south (See Figure 3.). This meant the user could easily move the probe in the x, y or z direction independently of the other two directions.

Figure 3. Australia from above (upper left), horizontal southerly view of

Australia (upper right), and horizontal easterly view of Australia (lower).

Potential Extensions

A project such as this is open ended. Currently the probe is a little clumsy (partially due to processor speed) and the nature of the output sound not the most appropriate for this data set. The probe could be restricted to move within the bounds of the data. Sampled sounds (such as wind or rain) could also be manipulated for use in a meteorological data set such as this.

Outcomes

The main aim of this project, to combine sound and visualisation, was achieved. The pitch and volume of a synthesized tone have been linked to a visualisation of the temperature at various altitudes above Australia. This enables the user to employ a second sensory dimension in the interpretation of the data. Sonification, like that implemented here, enables many more variables to be studied at one time.

Acknowledgements

We would like to thank Prof. Lance Leslie, and Dr. Russel Morison, Applied Mathematics, University of New South Wales, for the data set.

Thanks also go to Daniel Mitchell and Steven Manos, at Vislab, University of Sydney, for big help regarding compilation of AVS modules and general knowledge of AVS.

The AVS Sonify module was created by Brian Kaplan (kaplan@cica.indiana.edu) and John MacCuish from the Center for Innovative Computer Applications at Indiana University.

Literature

AVS Module Reference

AVS User’s Guide

Internet:

http://www.iavsc.org/repository/avs5 - AVS 5 module repository

http://www.cica.indiana.edu/projects/AVS.Tools/ - The AVS Sonify module was obtained here.

Appendices

temperature.fld

# AVS xyz.dat, temperature.dat

ndim=3

dim1=82

dim2=60

dim3=16

nspace=3

veclen=1

data=float

field=irregular

variable 1 file=/home/mcole/scivis/project/temperature.dat filetype=unformatted skip=0

coord 1 file=/home/mcole/scivis/project/xyz.dat filetype=unformatted skip=0 stride=3

coord 2 file=/home/mcole/scivis/project/xyz.dat filetype=unformatted skip=4 stride=3

coord 3 file=/home/mcole/scivis/project/xyz.dat filetype=unformatted skip=8 stride=3

topog.fld

# AVS xyz.dat