Software Visualisation

Peter Young

Visualisation Research Group
Centre for Software Maintenance

Department of Computer Science
University of Durham

18 June, 1996


Contents


1. Introduction

It is a widely known fact that software maintenance is considered the most costly activity in the software life-cycle, both in time and money [Standish84]. Regardless of the maintenance task being performed, it is highly likely that the maintainer will require at least some understanding of the code. This understanding often comes from a hard fought battle in which the maintainer must attempt to rediscover the intentions and design issues behind the code, typically from scarce resources and in the worst case from the source code alone. For this reason program comprehension is generally accepted as being the central task within software maintenance. Tool support for software maintenance typically targets this task as it offers the best opportunity for efficiency gain. Good computer support and tools which aid the software maintenance process through program comprehension would be of great value. Unfortunately good tools are hard to find and most, while still aiding comprehension to some extent, could be improved greatly.

One area of research which attempts to aid program comprehension and more generally software maintenance is software visualisation. Software visualisation makes use of the fact that the human brain is more suited to processing, manipulating and recognising visual images and structures. Reading of text is a special case of this visual processing, however this is translated or interpreted at a character or possibly word level. Such detail is too great and the brain will abstract each character, word or sentence into an internal meaning or representation. This is described in the program comprehension literature at a higher level, for example beacons [Brooks83] and chunks [Shneiderman80] are abstract internal representations of low-level textual structures.

Software visualisation attempts to aid the comprehension process by providing these abstractions in a visual form, thus reducing the interpretation load. By presenting the viewer with a pictorial model of more complex lower level information, they can rapidly generate an initial mental model of the software and use this as a basis for further investigation. Software visualisation is concerned with all areas of generating and manipulating these graphical models. In particular, some of the main problems which must be tackled are:

representation
How the various components are presented;
abstraction
What should be presented, and in how much detail;
navigation
How the user can move freely through the suite of visualisations that would constitute a full software system;
correlation
How to link the visual abstractions to the actual program code or documentation.

This document reviews a section of the literature regarding software visualisation and highlights specific points of interest. The first point addressed is the definition of the terms related to the field, and in particular the definition of software visualisation itself. Following this, the role software visualisation plays within software maintenance and program comprehension is discussed and the need for software visualisation is highlighted. A review of the current state of research in software visualisation, including the various systems developed is then given. This includes an introduction to the more specific area of three-dimensional software visualisation, which is followed by a detailed review of current systems utilising 3D graphics.

2. What is software visualisation?

2.1. Defining software visualisation

Software visualisation is still not a well defined term and lacks any concrete definition or meaning. Many researchers have tried to define the term, though the majority of these definitions contradict one another or offer varying levels of detail, scope and abstraction. At present the term is generally used to encompass almost all forms of visualisation concerned with representing any aspect of a software system. A number of more specialised areas have evolved under the guise of software visualisation, areas such as algorithm animation, data visualisation and code visualisation. These areas are also subject to varying definitions and scope of field. To compound matters, the actual term software visualisation is often used as being synonymous to program visualisation, computation visualisation and code visualisation.

A vast number of definitions and descriptions related to software visualisation are given in the literature, a large selection of these are now given. This list is intentionally long, to illustrate the variety of definitions and interpretations offered.

It can be seen from the above definitions and descriptions that the field of software visualisation is diverse, with research interests in many different directions. One commonality evident in the majority of definitions is the emphasis on visual representations. The term software visualisation or program visualisation is generally read as incorporating exclusively graphic representations. This, however, is misleading and overly restrictive. Software visualisation should allow for representations other than graphical. It is not unreasonable to foresee visualisation tools which make use of the other available human senses, in particular hearing, smell and touch. Price et al. redefine their use of the term visualisation to include such flexibility, as "the power or process of forming a mental picture or vision of something not actually present to the sight" [Price93].

One of the major differences between definitions such as those given above, is their specificity. Almost all of the definitions and descriptions are too specific in one point or another. Some definitions imply that visualisation must incorporate solely graphics or a particular media, whereas others state the use of specific techniques or systems. However, introducing too much generality will bring the term very close to the field of program comprehension. For instance, the definition "software visualisation is the use of visual (and other) media to aid understanding of software systems" could be construed as another definition for program comprehension given the flexibility of the "other media".

2.2. The position of software visualisation

Identifying the position of software visualisation with respect to other fields, and more specific sub-fields has been suggested by a large number of authors [Price93, Stasko92a, Stasko92b, Myers90]. Price et al. [Price93] view software visualisation as a field which encompasses the two distinct areas of program visualisation (including visual programming) and algorithm animation. Stasko [Stasko92a] proposes the field of computation visualisation which includes both software and hardware views. Fields encompassed by the topic of software visualisation are identified by Price et al. resulting in the structure shown in Figure 1. This structure does, however, pose some questions.

Figure 1
Figure 1. Scope and inclusion of software visualisation terms.
Equivalent to figure 1 in [Price93].

Visual programming, as defined by Roman and Cox, is "concerned with graphical specification of computer programs" [Roman92] and does not apply to the current onslaught of so-called visual languages such as Visual C++, Visual Basic, VisualAge and Visual Objects. These systems provide little more than a unified GUI (graphical user interface) to conventional program development tools [Najork95]. Price et al. identify the term program visualisation as including the field of visual programming. This notion is in contradiction to other views which consider program visualisation and visual programming as disparate fields [Roman93, Myers90]. Myers even states, "Program Visualisation is an entirely different concept from visual programming" [Myers90]. Conversely, it is not unreasonable to argue that in order to create a visual programming environment, you must first be able to visualise programs, hence program visualisation. This argument is open to interpretation, however in the classification by Price et al., the field of algorithm animation is not included as part of program visualisation. It seems unreasonable to make such a distinction in one case and not in the other.

Further, Price et al. describe program visualisation as being composed of both static and dynamic code and data visualisations, whereas visual programming is concerned only with static visualisations. Again, this point is open to debate. While visual programming is concerned with the creation or visual implementation of programs, this process may not be restricted purely to static visualisations. One simple argument is that debugging and tracing of code is considered an integral part of programming, and in a visual sense the only way in which the execution or tracing can be presented is through animated visualisations. Another argument is that we cannot constrain the visual programming paradigm to purely static or even graphic visualisations. A visual programming-by-example paradigm could be imagined in which the programmer specifies operations by manipulating the graphical interface in a dynamic fashion; by providing an example. An example could be copying or moving data within memory, this being achieved by dragging or moving appropriate areas within the visualisation.

The arguments posed above offer constructive criticism to the classification of definitions within software visualisation. The validity of these arguments does however hinge upon the readers' interpretation of the terms and their scope. Figure 1 above illustrates the scope of these terms as described by Price et al. Similarly, Figure 2 below shows a revised classification of these terms which incorporates the arguments mentioned above. In addition to the field of software visualisation, the term computation visualisation is introduced by Stasko [Stasko92a] as including both software and hardware views. This is incorporated in Figure 2.

Figure 2
Figure 2. Revised scope and inclusion of software visualisation terms.

A brief description of each of these fields is now given.

Computation visualisation
Computation visualisation is the use of computer graphics to explain, illustrate and show how computer hardware and software function. [Stasko92a]
Software visualisation
Software visualisation describes systems that use visual (and other) media to enhance understanding or comprehension of software systems.
Program visualisation
Program visualisation is the representation of source code and other documents to enhance understanding of an existing software system.
Visual programming
Visual programming is the use of graphical representations to specify and implement computer software.
Algorithm visualisation (Static and Dynamic)
Algorithm visualisation concentrates on the representation and animation of the abstract operation of software algorithms.
Static and dynamic code visualisation
Concentrates on illustrating the actual program code, either by enhancing its textual appearance [Hill93] or by representing it in a more abstract form.
Static and dynamic data visualisation
Concentrates on visualising the data defined and manipulated within the software.

2.3. Taxonomies of software visualisation

Several taxonomies have been applied to software visualisation systems, again these vary to the same extent as the definitions. There are two taxonomies which are predominant in the field, an earlier taxonomy by Myers [Myers90] and a more recent taxonomy by Price et al. [Price93]. Myers' taxonomy classifies systems along two main axes: the data they present, such as code, data or algorithm visualisations; and whether the visualisations are dynamic or static. While this taxonomy covers some of the important characteristics of a software visualisation system, it does not provide any scope for evaluating the effectiveness, usability, applicability or quality of systems. This is addressed to a greater extent in the taxonomy by Price et al.

Price et al. categorise systems in a more systematic manner and attempt to structure the taxonomy to allow flexibility for future expansion and revision. Their taxonomy is based upon a hierarchical structure of classifications which allows simple addition of new categories. The top level of this hierarchy is based on a very general model of software containing six categories: Scope; content; form; method; interaction; and effectiveness.

Other taxonomies exist, again using a different set of classifications. One such taxonomy is by Stasko and Patterson [Stasko92b] which introduces scaled dimensions into a four-category taxonomy. These categories cover aspect, abstractness, animation and automation. Another such taxonomy by Roman and Cox [Roman92] initially identified systems using the categories of scope, abstraction, specification method and technique. This initial taxonomy was refined further [Roman93] with the category "technique" being replaced with the two categories of interface and presentation. They view this visualisation as a mapping from programs to graphical representations [Roman92], with the categories of the taxonomy representing aspects of this mapping. The scope covers what aspects of the program are visualised. Abstraction and specification method cover the actual visualisation process; what information is conveyed and how it is constructed. Finally, interface and presentation (technique in earlier work) cover the actual graphical representation; how it is used to convey the information.

3. Why do we need software visualisation?

It is an acknowledged fact that software maintenance consumes the majority of resources within the software life-cycle. It is also a well known fact that program comprehension plays the major role in any maintenance activity. Whether the activity is adaptive, perfective, preventative or corrective, all require an understanding of the software system to some extent. The task of gaining this understanding falls under the broad scope of program comprehension.

Problems and deficiencies within the software development process result in software systems which are typically badly designed, badly implemented, and possibly worst of all, badly documented. Flaws in design and implementation do cause problems, however if these flaws or processes are correctly documented then steps can be taken to correct or work around them. However, if the documentation is lacking in any respect then such flaws can go unnoticed and cause severe difficulties during maintenance. Even well designed and implemented code can prove a problem if not correctly documented. Maintainers must then attempt to discover the design intentions behind the implemented system, in the worst case from the source code alone.

Unfortunately, problems with system documentation are widespread. Time constraints imposed upon projects often result in rushed implementations, with documentation receiving a low priority. As a result, documentation is often non-existent, incomplete, inaccurate, or worst of all, misleading. Thus, program comprehension is often limited to discovering an understanding of a system from the source code and any scant documentation. The various theories of program comprehension put emphasis on the construction of a mental model of the software within the mind of the maintainer [Young96a]. Theories on the construction of this mental model hypothesise various techniques employed by maintainers in extracting information from the program source code. Software visualisation attempts to provide tool support for generating this mental model or understanding.

The mental models proposed by various researchers in the field of program comprehension [Young96a] have a commonality in that they all are composed from semantic constructs. These constructs are typically abstractions, at various levels, of program features. The network formed from these constructs constitutes the maintainers understanding and representation of the program. Software visualisation attempts to provide a mapping from the program code to a visual (and other media) representation which matches the maintainer's mental model as closely as possible. Highlighting and automatically creating these semantic constructs would remove a great deal of effort from the maintainer, specifically they would require less time spent on scanning the source code. The field of software visualisation is thus considered an important area of research within the general field of program comprehension.

Roman and Cox [Roman92] highlight a number of arguments which are used generally in advocating the need for software visualisation. These arguments include: the importance of imagery in human communication; the speed with which humans can track and identify visual patterns; the expressive potential of a visual vocabulary which exploits multiple dimensions; and the inherent power of abstraction within visual representations. The credibility of such arguments are often underlined with reference to the success of graphical representations in areas such as human computer interfaces, scientific data visualisation and information visualisation.

4. What is the current state of research?

Current research into software visualisation is diverse, with activity in all areas of the subject. Initially the main research thrust was developing algorithm visualisation frameworks and systems. These systems were used predominantly for teaching algorithm design and analysis and were not typically used as general purpose tools. Another favourite topic in software visualisation is its application to program debugging and execution tracing, particularly within imperative languages. More recently, an increase in research has been directed towards structural or macroscopic visualisations of code. These visualisations concentrate on a large scale view of the code and typical examples are function call graphs, control flow graphs and file dependencies. In addition, there is a very active research interest in the visualisation of parallel programs, though generally concentrating on their execution behaviour [PVMTrace95, Koike95, Miller93, Kraemer93].

Algorithm visualisation research concentrates on producing meaningful and insightful visualisations and animations to aid understanding of algorithm operation. Generally these visualisations are not produced automatically and require an animator to design a suitable representation for each algorithm. The majority of algorithm animation frameworks created rely on the animator to instrument the algorithm implementation at critical points with additional calls to the animation environment or to a log file generator. This information is then used either during execution or post-mortem, respectively, to drive the animation. Many systems have been developed for algorithm animation, a large number of which are described in the various taxonomies [Price93, Myers90, Roman93]. Such systems include: BALSA; Balsa-II; Zeus; TANGO; Polka-3D; and ANIM.

Visual debugging, is not generally accepted as a separate topic of program visualisation though there are a large number of systems which use visualisation as a primary aid to debugging. Many of these systems are confined to tracing the execution of imperative languages, however various other languages and paradigms are also covered. The majority of these systems operate automatically, i.e. they do not need manual intervention to obtain the program information. This is generally obtained using either of two methods. One method provides an interpreter base in which the target program is 'executed' thus allowing the visualiser access to all information regarding the program and its execution. Alternatively the visualiser will automatically instrument the target source code with calls to the visualiser, or with statements which output information to a log file for post-mortem analysis. Again, these systems are described and classified within the various taxonomies. Such systems include: Pascal Genie; UWPI; TPM; Pavane; LogoMedia; CentreLine ObjectCentre (Saber-C++); ZStep 94 [Lieberman95, Fry95]; and PVMTrace.

Structural visualisations concentrate mainly on large scale or macroscopic views of the source code. These views typically consist of visualising dependencies between the various components that make up a software system. Components such as variables, classes, functions, methods, files and modules are displayed to produce the stereotypical views of call-graphs, control-flow graphs, module dependencies and data dependencies. The key factor of these views being their inherent graph structure. These graphs are often very large and difficult to display effectively. Such systems are not covered in current taxonomies, though they should easily be incorporated by the various taxonomic criteria. Examples of these systems include: SMARTsystem; McCabe; Logiscope [Meekel88]; SNiFF; Rigi [Storey95]; Narcissus [Hendley95]; GraphLog [Consens92]; SemNet [Fairchild88]; and GraphVisualiser3D [GV3D95].

The majority of software visualisation systems to date have concentrated on producing two dimensional representations and animations of various aspects of a software system. In addition, a growing area of research is investigating the application of 3D graphics and virtual environment technology to software visualisation. Research into 3D software visualisation attempts to address the problem both in terms of the data to represent and the users' ability to navigate this data. The former aspect stems from a growing need for more expressive powers to represent the inherently multi-dimensional data, facts and relationships which constitute a software system. The latter plays upon the naturally developed capabilities of humans for navigating and interacting within three dimensional environments.

Software systems, and in particular the large, highly inter-related information bases that they constitute, are inherently multi-dimensional and thus difficult to visualise [Brooks87]. Software visualisation systems typically isolate one aspect or view of this information for display, effectively hiding all remaining information from view. This isolated view generally consists of a large and complex directed graph which is displayed on a two dimensional canvas using one of the many layout algorithms developed within the field of graph layout. Regardless of the quality of the layout algorithm or viewing technique (such as fisheye) used, the user is typically presented with an incomprehensible birds-nest graph or must reduce the amount of information displayed thus removing potentially important information from view [Fairchild88]. While the use of 3D or virtual environment technology cannot immediately solve this problem, it does offer a number of advantages such as: greater working volume for information presentation; focus, context and point of interest are managed dynamically by the user's viewpoint on the information; and greater flexibility with representation, organisation and presentation of the information. These arguments do not in themselves justify 3D as superior to 2D representations, however they do highlight possibilities for advances in software visualisation using 3D graphics.

5. 3D software visualisation systems

Although the full potential of using three dimensional views and interaction techniques has not yet been realised, a number of software systems have been developed to investigate its worth. In general, these systems do not make full use of the 3D graphics or advance software visualisation to any great extent. The majority simply extend and adapt already established two dimensional visualisation techniques into 3D [Brown93, Reiss94]. An example of this is highlighted by Brown and Najork [Brown93] who describe a variety of 3D graphics techniques for algorithm animation. They describe three fundamental uses of 3D:

These techniques take little advantage of the additional dimension and merely offer 'window dressing' on existing techniques. They do, however, provide useful additional information such as the execution history or extended attributes, though these could be facilitated in other ways such as animation or annotations, respectively. Similarly, 3D structural visualisation systems offer simple extensions of 2D graph techniques into 3D.

This section now describes a number of 3D software visualisation systems, varying from algorithm to structural visualisations.

5.1. Zeus

Zeus forms the latest in a series of algorithm animation systems developed at Brown University and the Digital Equipment Corporation (DEC). BALSA and BALSA-II were considered as the first major interactive software visualisation systems. This early system (BALSA) was used widely as an aid to teaching algorithm design and analysis and allowed students to interactively view animations prepared by a lecturer. Functions such as stop, play, pause, speed control and direction control were provided. Zeus offered a number of enhancements over these earlier systems in particular the use of colour, sound, and more interestingly, of 3D graphics.

Brown and Najork [Brown93] describe a variety of 3D graphics techniques for algorithm animation. These techniques are highlighted by examples created using Zeus. Constructing animations within Zeus relies on the animator identifying critical areas within the algorithm code, such as data manipulation, conditional evaluation or procedure calls. These areas are then instrumented with calls to the Zeus "interesting event" manager. During execution of the algorithm these interesting events will be reported to Zeus, which will then forward them on to any interested views. The views must be predefined and designed to respond appropriately to events in the code. During the visualisation the user is still able to alter various aspects of a view by changing its associated control panel. This allows both generic parameters, such as lighting level, and view-specific parameters to be changed.

5.2. PVMTrace

PVMTrace [PVMTrace95] is a project underway at the University of New Brunswick which is concerned with the visualisation of parallel processes with an emphasis on debugging (Figure 3). Understanding and debugging complex sequential programs is often very difficult, this problem grows drastically when trying to understand the execution of communicating parallel processes. There is a need for good tools to aid the understanding of such programs and visualisation promises to provide an effective communication medium between the execution data and the programmer.

PVMTrace will concentrate on visualising parallel programs which are running on PVM. PVM is a library which simulates a parallel computer by running processes on multiple workstations in a network. The project will investigate different visualisations and representations for presenting the execution of parallel processes and messages between them. Particular attention will be given to the application of three dimensional animated visualisations.

Figure 3

Figure 3. Execution of parallel processes, shown using PVMTrace.
Image courtesy of Glenn Frank, University of New Brunswick.
http://www.omg.unb.ca/hci/projects/hci-pvmtrace.html

5.3. VOGUE

VOGUE is a visualisation system developed at Koike Labs, which provides a framework for creating 3D software visualisations. VOGUE concentrates on the concept of integrating a number of 2D views to create a more powerful 3D visualisation. This system has been applied in a number of different visualisation scenarios:

VOGUE is used in creating a 3D visualisation which integrates version control and module management (Figure 4(a)). The visualisation creates a two dimensional tree from the module dependencies at each revision. This set of 2D trees are then placed in series along the Z axis of the display space, with the most recent revision placed at the front. Relationships between source code modules are visible both between different versions of the same module through time, and as compilation dependencies between modules for each revision.

The 3D class library browser implemented using VOGUE integrates two sets of information concerned with class inheritance hierarchies (Figure 4(b)). The first set concerns the actual inheritance hierarchy and depicts which classes are derived from other classes. The second set of information shows which methods belong to particular classes. These two sets are integrated into a single 3D visualisation to provide immediate indication of which method is actually executed when called from any particular class. By shifting viewpoint orientation the programmer can obtain a display of either data set, or view both sets and any dependencies between them. This clarifies the problem by reducing the need to mentally integrate both sets of information.

Figure 4(a)
Figure 4(b)
Figure 4(a). VOGUE version control management
Figure 4(b). VOGUE C++ class browser
Images courtesy of Hideki Koike,
Koike Labs, University of Electro-Communications, Tokyo.
http://www.vogue.is.uec.ac.jp/

5.4. VisuaLinda

VisuaLinda is a program visualisation system developed at Koike Labs which uses 3D visualisations to clarify the operation of parallel programs [Koike95]. VisuaLinda is built upon the VOGUE visualisation system, also developed at Koike Labs.

Linda is best described as a co-ordination language and not a complete language [Gelernter92]. Such a co-ordination language would be integrated with a computational language to create a complete programming language. Linda allows the co-ordination of processes through the use of a shared tuple space. VisuaLinda is both a visualisation tool for parallel Linda programs, and a Linda server itself. This affords a number of advantages, the most noteworthy being that no code instrumentation is required to produce the visualisations, as this can be handled transparently by the server.

VisuaLinda makes use of 3D graphics to visualise the execution of parallel processes (Figure 5). This allows the programmers to view both the relationships between processes and a time flow of the processes simultaneously.

Figure 5

Figure 5. VisuaLinda implemented using VOGUE
Image courtesy of Hideki Koike,
Koike Labs, University of Electro-Communications, Tokyo.
http://www.vogue.is.uec.ac.jp/

5.5. SemNet

SemNet [Fairchild88] is a research prototype developed to advance understanding of the complex relationships and structures of large, arbitrary knowledge bases (Figure 6). The aim of SemNet was not to provide a solution for this general problem, but rather to explore the application of 3D visualisation techniques and to investigate the specifics of the problem. The main problem addressed by SemNet is how to present large knowledge bases to enable improved or more efficient comprehension. Fairchild and Poltrock hypothesise that for comprehension of a knowledge base, a user must recognise:

Knowledge bases represent information about relationships between objects or records. Fairchild and Poltrock believe that graphical representations offer an effective way of communicating such relationships. This coupled with the high level of visual processing and spatial reasoning capable in the human brain, makes 3D representations a good candidate for such a task.

Fairchild and Poltrock are quick to point out that simply using a graphical representation does not immediately solve the problems inherent with exploring, manipulating, understanding and modifying large knowledge bases. Very large knowledge bases become very large directed graphs! This highlights an important need for reducing the visual complexity of the graphical representations, a need which is also addressed in SemNet.

Graph layout and information hiding within SemNet is debated by Fairchild and Poltrock. A number of methods are described with their advantages and disadvantages noted. The authors agree that the positioning of knowledge elements within the visualisation can greatly affect comprehension of the knowledge base structure. Layout of the visualisation in SemNet can be performed using one of two methods:

Information hiding in SemNet is provided both by the inherent perspective based reduction in detail due to distancing in the 3D display, and also by application of additional fish-eye techniques. The former method is apparent if proximity-based positioning has been used, thus objects which are unrelated to the current interest will be further from the viewpoint, hence reduced in detail and size. This provides a fish-eye view based purely on three-dimensional perspective. This was combined with a cluster based fish-eye view which attempted to reduce the complexity of the graph while still maintaining the context of the overall structure. This technique assigns elements within similar areas of 3D space to clusters, with neighbouring clusters being assigned to higher level clusters. The technique is similar to the nested graphs of Harel [Harel88]. Using these clusters, only knowledge elements close the viewpoint are displayed individually, at further distances only the clusters are visible.

A definite similarity is evident between the structure of large knowledge bases and the structure of software systems. Software systems can be considered as large, complex knowledge or fact databases consisting of many related components. Ideas of comprehension and visualisation of knowledge bases should be applied relatively easily to the problems of software visualisation. There is already a great deal of active research in the field of information visualisation, and more recently investigating the application of 3D graphics and virtual environment technology [Young96b]. It should be possible to advance 3D software visualisations using ideas applied in the fields of 3D information visualisation, graph layout and virtual environment design.

Figure 6

Figure 6. SemNet visualisation
Image courtesy of Kim Fairchild,
Institute of Systems Science, National University of Singapore.
http://panda.iss.nus.sg:8000/kids/fair/webdocs/viz/viz-1.html

5.6. GraphVisualizer3D

GraphVisualizer3D (GV3D) [Ware93, GV3D95] is a research project under development at the University of New Brunswick (Figure 7, a-d). The aim of GV3D is to utilise 3D visualisations and virtual environment technology to provide a more readable and comprehensible graph visualisation. The project is aimed at displaying predominantly directed graphs, though the focus of the development has been to visualise software structures. The structures in this case are component relationships within object oriented C++ programs, examples cited are: software module dependencies; software usage; and inheritance relations. Ware et al. [Ware93] describe the design issues and architecture of GV3D and include details of empirical evaluations performed on the system, which include an investigation into various interfaces and interaction techniques.

Ware et al. investigate the use of a number of different interaction techniques both for inclusion in the GV3D project [Ware93] and also as a separate study [Ware94]. The study evaluated the occurrence of errors in a graph comprehension task in which the users were restricted to using a number of different interaction techniques. The results of the study highlight an important decrease in errors, thus an increase in graph cognition, when a combination of stereoscopic display and viewpoint manipulation techniques are used. The study also confirmed previous research which indicated that the use of viewpoint manipulation alone was more effective than a stereoscopic display alone. These results indicate the importance of the virtual environment interface in producing useable and useful visualisations.

GV3D draws on a number of ideas from SemNet and is hinged upon the same concepts, however GV3D aims to improve upon SemNet in a number of ways. One area of divergence is a shift from the predominantly automatic node placement of SemNet to the more manual oriented placement of GV3D. Ware et. al believe that the most important heuristic for graph layout is semantic clustering, that is grouping nodes with respect to their semantic meaning as opposed to their position within the graph structure. They state that it is unreasonable to expect current layout algorithms to achieve more than a first approximation to the layout of software entities in space. The reason being that good layout is based greatly upon the high level semantics of the nodes, this cannot be formalised or automated at present. The graph layout used in GV3D is based upon elementary layout algorithms but supplemented by extensive manipulation and interaction techniques.

Similar to SemNet, GV3D also includes the notion of composite nodes and arcs [Harel88]. This technique allows reduction of the visual complexity of a graph structure by representing similar nodes and arcs as a single composite node or arc (the measure of similarity is arbitrary). These composite nodes can then be collapsed or expanded (opened or closed) as needed, depending on the focus of the viewer. GV3D copes with composite nodes by performing an identical layout on the sub-graph enclosed in the composite node. The composite node is then scaled to completely enclose the sub-graph, for this reason the node size cannot be used to display attributes. Opening or expanding a node results in the contents being displayed.

Input to GV3D is provided via a proprietary Graph Description Language (GDL). This language allows the permanent storage of graph structure, visual representation and layout information. Graph structures to be visualised, such as the C++ software components, must first be converted to the GDL format. Interaction with the GV3D visualisations is provided by two main methods, both of which use a standard high-resolution monitor for display. The viewer can use the monitor alone, making use of 3D Widgets to control position and velocity along each of the co-ordinate axes. Alternatively the viewer can also use a head tracked stereoscopic display. This provides the viewer with a stereo display and also allows head movement to manipulate the viewpoint thus providing essential perceptive cues which greatly aid depth perception and object recognition.

Figure 6
Figure 6
Figure 6
Figure 6
(a)
(b)
(c)
(d)
Figure 7 (a-d). Various views of a network using GraphVisualiser3D
Images courtesy of Glenn Franck, University of New Brunswick.
http://www.omg.unb.ca/hci/projects/hci-gv3D.html

5.7. FDP-based Software Visualisations

A number of prototype software visualisation systems have been produced which investigate using 3D to provide a structural overview of software. This structural overview typically involves visualising the dependencies between software components as a (generally large) directed graph. Layout of such a graph is non-trivial and two dimensional representations very rapidly become an unreadable mess. One approach adopted by several research systems is to use a force directed placement (FDP) algorithm to perform the layout. This algorithm was originally applied to 2D graph layout, however extensions to 3D have shown a great deal of promise. The use of 3D in such a case is justified solely by providing a greater working volume for the graph layout. This increase in available space does not solve the problem of performing layout on large graphs, it merely increases the graph size limit.

The FDP algorithm [Young96b, Fruchterman91] is a simple, yet effective technique of modelling the graph structure as a physical system. There are countless variations on this theme including various models for the system, such as simulated annealing. One simple FDP algorithm models the nodes of the graph as having a repulsive force, with nodes repelling one another. The magnitude of this repulsive force increases as nodes are brought closer together. To maintain an equilibrium in the graph, the arcs between nodes are modelled as imparting an attractive force between the nodes they connect. This attractive force is typically proportional to the length of the arc and exhibits similar behaviour to a spring. The initial graph layout will generally have the system in a state of high energy. After a number of iterations the nodes will reorganise themselves in an attempt to achieve a minimum energy state. On achieving this state, the layout will be optimised for arc length and node spacing, though this does not necessarily mean a clear or understandable graph.

A number of research systems have been developed using the FDP algorithm to produce 3D graph layouts. A few of these systems are described briefly here.

5.7.1. Narcissus

Narcissus [Hendley95] is an information visualisation system developed at the University of Birmingham for creating 3D visualisations of large directed graph structures. Specifically, Narcissus was aimed towards visualising dependencies between components in a software system, but was closely related to work on a system called Hyperspace which aimed at visualising collections of hypertext documents. The visualisations in Narcissus are created as wire-frame 3D graphs with the software components represented as spherical nodes, and the dependencies as directed arcs. Layout of the graph is performed using a force directed placement algorithm which runs continuously meaning that new nodes introduced will move immediately towards their equilibrium point.

5.7.2. SHriMP Views

SHriMP (Simple Hierarchical Multi-Perspective views) [Storey95] is a tool for visualising large graphs, in particular for visualising component dependencies within software systems. SHriMP is incorporated within the Rigi reverse engineering system and aims to provide an overview of software structure allowing both detailed and high-level visualisations while still maintaining a notion of position or context within the complete graph structure. SHriMP makes use of two techniques, nested graphs [Harel88] and fisheye views to provide both abstraction and focusing of detail as necessary.

5.7.3. Source Code Visualiser

An FDP based source code visualiser was developed by Kings [Kings95] at British Telecomm. The system took facts extracted from C source code using a proprietary code analysis tool and produced a three dimensional graph structure of these facts and their relations. The visualiser is implemented as a set of utilities or tools rather than a single program. It makes use of other available applications such as the Mosaic WWW browser for fetching files from a central server.

5.7.4. Zebedee

Zebedee is another FDP based 3D graph layout tool. It takes a proprietary graph file as input and produces a 3D layout within a virtual reality system (Figure 8). Graph structures to be visualised, such as program information, are first converted to this file format. This allows for greater flexibility using small, task specific conversion tools. Zebedee has been applied to visualising software as both control flow and function call hierarchies. One feature of Zebedee enables nodes to placed into tiers and constrained to moving only within that tier. This forces a hierarchy upon the graph structure and preserves a more organised and meaningful appearance. Zebedee has also been integrated into a WWW tool using the Netscape Navigator browser. This allows limited interaction between other code, visualisation and documentation views.

Figure 8

Figure 8. Call relationships within a module, visualised using Zebedee.

6. Conclusions

This report has described the current state of the art in software visualisation, and in particular the rapidly developing field of 3D software visualisation. A number of 3D software visualisation systems have also been described in order to illustrate the direction in which the research is heading. Particular attention has been paid to the definition of software visualisation and related topics; to it's application to software maintenance; and to the current state and direction of research effort. There are a number of conclusions which can be made:

The field of software visualisation is still not clearly defined.
As yet, there is still no common consensus on the definition of software visualisation and other related terms. Worse still, many of the terms are used within different context and are often interchanged, such as software visualisation and program visualisation. A firm definition of the terms within the field of software visualisation, or even computation visualisation as suggested by Stasko [Stasko92a], is needed to enable meaningful and unambiguous discussion.
The scope and range of current software visualisation systems is too small.
At present, software visualisation systems concentrate only on small, 'toy' programs, with few exceptions. For software visualisation to become a viable tool, widely accepted within the software maintenance community, then it must address the needs of industrial applications. The majority of techniques available in the software visualisation arsenal tend to fail when applied to large, industrial strength applications. It may not be possible to devise generic techniques which apply for both small and large programs. In such a case, more research is needed to investigate the development and, in particular, the integration of separate techniques.
Their is a current lack of, and need for, empirical evaluations of software visualisation systems.
A large number of software visualisation systems have already been produced, the majority of which are research prototypes. At present their is no empirical method for evaluating the effectiveness or worth of particular systems. Similarly, other than the various taxonomies for classifying systems, there is no empirical method for comparison between systems. Without a suitable method for evaluating systems there is no method for evaluating new techniques. This can result in acceptance of a technique because 'it looks pretty' or 'sounds like a good idea'. There is little point devising new visualisations without the ability to determine if they provide any improvement.
An improved level of automation and integration is required.
Many of the software visualisation systems developed rely heavily on user interaction and aid in generating the visualisations. This aid can come in various levels, from instrumenting or pre-parsing source code to defining and creating the actual visualisation itself. There is a definite need for a greater level of automation to increase user acceptance of such tools. Similarly, integration of visualisation tools within the software development environment is an important goal. Only when these two areas have been addressed more thoroughly will we begin to see visualisations become more common place. The effort required in visualising software (source code) should not greatly exceed that required to compile and run it.
Research must address the need of users of the visualisations.
Software visualisation research needs to take a more detailed look at what the user actually wants to see, and how they would like to see it. At present, many systems produce wonderful, interesting pictures or animations which, while looking very impressive, provide little information of use to the user. Alternatively they present information which could often be obtained through more conventional approaches, imparting a lower overhead cost (either in computing resources or time). Similarly, the information they present may be clearer or more immediately identified when obtained from other sources. Research needs to concentrate on how the technology can aid the user, rather than creating merely 'pretty pictures'.

Regardless of the above comments, it seems safe to assume that research and interest into software visualisation will continue to grow. As hardware technology improves and prices continue to fall, the overheads of creating visualisations will become less of a concern. Software visualisation has great potential to provide useful, time saving support to the software maintenance (and possibly development) process. Whether or not this potential is realised remains uncertain, though the deciding factor will undoubtedly be it's acceptance by industry.

References

[Brooks83]
R. Brooks,
Towards a Theory of the Comprehension of Computer Programs,
International Journal of Man-Machine Studies,
Vol. 18, No. 6, pp. 543 - 554, 1983

[Brooks87]
F.P. Brooks Jr.,
No Silver Bullet: Essence and Accidents of Software Engineering,
IEEE Computer, No. 20, pp. 10-19, 1987

[Brown93]
M.H. Brown and M.A. Najork,
Algorithm Animation Using 3D Interactive Graphics,
DIGITAL, Systems Research Center, September 1993
http://www.research.digital.com/SRC/publications/src-rr.html

[Consens92]
M. Consens, A. Mendelzon and A. Ryman,
Visualizing and Querying Software Structures,
14th International Conference of Software Engineering ICSE'92,
Melbourne, Australia, May 11-15, 1992

[Domingue92]
J. Domingue, B.A. Price and M. Eisenstadt,
A Framework for Describing and Implementing Software Visualization Systems,
In Proceedings of Graphics Interface'92,
pp. 53-60, Vancouver, Canada, May 1992
ftp://watson.open.ac.uk/pub/documents/Domingue-Viz-GI92.ps.Z

[Fairchild88]
K.M. Fairchild, S.E. Poltrock and G.W. Furnas,
Semnet: Three-dimensional Graphic Representations of Large Knowledge Bases,
in "Cognitive Science and Its Applications for Human-Computer Interaction",
Fairlawn, NJ: Lawrence Erlbaum Associates, 1988
http://panda.iss.nus.sg:8000/kids/fair/webdocs/semnet/semnet-1.html

[Fruchterman91]
T.M.J. Fruchterman and E.M. Reingold,
Graph Drawing by Force-Directed Placement,
Software Practice and Experience,
Vol 2, Part 11, November 1991

[Fry95]
C. Fry and H. Lieberman,
Programming as Driving : Unsafe at any Speed?,
Demonstration, ACM Conference on Computers and Human Interface [CHI-95],
Denver, April 1995
http://lieber.www.media.mit.edu/people/lieber/Lieberary/ZStep/57-Chevy/57-Chevy.html

[Gelernter92]
D. Gelernter and N. Carriero,
Coordination Languages and their Significance,
Communications of the ACM, Vol. 35,
No. 2, pp. 97 - 107, February 1992

[GV3D95]
The GraphVisualizer3D Project,
On-Line project overview, University of New Brunswick, 1995
http://www.omg.unb.ca/hci/projects/hci-gv3D.html

[Harel88]
D. Harel,
On visual formalisms,
Communications of the ACM, 31(5), May 1988

[Hendley95]
R.J. Hendley, A.M. Wood, R. Beale and N.S. Drew,
Narcissus: Visualising Information,
University of Birmingham, 1995
ftp://ftp.cs.bham.ac.uk/pub/authors/R.J.Hendley/ieeeviz.ps.Z

[Hill93]
W.C. Hill and J.D. Hollan,
History-Enriched Source Code,
Computer Graphics and Interactive Media Research Group,
Bell Communications Research, Submitted to ACM UIST '93, 1993

[Jerding94]
D.F. Jerding and J.T. Stasko,
Using Visualisation to Foster Object-Orientated Program Understanding,
GVU Center, College of Computing,
Georgia Institute of Technology,
Technical Report GIT-GVU-94-33, July 1994
ftp://ftp.cc.gatech.edu/pub/gvu/tech-reports/94-33.ps.Z

[Kings95]
N.J. Kings,
Software Visualisation,
BT Corporate Research Programme, 1995

[Koike95]
VisuaLinda: 3D Visualisation of Parallel Linda Programs,
On-Line Project Abstract, Koike Labs,
University of Electro-Communications in Tokyo, 1995
http://www.vogue.is.uec.ac.jp/vlinda.html

[Kraemer93]
E. Kraemer and J.T. Stasko,
The Visualisation of Parallel Systems: An Overview,
Journal of Parallel and Distributed Computing (18),
pp. 105 - 117, 1993

[Lieberman95]
H. Lieberman and C. Fry,
Bridging the Gulf Between Code and Behaviour in Programming,
ACM Conference on Computers and Human Interface [CHI-95],
Denver, April 1995
http://lieber.www.media.mit.edu/people/lieber/Lieberary/ZStep/Bridging/Bridging.html

[Meekel88]
J. Meekel and M. Viala,
Logiscope : A Tool For Maintenance,
Proceedings of ICSM '88, pp. 328 - 334, 1988

[Miller93]
B.P. Miller,
What to Draw? When to Draw? An Essay on Parallel Program Visualisation,
Journal of Parallel and Distributed Computing (18),
pp. 258 - 264, 1993

[Muthukumarasamy95]
J. Muthukumarasamy and J.T. Stasko,
Visualising Program Executions on Large Data Sets Using Semantic Zooming,
GVU Center, College of Computing,
Georgia Institute of Technology,
Technical Report GIT-GVU-95-02, 1995
ftp://ftp.cc.gatech.edu/pub/gvu/tech-reports/95-02.ps.Z

[Myers90]
B.A. Myers,
Taxonomies of Visual Programming and Program Visualisation,
Journal of Visual Languages and Computing,
(1), pp. 97-123, 1990

[Najork95]
M. Najork,
Visual Programming in 3D,
Dr. Dobb's Journal, pp. 18-31, No. 237, December 1995

[Price93]
B.A. Price, R.M. Baeker and I.S. Small,
A Principled Taxonomy of Software Visualisation,
Journal of Visual Languages and Computing,
No. 4, pp. 211-266, 1993
http://www-cs.open.ac.uk/~doc/jvlc/JVLC-Body.html

[PVMTrace95]
The PVMTrace project,
On-Line thesis proposal, University of New Brunswick, 1995
http://www.omg.unb.ca/hci/projects/hci-pvmtrace.html

[Reiss94]
S.P. Reiss,
3-D Visualization of Program Information,
In proceedings of GD'94: DIMACS international workshop on graph drawing,
pp. 12-24, Princeton, New Jersey,
USA, October 1994

[Roman92]
G. Roman and K.C. Cox,
Program Visualization: The Art of Mapping Programs to Pictures,
Department of Computer Science,
Washington University, 1992

[Roman93]
G. Roman and K.C. Cox,
A Taxonomy of Program Visualisation Systems,
IEEE Computer, December 1993

[Shneiderman80]
B. Shneiderman,
Software Psychology,
Cambridge MA: Winthrop Publishers Inc., 1980

[Sommerville92]
I. Sommerville,
Software Engineering,
Fourth Edition, Addison-Wesley publishers, 1992

[Standish84]
T.A. Standish,
An essay on Software Reuse,
IEEE transactions on Software Engineering,
Vol. SE-10, No. 5, pp. 494-497, September 1984

[Stasko92a]
J.T. Stasko,
Three-Dimensional Computation Visualisation,
GVU Center, College of Computing,
Georgia Institute of Technology,
Technical Report GIT-GVU-92-20, 1992
ftp://ftp.cc.gatech.edu/pub/gvu/tech-reports/92-20.ps.Z

[Stasko92b]
J.T. Stasko and C. Patterson,
Understanding and characterizing software visualisation systems,
Proceedings of the IEEE 1992 workshop on Visual Languages,
Seattle, Washington, pp. 3 - 10, 1992

[Storey95]
M-A.D. Storey and H.A. Muller,
Manipulating and Documenting Software Structures Using SHriMP Views,
Proceedings of ICSM '95, pp. 275 - 284,
Opio (Nice), France, October 17 - 20, 1995

[Ware93]
C. Ware, D. Hui and G. Franck,
Visualizing Object Oriented Software in Three Dimensions,
In proceedings of CASCON '93 (IBM Centre for Advanced Studies),
pp. 612-620, Toronto, Ontario, Canada, October 1993
http://www.omg.unb.ca/hci/papers/IBM-CAS93-visualizing.ps.gz

[Ware94]
C. Ware and G. Franck,
Viewing a Graph in a Virtual Reality Display is Three Times as Good as a 2D Diagram,
In 1994 IEEE Conference on Visual Languages,
pp. 182-183, St. Louis, Missouri,
USA, October 1994

[Watson95]
A.B. Watson and J.T. Buchanan,
Towards Supporting Software Maintenance with Visualisation Techniques,
University of Strathclyde, 1995

[Young96a]
P. Young,
Program Comprehension,
Internal Report, Centre for Software Maintenance,
University of Durham, May 1996
http://www.dur.ac.uk/~dcs3py/pages/work/Documents/index.html

[Young96b]
P. Young,
Three Dimensional Information Visualisation,
Technical Report, Centre for Software Maintenance,
University of Durham, March 1996
http://www.dur.ac.uk/~dcs3py/pages/work/Documents/index.html


This page is maintained by Peter Young, please send any comments, jokes, insults or general abuse to (peter.young@durham.ac.uk).
Last updated: Monday 1 July, 1996.