Multimodal analysis and digital technology

Abstract

The aim of this chapter is to explore directions for advancing multimodal analysis through the use of digital technology which is the research agenda currently underway in the Multimodal Analysis Lab, Interactive & Digital Media Institute (IDMI) at the National University of Singapore. Historically, the impact of technology on knowledge is evident from the introduction of the printing press in the late fifteenth century (Eisenstein, 1979). Today, 'computerization turns media into computer data" (Manovich, 2001: 45). The recursive nature of the computerisation process, where 'images scater into data, data gathers into images' (Galison, 2002: 322) permits the visual display of data flow which is experienced over time, '[be it in science, statistics, archiecture, design, digital art, or any combination of these' (Pau, 2003: 175). There are few constants in the digital environment where everyhing is variable and open to manipulation and recontextualisation. However, does digital technology function to reproduce more effectively and efficiently knowledge which already exists, or does it produce new theories and approaches? How does digital technology function to constrain knowledge? What are the implications for multimodal analysis? In what follows, these questions are explored. The theoretical basis for the discussion is Michael Halliday's social semiotic theory (Halliday, 1978, 1994 [1985]; Halliday, Matthiessen, 200).