Nicole Sansone Ruiz

About

How to See an Image in Data: Uncovering Representation in Digital Images

Following our recent exhibition programme on the historical relationship between drawing and photography, Nicole Sansone Ruiz considers the contemporary digital image and its relationship to drawing, with respect to changes in our theoretical understanding of representation.

What is it about drawing and photography that seems to make them such a natural fit for each other? They work in very different ways: drawing erupts from the mechanical energy of the drawing body, while photography reveals itself in the contours of light and shade, over time. So, is the difference between these two mediums one in kind or in degree

One similarity between drawing and photography is that they are two mediums that have lead the participation of the arts in the sciences. Sixteenth-century anatomists and botanists often worked with artists and printers to produce images that would support their written work.[1] The mid-19th century text The Pencil of Nature outlined all the ways in which photography could be useful to collectors for documenting their collections. These museological benefits proved doubly useful in the documenting of maladies. “Photography became the paradigm of the scientist’s true retina” and a mantra for, among others, Dr. Jean-Martin Charcot, whose pioneering work in neurology was heavily reliant on his photographic practice.[2] 

This shared point of reference supports two ideas about the relationship between photography and drawing. First, that they share some quality, by virtue of the technical properties of their medium, that allows humans to see things for what they ‘really’ are. The second point, which follows from the first, is that photography and drawing can both be used as supplements and/or complements to the naked human senses. This is an idea that actually has much deeper historical roots that go back to Galileo’s use of the telescope to describe the surface of the moon. In the Siderius Nuncius, Galileo not only used drawings to enhance his textual arguments (which historians often credit as the first example of drawing entering into astronomy) but he spends a great deal of time in his writing elaborating how a telescope works, and what it might be used for. This three-pronged approach at laying out a scientific exegesis introduced the idea that the human senses alone were not sufficient in grasping all that the natural world has on display, and by extension, that human vision was in itself incomplete and perhaps not to be trusted.[3]

Today we continue this legacy of questioning the relationship between optics and representation but with an added emphasis on technology. The proliferation of digital images and the media that support them, and their move into mainstream politics and culture, all beg the question of where representation happens, and who — or what — controls it. Scholarship on issues of gender and sexuality, coloniality, and identity are useful here in thinking about the non-optic or extra-optic ways that an image can come into being. Through shared interest in vision as a mode of objectification, race and gender-critical scholars have posed effective challenges to the regimes of vision, representation, and reality.

Luciana Parisi and Tiziana Terranova have taken this perspective as the foundation from which to argue for an expanded engagement with digital images and aesthetics. [4] They credit these critical canons with opening up the field of image study to questions beyond representation and reality, and note that this line of inquiry also has a historical legacy in the philosophical investigation of the materiality of images themselves.[5] Freed from the dialectical questions of representation and reality we can begin to engage with digital images not as “deceptive, unreal simulations, threatening embodied experience” but instead for the ways in which images more fully speak to the experiences and potentials of bodies — emotional, fleshy, gendered, etc. — in space.[6]

Parisi and Terranova believe that in place of questions of representation and reality what digital aesthetics can more productively address itself to are questions of affect and how affect as a digital aesthetic situates itself in sociocultural contexts. Central to the idea of affect is what Brian Massumi has noted as the gap between content and effect; that “the strength or duration of an image’s effect is not logically connected to the content in any straightforward way.”[7] The break between content and effect (or our inability to reconcile the two) becomes more acutely linked to materiality in the digital image. In the digital image layers of abstracted code and logic become the viewable image, and the means by which that image appears on phones or screens is always hidden, out of view. This makes asking the question of where affective relations take place, or are organized, particularly tricky.

It’s also a question that becomes more important as we come to rely on larger data sets to make digital images. The 2009 unveiling of the Google Earth Engine revealed that Google had recently been successful in downloading and cataloguing over forty years of satellite data, elevation data and atmospheric data — petabytes of information. This information, they explained, “had been difficult to get access to … and there’s so much data that even if you could access it, it’s very difficult to analyze.”[8] The tools that made this possible also power Google Earth and are key to Google’s philanthropic projects, tracking environmental risk factors such as malaria and deforestation, many times in conjunction with policy makers.

Hito Steyerl has written of the problematic use of big data at the levels of government and bureaucracy. She calls it apophenia, or the phenomenon of perceiving patterns in random data sets. Steyerl notes that in today’s society the picture that we create of ourselves and that is visible to policymakers and modes of governance is one that is taken as definitive, despite being admittedly “probabilistic projections.”[9] “In practice you become coextensive with the data-constellation you project,” Steyerl writes; “Social scores of all different kinds — credit scores, academic scores, threat scores — as well as commercial and military pattern-of-life observations impact the real lives of real people, both reformatting and radicalizing social hierarchies by ranking, filtering, and classifying.”[10]

Works by Nicholas O’Brien, Jesse Maclean, and the Scandinavian Institute for Computational Vandalism (S.I.C.V.)— which all featured in the exhibition Contours at The Photographers’ Gallery in spring this year — can elicit a number of different productive readings from the relationship between drawing and photography. I think where they are most usefully considered is in how they might be seen to take aim at precisely the gap between content and effect, material and meaning. Nicholas O’Brien makes use of a free version of a popular CAD modeling software often used by professionals engaged in spatial design. His movements and on-screen creations are perverted, unimaginable, un-realizable; their images, art. Jesse McLean assembles and dissembles images of Iraq to pull at the seams of the sociolinguistic qualifications that are supposed to give images in circulation meaning and potency. And yet arguably the effect of some images remain — harrowing and uncanny in their hollow incompletion. S.I.C.V.’s algorithm knocks down the mysticism that’s attributed to realistic capture and representation by playing on what Matthew Fuller has described as computers’ stupidity; stupid in the sense that “Computers … do exactly as they are told. Their capacity for memory, like this function of stupefied perfect recall, is what makes them so effective for archiving, and indeed so disturbing as an agent of social control.”[11]

Seen in this way, the works in the exhibition adopt a variety of approaches and materials to challenge ideas of meaning: of what meaning is, how it is created, and how it functions materially. Drawing and photography has historically emphasized the importance of the image as the site for meaning. We look at a photograph to verify that something has happened. Police sketches are compiled from oral statements to help track down perpetrators. New media artists like those exhibiting in Contours add a healthy dose of skepticism to this practice, not by investigating the images they create but by searching out the logical boundaries of the image as it is being created. At what point can we say a digital process congeals into the image? As technology allows for images increasingly outside of the human sensible range — from satellites in space, from uninhabitable parts of the earth, from deep inside our bodies — keeping an eye towards the site of representation becomes our most important critical task.[12]

Notes

[1] Mary G. Winkler and Van Helden Albert, “Representing the Heavens: Galileo and Visual Astronomy,” Isis, n.d. The authors also note that this practice already in development in the late 15th century; see Schultz, Art and Anatomy in Renaissance Italy.

[2] For more on this, see: Georges Didi-Huberman and J. M. Charcot, Invention of Hysteria: Charcot and the Photographic Iconography of the Salpêtrière (Cambridge, Mass: MIT Press, 2003).

[3] For more on this, see: Joseph Vogl, “Becoming-Media: Galileo’s Telescope,” trans. Brian Hanrahan, Grey Room, Inc., New German Media Theory, No. 29, no. Fall, 2007 (n.d.): 14–25.

[4] For an excellent overview of this argument see: Luciana Parisi and Tiziana Terranova, “A Matter of Affect: Digital Images and the Cybernetic Re-Wiring of Vision,” Parallax 7, no. 4 (October 2001): 122–27, doi:10.1080/13534640110089294.

[5] “… it would be misleading to think of the material qualities of the image as a new phenomenon exclusively associated with the emergence of digital media, as if digitization constituted a mere technical innovation in the homogenous negative tradition of vision and its relation to the body. … It is possible to refer to another history of images, vision and their relation to the body which is less concerned with central or subjective perspectives. Lucretius’s simulacra, for example, affirm the sensible qualities of images, sounds, and smells. Baruch Spinoza refers to images as compositions of ‘anonymous particles’ endowed with the power to affect. Bergson’s matter is defined as an ‘aggregate of images’, where a body is ‘an image which acts like other images, receiving and giving back movement’, and therefore a centre of action that cannot give birth to a representation. Walter Benjamin’s work also expresses a preoccupation with the tactile qualities of photography and its capacity to capture images which escape natural vision; and Luce Irigaray’s asserts that the shadows on the Platonic cave are no reflections of a higher reality but the essence of matter itself.” Quoted from ibid., 124–125.

[6] Ibid., 125.

[7] Brian Massumi, Parables for the Virtual: Movement, Affect, Sensation, Post-Contemporary Interventions (Durham, NC: Duke University Press, 2002), 218.

[8] Google Developers, Introduction to Planetary-Scale Geospatial Analysis with Google Earth Engine, accessed April 13, 2016, https://www.youtube.com/watch?v=Hr3F0s9oKwQ&feature=youtu.be.

[9] “A Sea of Data: Apophenia and Pattern (Mis-)Recognition | E-Flux,” accessed May 20, 2016, http://www.e-flux.com/journal/a-sea-of-data-apophenia-and-pattern-mis-recognition/.

[10] Ibid.

[11] “Vandalist Iconophilia,” accessed June 14, 2016, http://www.editorialconcreta.org/Vandalist-Iconophilia.

[12] This is a lesson, in part, already explored in the dismanting of photography’s truth claim, and it is worth revisiting again with the rise of the digital image. For more on this, see and Susan Sontag’s On Photography; Tom Gunning’s What’s the point of an index? or, Faking Photographs; Lev Manovich’s What is Digital Cinema?, for e.g.