Tabs, Pads and Boards: Why Apple et al will make a HDTV

Xerox Palo Alto Researchers using Tabs, Pads and Boards (Weiser 1991)

There are many discussions across the internet about why Apple will not make a HDTV, ranging from simple cost, likely small return on investment, competitive environment, difficulty in innovating, long replacement cycles, and perceived customer lack of interest (Arment 2011, Dixon 2011, Hughes 2011, Lisagor 2011, cf. McCracken 2011). However, the majority of these articles make the fundamental mistake of treating a stand-alone product as a self-standing revenue stream, rather than considering it within the growing ecology of media appliances that Apple has been building. At the reception end of the very impressive Apple vertical media system Apple has, surprisingly, a rather poor product line. There are computers (laptop and desktop), which aren't ideal for watching mass media like films, iPads, which whilst very innovative are uncomfortable to hold for the duration of a television programme or film, iPhones and iPods, which whilst handy out-and-about are unlikely to be used to consume mass time-based media, suffering, as they do, from the same issue as the iPad.1 Therefore that leaves only the television, that much maligned representative of 20th Century media as the key communal access point into the new media environment, and clearly Apple currently doesn't manufacture or sell one.

Of course, they do make the so-called 'AppleTV', a television-add-on product that connects to an existing TV HDMI port. But I believe this is a transitional product that allows wide-spread testing of the value of a strategic technical innovation in HDTV, representing the beginning of the development of a protocol that enables this to take place through its AirPlay technology (see Gillmor 2011). As it stands AppleTV is a curious media-receiver for the television, supplying some computational intelligence into a dumb output device and pointing to the possibilities of streaming media. This, in itself, raises the question: is the television of the 20th Century still fit for purpose?

First, we should note that something has fundamentally changed in the media landscape. Foremost to this are the rise of computational media, and the ubiquity of the computer microprocessor. This has led to a rapid tsunami of disruptive change across the traditional media sector as media have become increasingly computational in form and function. Strangely, this has caught the traditional media completely by surprise even as it revolutionised the production and distribution mechanisms for media content and which they took advantage of to reduce their costs (Kiss 2011). At the heart of this transformation lies the magic of software/code which has enabled media to flow increasingly rapidly between various devices, spaces and places (Berry 2011), and which the traditional media has viewed with great suspicion and attempted to prevent through a mixture of law, digital rights management and sheer bloody-mindedness. The music industry missed the change in customers behaviour when they no longer wanted to purchase lumps of expensive plastic to play on clunky music players, and this has been followed by a similar change in television and film consumption. As Steve Jobs observed in 2010,

The way that we market movies is undergoing a radical shift. It used to be that you spent a fortune on advertising on TV running your trailers. But now you can advertise on the Web….When we went to the music companies, we said “who is your customer?” And they said, “Best Buy, Tower”…their distribution partners. But that wasn’t their customer. They needed to recognize who their true customer was….So what changed in the music business was not the back end, but the front end. The way that you market to the consumer….The film industry needs to embrace that. And it needs to let people watch the content they want to watch, when they want to watch it and where they want to watch it (Jobs quoted in Paczkowski 2010, emphasis added).

The key here is that it is the front-end that is suddenly under pressure from the massive changes bought forward by the rapid increase in processing ability, software/code and storage. The television and film industry isn't quite sure who their customer is anymore, and they are pretty certain it is not the kinds of people that rip their films and programmes and stream them around their houses. But these new technologies, that are now cheap enough to build into so-called smart televisions, are about to make everyone interested in real-time streaming information and data. The media industry also do not understand the new ways in which people want to consume their media both across and through devices – starting on one and finishing on another.

Indeed, we are on the cusp of a fundamental redefinition of what the television screen is: no longer the dumb output device that sits at the end of a uni-directional cable so that a corporation can sell your attention to the advertising industry.  Simultaneously it is clear that the personal computer is also in the middle of a pretty fundamental shift itself, with Jobs memorably describing desktop computers as 'going to be like trucks' (Fried 2010). The success of the iPad, and other new tablet-like devices, shows that what people want to be able to do with their media will become increasing important in both differentiating computational products, but also in structuring the technology and media industries.2

However, momentous as these changes appear to be, they were heralded by research undertaken at the Xerox Palo Alto research centre in the 1990s. There researchers were given the space and time to think about how we might use technologies to augment the work (and play) that we undertake in our everyday lives. More importantly they questioned the 'personal' in the personal computer and wondered if,

The arcane aura that surrounds personal computers is not just a "user interface" problem. My colleagues and I at PARC think that the idea of a "personal" computer itself is misplaced, and that the vision of laptop machines, dynabooks and "knowledge navigators" is only a transitional step toward achieving the real potential of information technology. Such machines cannot truly make computing an integral, invisible part of the way people live their lives. Therefore we are trying to conceive a new way of thinking about computers in the world, one that takes into account the natural human environment and allows the computers themselves to vanish into the background (Weiser 1991: 78).

Drawing on substantial theoretical and empirical research they highlighted the importance of a technology falling into the background of the activity that the user wanted to undertake. This they christened ubiquitous computing. Clearly, key to this is that the technical device or object should be merely a surface interface to the underlying task that a user wished to accomplish, whether playing a film, sending a letter, reading a book or so forth. For this to be less a problem of using the computer, with its arcane and clunky keyboard interface, and more a problem of accessing the media directly they argued,

Such a disappearance is a fundamental consequence not of technology, but of human psychology. Whenever people learn something sufficiently well, they cease to be aware of it. When you look at a street sign, for example, you absorb its information without consciously performing the act of reading.. Computer scientist, economist, and Nobelist Herb Simon calls this phenomenon "compiling"; philosopher Michael Polanyi calls it the "tacit dimension"; psychologist TK Gibson calls it "visual invariants"; philosophers Georg Gadamer and Martin Heidegger call it "the horizon" and the "ready-to-hand", John Seely Brown at PARC calls it the "periphery". All say, in essence, that only when things disappear in this way are we freed to use them without thinking and so to focus beyond them on new goals (Weiser 1991: 78).

Through a number of refinements and empirical experiments they settled on range of device categories that seemed to be needed to negotiate a computational media landscape, dividing them into three classes: tabs, pads, and boards: tabs are 'inch-scale machines that approximate active Post-It notes', pads are 'foot-scale ones that behave something like a sheet of paper (or a book or a magazine)', and boards are 'yard-scale displays that are the equivalent of a blackboard or bulletin board' (Weiser 1991: 80). It does not take much imagination to see that Apple's strategy has followed the Xerox research to a remarkable degree, except for one glaring exception (cf. Ozzie (2009) where he outlines Microsoft's 'three screens and a cloud definition'):3

• Tabs: iPhone, iPod
• Pads: iPad, iPad Mini, Macbook,
• Boards: AppleTV?

Again, it is critical that the Xerox team saw computation as a distributed system, not a self-contained device. That is, that they understood the importance of the network for computational media. This immediately transformed the kinds of information that each of these classes of technical device was able to use and transmit to others, and most importantly these devices were programmed to understand the importance of the real-time stream, above and beyond that of historical data and media. Indeed, they even referred to 'liveboards':

Liveboards can also be used as bulletin boards. There is already too much data for people to read and comprehend all of it, and so Marvin Theimer and David Nichols at PARC have built a prototype system that attunes its public information to the people reading it. Their "scoreboard" requires little or no interaction from the user other than to look... (Weiser 1991: 86).

Twitter (Gruber 2008)

Again, it is striking how contemporary the problems and solutions are that the Xerox team are gesturing towards. We are today entering a rapidly changing computational ecology being structured around the concept of the real-time stream (Berry 2011). Currently, most tools created to deal with the real-time stream have favoured temporality as a means of both representing and presenting the data, such as with Twitter, where the real-time stream tends to be viewed as a rapidly changing flow of information down a page (see image). However, this kind of representation is actually rather poor when you want to negotiate the information is contains, it needs, in other words, to allow the transformation of time-axis manipulation which Friedrich Kittler theorised as that '[which] shift[s] the chronological order of time to the parallel order of space – and spaces are things that can principally be restructured – [thus] written media become elementary forms that not only allow temporal order to be stored but also to be manipulated and reversed' (Krämer 2006).

This requires a large surface area that allows information to be moved around, recombined and re-presented in different ways. It also enables the juxtaposition, rotation, reversal and visualisation of data in interesting new ways. As Latour (1986) points out,

on paper, hybrids can be created that mix drawings from many sources. Perspective is not interesting because it provides realistic pictures ; on the other hand, it is interesting because it creates complete hybrids : nature seen as fiction, and fiction seen as nature, with all the elements made so homogeneous in space that it is now possible to reshuffle them like a pack of cards.

For the Xerox scientists this space could be created by the use of large screens that they called 'boards'. These boards allowed the user to project, send, manipulate, write on and generally visualise different forms of media on a collaborative, multiply authored, multiply viewable space in a communal environment. They explained:

Yard-size displays (boards) serve a number of purposes: in the home, video screens and bulletin boards; in the office, bulletin boards, whiteboards or flip charts. A board might also serve as an electronic bookcase from which one might download texts to a pad or tab. For the time being, however, the ability to pull out a book and place it comfortably on one's lap remains one of the many attractions of paper. Similar objections apply to using a board as a desktop; people will have to get used to using pads and tabs on a desk as an adjunct to computer screens before taking embodied virtuality even further... Boards built by Richard Bruce and Scott Elrod at PARC currently measure about 40 by 60 inches and display 1024x768 black-and-white pixels. To manipulate the display, users pick up a piece of wireless electronic "chalk" that can work either in contact with the surface or from a distance. Some researchers, using themselves and their colleagues as guinea pigs, can hold electronically mediated meetings or engage in other forms of collaboration around a liveboard. Others use the boards as testbeds for improved display hardware, new "chalk" and interactive software (Weiser 1991: 85).

We are already seeing the beginning of experiments in these new ways of spatialising the real-time stream, such as Pulse, Flipboard and Showyou on a pad device, but with the greater size of a television screen, or board, new ways to present, negotiate and visualise media will become increasingly possible. However, the Xerox engineers cautioned:

The technology required for ubiquitous computing comes in three parts: cheap, low-power computers that include equally convenient displays, a network that ties them all together, and software systems implementing ubiquitous applications (Weiser 1991: 85).

Clearly, the first two have been available for a while, with tablets and smart phones able to leverage their computational power to provide very convincing interfaces to the media environment, and networks built around WiFi, now starting to enable the transmission of real-time streams of data and media that allow live viewing and interaction. However, until now the software systems that would enable this ubiquitous interaction of different devices both of and in the media network have been slow to be developed because of the reluctance, previously mentioned, of the traditional media to give up their profitable transmission monopolies.

With AirPlay, however, Apple, has created a real-time streaming system that combined with technologies like iTunes Home Sharing, Twitter, Facebook, Youtube, Flickr and so forth requires only the terminal output device, the board, to complete the vertical media infrastructure and consumption network (cf. Gerrish 2011 who argues the HDMI interface ports will be the key site of media competition). Haptic, networked, smart and communal, this would complete the ubiquitous computing trinity that Xerox dreamed of over twenty years ago, and points the way towards both the real-time streams and the online media clouds that allows a whole new digital media ecology to blossom and grow.4


Notes

1. Other problems remain with tab and pad type devices most noticeably the current tethered reliance on the PC, see Gruber (2011) for an interesting discussion of this. However, increasingly the tablets are becoming self-standing with recent Apple iterations of the iPad being self-booting and configuring and the Microsoft Surface taking on a hybrid laptop/tablet form. 
2. Here one can only note the remarkable new self-definition of Apple that Steve Jobs introduced in a keynote in 2010 where he stated that Apple lay between liberal arts and technology, and which he reiterated again in 2011 stating: 'it's in Apple's DNA that technology alone is not enough. That it's technology married with liberal arts, married with the humanities, that yields us the result that makes our hearts sing' (see Macworld 2011). 
3. 'I don't think it's – they'll be totally cloud-based in the realm, in the – let me back up. There's kind of a – in order to get things going across the company you need meetings, you need to say things, say them again, and say them again. So we say three screens and a cloud, three screens and a cloud, three screens and a cloud, throughout the company. And what that means is everything we deliver, from a user experience perspective, will be – will have some aspect of its value delivered across the PC class of device, the phone class of device, and the TV class of device. Every one of them will have something, and all will be connected to the cloud. That will bring them all together' (Ozzie 2009).
4. This is how Apple describe airplay: 'You have great HD videos on your iPad and some friends on your couch. Or you’re in the middle of an epic action scene that could use a little more screen. Just tap the AirPlay icon on your iPad and see it on your HDTV. Make sure your iPad and Apple TV are connected to the same Wi-Fi network, and the AirPlay icon appears automatically' (Apple 2011). 

Bibliography

Apple (2011) Stream Movies and Music Wirelessly with AirPlay, accessed 18/04/2011, http://www.apple.com/ipad/features/airplay.html

Arment, M. (2011) The often-rumored Apple HDTV, accessed 18/04/2011, http://www.marco.org/2011/04/16/rumored-apple-hdtv

Berry, D. M. (2011) The Philosophy of Software: Code and Mediation in the Digital Age, London: Palgrave Macmillan.

Dixon, C. (2011) Apple and the TV Industry, accessed 18/04/2011, http://cdixon.org/2011/04/17/apple-and-the-tv-industry/

Fried, I. (2010) Steve Jobs at D8: Post-PC era is nigh, accessed 18/04/2011, http://news.cnet.com/8301-13860_3-20006526-56.html

Gerrish, C. (2011) Screen and Cloud: Wrong Way Wrong, Echovar: economies, internet, language, and culture, accessed 18/04/2011, http://blog.echovar.com/

Gillmor, S. (2011) Season 7, Techcrunch, accessed 18/04/2011, http://techcrunch.com/2011/04/04/season-7/

Gruber, J. (2008) The Unsatisfying State of Twitter Web Clients for the iPhone, Daring Fireball,  accessed 18/04/2011, http://daringfireball.net/2008/04/twitter_web_clients_for_the_iphone

Gruber, J. (2011) Cutting That Cord, Daring Fireball, accessed 18/04/2011, http://daringfireball.net/2011/04/cutting_that_cord

Hughs, N. (2011) Evidence points toward Apple releasing HDTV this year, accessed 18/04/2011,  http://www.appleinsider.com/articles/11/04/13/evidence_points_toward_apple_releasing_hdtv_this_year_report.html

Kiss, J. (2011) Will streaming TV online lead to the death of the big media players?, The Guardian, accessed 18/04/2011, http://www.guardian.co.uk/media/2011/apr/18/digital-video-streaming-online-netflix

Krämer, S. (2006) The Cultural Techniques of Time Axis Manipulation: On Friedrich Kittler’s Conception of Media, Theory Culture Society, 2006; 23; 93.

Latour, B. (1986) Visualisation and Cognition; Thinking with Eyes and Hands, Knowledge and Society: Studies in the Sociology of Culture Past and Present, 6: 5.

Lisagor, A. (2011) Apple TV Set, lonelysandwich.com, accessed 18/04/2011, http://lonelysandwich.com/post/4720873047/apple-tv-set

Macworld (2011) The iPad event in pictures, Macworld, accessed 18/04/2011, http://www.macworld.com/article/158288/2011/03/tech.html

McCracken, H. (2011) Saying Apple Will Never Do Something: Always Dangerous, accessed 18/04/2011, http://technologizer.com/2011/04/17/saying-apple-will-never-do-something-always-dangerous/

Ozzie, R. (2009) Remarks by Ray Ozzie, Chief Software Architect, Microsoft, on the Potential of Cloud Computing, Churchill Club, accessed 18/04/2011, http://www.microsoft.com/presspass/exec/ozzie/06-04-09ChurchillClub.mspx

Paczkowski, J. (2010) Apple CEO Steve Jobs Live at D8: All We Want to Do is Make Better Products, accessed 18/04/2011, http://d8.allthingsd.com/20100601/steve-jobs-session/

Weiser, M. (1991) The Computer for the 21st Century, Scientific American, accessed 18/04/2011, http://nano.xerox.com/hypertext/weiser/SciAmDraft3.html