Статті в журналах з теми "4607 Graphics, augmented reality and games"

The next generation of Augmented Reality (AR) games will require real and virtual objects to coexistin motionin immersive game environments. This will require the illusion that real and virtual objects interact physically together in a plausible way. TheMotion in Augmented Reality Games(MARG) engine described in this paper has been developed to allow these kinds of game environments. The paper describes the design and implementation of the MARG engine and presents two proof-of-concept AR games that have been developed using it. Evaluations of these games have been performed and are presented to show that the MARG engine takes an important step in developing the next generation of motion-rich AR games.

The sense of presence in augmented reality (AR) has been studied by multiple researchers through diverse applications and strategies. In addition to the valuable information provided to the scientific community, new questions keep being raised. These approaches vary from following the standards from virtual reality to ascertaining the presence of users’ experiences and new proposals for evaluating presence that specifically target AR environments. It is undeniable that the idea of evaluating presence across AR may be overwhelming due to the different scenarios that may be possible, whether this regards technological devices—from immersive AR headsets to the small screens of smartphones—or the amount of virtual information that is being added to the real scenario. Taking into account the recent literature that has addressed the sense of presence in AR as a true challenge given the diversity of ways that AR can be experienced, this study proposes a specific scope to address presence and other related forms of dimensions such as immersion, engagement, embodiment, or telepresence, when AR is used in games. This systematic review was conducted following the PRISMA methodology, carefully analysing all studies that reported visual games that include AR activities and somehow included presence data—or related dimensions that may be referred to as immersion-related feelings, analysis or results. This study clarifies what dimensions of presence are being considered and evaluated in AR games, how presence-related variables have been evaluated, and what the major research findings are. For a better understanding of these approaches, this study takes note of what devices are being used for the AR experience when immersion-related feelings are one of the behaviours that are considered in their evaluations, and discusses to what extent these feelings in AR games affect the player’s other behaviours.

Purpose Evacuation drills should be more realistic and interactive. Focusing on situational and audio-visual realities and scenario-based interactivity, the authors have developed a game-based evacuation drill (GBED) system that presents augmented reality (AR) materials on tablet computers. The paper's current research purpose is to improve visual reality (AR materials) in our GBED system. Design/methodology/approach The author's approach is to develop a new GBED system that superimposes digital objects [e.g. three-dimensional computer graphics (3DCG) elements] onto real-time vision using a marker-based AR library, a binocular opaque head-mounted display (HMD) and other current easily available technologies. Findings The findings from a trial experiment are that the new GBED system can improve visual reality and is appropriate for disaster education. However, a few problems remain for practical use. Research limitations/implications When using the GBED system, participants (i.e. HMD wearers) can suffer from 3D sickness and have difficulty in moving. These are important safety problems in HMD-based systems. Social implications The combination of AR and HMDs for GBEDs (i.e. integrating virtual and real worlds) will raise questions about its merits (pros and cons). Originality/value The originality of the research is the combination of AR and an HMD to a GBED, which has previously been realized primarily as simulation games in virtual worlds. The authors believe that our research has the potential to expand disaster education.

3D registration plays a pivotal role in augmented reality (AR) system. The existing methods are not suitable to be applied directly in the mobile AR system for the built environment, with the reasons of poor real-time performance and robustness. This paper proposes an improved 3D registration method of mobile AR for built environment, which is based on SURFREAK and KLT. This method increases the building efficiency of algorithm descriptors and maintains the robustness of the algorithms. To implement and evaluate the registration method, a smart phone-based mobile AR system for built environment is developed. The experimental result shows that the improved method is endowed with higher real-time performance and robustness, and the mobile AR 3D registration can realize a favorable performance and efficiency in the complex built environment. The mobile AR system could be used in building recognition and information augmentation for built environment and further to facilitate location-based games, urban heritage tourism, urban planning, and smart city.

The application of digital technology can provide solutions to community problems, especially in the education sector. But there are still many teachers whose teaching abilities have not been supported by these digital technologies. SDN 14 Bonto-Bonto located in Pangkep Regency is the place for the digital learning training process as well as a form of community service carried out by the team of the Informatics Engineering Department of the Faculty of Engineering, Hasanuddin University. This activity is also a venue for socialization and introduction of research results from lecturers and students. Digital learning models can be divided into 2 namely electronic based learning and internet based. Some digital learning methods taught include Augmented Reality, Mind Mapping, and Educational Games. Augmented Reality is a technology that involves overlaying computer graphics in the real world, where three-dimensional virtual worlds can be brought into real-world environments in real-time. Learning to use AR makes students' visual appeal in learning as well as educational games. Mind Mapping is a learning method in the form of visualizing ideas, thoughts or plans to form interconnected diagrams. With this method, students are trained to think structured and easy to solve problems. Mind mapping application itself has been widely available and can be downloaded for free. The training in developing digital learning media that is held is expected to help teachers and students understand the concepts while at the same time foster motivation in learning. This service activity also adds insight to teachers about new learning methods that can make the learning process more enjoyable.

Background Using commercial off-the-shelf video games rather than custom-made computer games could have several advantages for reducing stress and anxiety, including their low cost, advanced graphics, and the possibility to reach millions of individuals worldwide. However, it is important to emphasize that not all commercial video games are equal, and their effects strongly depend on specific characteristics of the games. Objective The aim of this systematic review was to describe the literature on the use of commercial off-the-shelf video games for diminishing stress and anxiety, examining the research outcomes along with critical variables related to computer game characteristics (ie, genre, platform, time of play). Methods A systematic search of the literature was performed following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analysis) guidelines. The search databases were PsycINFO, Web of Science, Medline, IEEExplore, and the Cochrane Library. The search string was: [(“video game*”) OR (“computer game*”)] AND [(“stress”) OR (“anxiety”) OR (“relaxation”)] AND [(“study”) OR (“trial”) OR (“training”)]. Results A total of 28 studies met the inclusion criteria for the publication period 2006-2021. The findings demonstrate the benefit of commercial off-the-shelf video games for reducing stress in children, adults, and older adults. The majority of the retrieved studies recruited young adults, and fewer studies have involved children, middle-aged adults, and older adults. In addition to exergames and casual video games, other genres of commercial off-the-shelf games helped to reduce stress and anxiety. Conclusions Efficacy in reducing stress and anxiety has been demonstrated not only for exergames and casual video games but also for other genres such as action games, action-adventure games, and augmented reality games. Various gaming platforms, including consoles, PCs, smartphones, mobile consoles, and virtual reality systems, have been used with positive results. Finally, even single and short sessions of play had benefits in reducing stress and anxiety. Trial Registration International Platform of Registered Systematic Review and Meta-analysis Protocols INPLASY202130081; https://inplasy.com/?s=INPLASY202130081

Smartphones have limited battery capacity, so efficient power management is required for high-performance applications and to increase usage time. In recent years, efficient power management of smartphones has become very important as the demand for power use of smartphones has grown due to deep learning, games, virtual reality, and augmented reality applications. Existing low-power techniques of smartphones focus only on lowering power consumption without considering actual power consumption based on utilization of the central processing unit (CPU) and graphics processing unit (GPU), which are major components of smartphones. In addition, they do not take into consideration the strict use of resources within the component and what instructions are being processed to operate them. In this paper, we propose a low-power technique that manages power by calculating the actual power consumption of smartphones at execution time and classifying the detailed resource operating states of CPUs and GPUs. The proposed technique was implemented by linking the kernel and native app on a Galaxy S7 smartphone equipped with Android. In experiments with 15 workloads, the proposed technique achieves an energy reduction of 18.11% compared to the low-power technique of the interactive governor built into the Galaxy S7 with a small FPS reduction of 3.12%.

Metaverse is a virtual reality (VR) environment with avatars and software agents using the internet and augmented reality (AR). This one is described as a new iteration of the internet, Metaverse uses VR headsets, blockchain technology and avatars in a new fusion of real and virtual worlds. The role of the programmer in the metaverse is to create and maintain programs that control the behavior of virtual objects and entities within the virtual space. This can include characters controlled by the computer, animations, special effects, and other visual and interactive elements. Programmers in the metaverse may work on a variety of different projects, such as developing new virtual worlds, creating games or other interactive experiences, or building tools and applications for use within the metaverse. They may also be responsible for maintaining and updating existing programs, fixing bugs, and optimizing performance. The main challenge facing programmers in the metaverse is creating a seamless and immersive experience for users. This involves designing and implementing a wide range of features, such as realistic and interactive 3D environments, avatars, physics and interactions, network communication and synchronization, and AI-powered non-player characters. The development of the metaverse requires a wide range of skills and technologies, including expertise in computer graphics, networking, AI, and user experience design.

The publication substantiates the need to find and master the latest trends and technologies in multimedia, dictated by the modern labor market. The author outlines the history of development of the Multimedia design department, provides the information on the basic scientific achievements and participation in development of scientific state budgetary subjects, and notes the international activity of the department. The focus of the research is aimed at highlighting creative experiments and projects that have been and are crucial in the modern search for a place of specialization in the field of multimedia. At the moment, the fundamental area of our creative search is animation in all its varieties and forms. An indicative result of this work is the successful participation of MD students in numerous national and international animation competitions. In general, the course for the training of professional animators is of great importance for the formation of this art form in Ukraine. The natural interest of young people in music finds its professional solution in the creation of numerous videos that visualize musical works. We have started systematic work in the direction of fractal animation in order to “revive” fractal worlds. The MD department conducts theoretical research in motion design, creates various developments of animated typography, infographic videos, etc. The design of computer games in all their varieties is also widely presented. In recent years, the department has begun practical work in the field of projection technology, and has already achieved the first successes in international competitions of architectural 3D mapping. Software is the practical development and research of augmented reality in image recognition technologies for both flat objects (painting, graphics, illustration) and in the urban environment. In the last two years, we have successfully started developing of virtual reality projects.

Virtual models’ production is of high pertinence in research and business fields such as architecture, archeology, or video games, whose requirements might range between expeditious virtual building generation for extensively populating computer-based synthesized environments and hypothesis testing through digital reconstructions. There are some known approaches to achieve the production/reconstruction of virtual models, namely digital settlements and buildings. Manual modeling requires highly-skilled manpower and a considerable amount of time to achieve the desired digital contents, in a process composed by many stages that are typically repeated over time. Both image-based and range scanning approaches are more suitable for digital preservation of well-conserved structures. However, they usually require trained human resources to prepare field operations and manipulate expensive equipment (e.g., 3D scanners) and advanced software tools (e.g., photogrammetric applications). To tackle the issues presented by previous approaches, a class of cost-effective, efficient, and scarce-data-tolerant techniques/methods, known as procedural modeling, has been developed aiming at the semi- or fully-automatic production of virtual environments composed of hollow buildings exclusively represented by outer façades or traversable buildings with interiors, either for expeditious generation or reconstruction. Despite the many achievements of the existing procedural modeling approaches, the production of virtual buildings with both interiors and exteriors composed by non-rectangular shapes (convex or concave n-gons) at the floor-plan level is still seldomly addressed. Therefore, a methodology (and respective system) capable of semi-automatically producing ontology-based traversable buildings composed of arbitrarily-shaped floor-plans has been proposed and continuously developed, and is under analysis in this paper, along with its contributions towards the accomplishment of other virtual reality (VR) and augmented reality (AR) projects/works oriented to digital applications for cultural heritage. Recent roof production-related enhancements resorting to the well-established straight skeleton approach are also addressed, as well as forthcoming challenges. The aim is to consolidate this procedural modeling methodology as a valuable computer graphics work and discuss its future directions.

In recent years, cinematography and other digital content creators have been eagerly turning to Three-Dimensional (3D) imaging technology. The creators of movies, games, and augmented reality applications are aware of this technology’s advantages, possibilities, and new means of expression. The development of electronic and IT technologies enables the achievement of a better and better quality of the recorded 3D image and many possibilities for its correction and modification in post-production. However, preparing a correct 3D image that does not cause perception problems for the viewer is still a complex and demanding task. Therefore, planning and then ensuring the correct parameters and quality of the recorded 3D video is essential. Despite better post-production techniques, fixing errors in a captured image can be difficult, time consuming, and sometimes impossible. The detection of errors typical for stereo vision related to the depth of the image (e.g., depth budget violation, stereoscopic window violation) during the recording allows for their correction already on the film set, e.g., by different scene layouts and/or different camera configurations. The paper presents a prototype of an independent, non-invasive diagnostic system that supports the film crew in the process of calibrating stereoscopic cameras, as well as analysing the 3D depth while working on a film set. The system acquires full HD video streams from professional cameras using Serial Digital Interface (SDI), synchronises them, and estimates and analyses the disparity map. Objective depth analysis using computer tools while recording scenes allows stereographers to immediately spot errors in the 3D image, primarily related to the violation of the viewing comfort zone. The paper also describes an efficient method of analysing a 3D video using Graphics Processing Unit (GPU). The main steps of the proposed solution are uncalibrated rectification and disparity map estimation. The algorithms selected and implemented for the needs of this system do not require knowledge of intrinsic and extrinsic camera parameters. Thus, they can be used in non-cooperative environments, such as a film set, where the camera configuration often changes. Both of them are implemented with the use of a GPU to improve the data processing efficiency. The paper presents the evaluation results of the algorithms’ accuracy, as well as the comparison of the performance of two implementations—with and without the GPU acceleration. The application of the described GPU-based method makes the system efficient and easy to use. The system can process a video stream with full HD resolution at a speed of several frames per second.

Thanks to the joint work of industry companies, new devices are already entering the market and 5G networks and infrastructure are being actively deployed. And very soon real 5G gadgets will go on sale! In this article I want to talk about the new opportunities that the 5G era can give people. And these features, including those we have already talked about, are getting closer to users. Let’s talk today about what the advent of the 5G era means for games and communications. The key to the success of 5G cloud gaming is a good strategy that brings the right gaming games to an affordable market with the right infrastructure and coverage and provides a business case that delivers growth and expected ROI. 5G allows developers to deal with the problem of delays and gives them and users completely new features. We believe that with fifth-generation network technology, the very concept of "lag" in games will soon be forgotten, and the combination of 5G and augmented reality in multiplayer mobile AR-games will help bring mobile gaming to a whole new level. Using only smartphones, you and your friends will be able to play augmented reality paintball to combine the real world with the elements of the virtual. As in today’s paintball battle, when you need to run around the field and shoot balls of paint at your friends, you will catch them in the "sight" of your smartphone and tap the screen to fire digital paintball shells. In addition to reducing latency, 5G technology and artificial intelligence will help to ensure complete immersion in the game with quality graphics and new effects. For example, when one such virtual projectile hits the clothes, the ball will not just bounce off it, as in 4G games, it will make the fabric bend at the point of impact, and then in real time it will fill it with a bright.

The game, which is one of the most basic entertainment elements in life, has become an indispensable part of daily life from the first age people to today's post-modern individuals. Nowadays, most people's need to play games has been transferred to the digital environment thanks to new media channels, and games have been transformed into a phenomenon cultural form. However, the fact that pure visuality is not sufficient for the target audience in game designs has led the designers to seek new applications in the context of visual design. Therefore, with motion graphics and illustrations in game design, designers have handled gamification and advergames, which are also used in many disciplines such as advertising and marketing, with different trends such as artificial intelligence, augmented and virtual reality. Therefore, the aim of this study is to focus on illustrative-based visual approaches or language developed in terms of graphic design by explaining gamification and advergames in the context of design and new media. Based on all this information, a wide literature review was made within the scope of the study and the data selected within the framework of purposeful sampling were analyzed with qualitative and descriptive methods, and the research article was concluded.

Three-dimensional data are generally represented by triangular meshes. The 3D data are used in several fields including remote 3D games, 3D medical application, 3D virtual worlds and 3D augmented reality application. These applications require displaying, printing or exchanging the 3D models through the network to optimize the rendering of the 3D models and 3D applications, which include different treatments, for example, smoothing, compression, re-meshing, simplification, watermarking, etc. However, these processes generate distortions that affect the quality of the rendered 3D data. Thus, subjective or objective metrics are required for assessing the visual quality of the deformed models to evaluate the efficiency of the applied algorithms. In this context, we introduce a new perceptual full-reference metric that compare two 3D meshes based on their 3D content information. The proposed metric integrates the relativity and selectivity properties of the Human visual system (HVS) independent of the mesh type and connectivity (e.g. Triangular, Quadrilateral, Tetrahedron, Hexahedron), which represent a limit in the existing method, in order to capture the perceptual quantity of the distortion by the observer. The results of the proposed approach outperform the existing metrics and have a high correlation with the subjective measures. We use the two correlation coefficients Spearman Rank ([Formula: see text]) and Pearson Rank (Rp) in order to assess the performance of the proposed metric.

In rejoinder to a New York Times’s article claiming, “the value of used pianos, especially uprights, has plummeted … Instead of selling them … , donating them … or just passing them along … , owners are far more likely to discard them” (Walkin), artists Kathleen Irwin (scenography) and Jeff Morton (sound/composition) responded to this ignoble passing with an installation playing with the borders delineating music, theatre, digital technology, and economies of value using two upright red pianos, sound and video projection—and the sensibility of relational aesthetics. The installation was a collaboration between two artists who share a common interest in the performative qualities of public space and how technological augmentation is used in identificatory and embodied art processes as a means of extending the human body and enhancing the material space of person-to-person interaction. The title of the installation, PLAY, referenced the etymology of the word itself and how it has been variously understood over time, across artistic disciplines, and in digital and physical environments. Fundamentally, it explored the relative value of a material object (the piano) and how its social and cultural signification persists, shifts, is diminished or augmented by technology. The installation was mounted at the Dunlop Art Gallery, in the Regina Public Library (Saskatchewan, Canada, 14 June - 25 August 2013) and, as such, it illustrated the Library’s mandate to support all forms of literacy through community accessibility and forms of public outreach, social arrangements, and encounters. Indirectly, (as this was not the initial focus), it also exemplified the artists’s gentle probing of the ways, means, claims, and values when layering information and enhancing our visual experience as we interact with (literally, walk through) our physical landscapes and environments—“to see the world for what it is,” as Matt Turbow says “and to see the elements within” (Chapeau). The installation reflected on, among other things, the piano as a still potent cultural signifier, the persistent ability of our imagination to make meaning and codify experience even without digital overlay, and the library as an archive and disseminator of public knowledge. The artists questioned whether old technologies such as the piano will lose their hold on us entirely as technological augmentation develops the means to enhance or colonize the natural world, through graphics, sounds, haptic feedback, smell and, eventually, commodified experiences. This paper intends to reflect on our work and initiate a friendly (playful) interdisciplinary discussion about material objects in the age of physical and digital interactivity, and the terms of augmentation as we chose to understand it through our installation. In response to the call proposed by this journal on the subject of augmentation, we considered: 1. How audio/visual apparatuses in the gallery space augmented the piano’s expressivity; 2. How the piano augmented the social function of its physical situation; 3. How the technology augmented random and fragmentary musical phrases, creating a prolonged musical composition; 4. How each spectator augmented the art through his/her subjective engagement: how there is always meaning generated in excess of the artists’s intention. Image 1: Piano installed outside Dunlop Gallery/ Regina Public Library (photo credit: Jeff Morton) To begin, a brief description of the site of the installation is in order. The first of the red pianos was installed outside the main doors of the Central Library, located in the city’s downtown. The library’s entrance is framed within a two-story glass atrium and the red piano repeated the architecture’s function to open the space by breaking down perceived barriers, and beckoning the passersby inside. Reflecting Irwin’s community-oriented, site-specific practice, this was the relational catalyst of the work—the piano made available for anyone to play and enjoy, day or night, an invitation to respond to an object inserted into the shared space of the sidewalk: to explore, as Nicolas Bourriaud suggests, “the art as a state of encounter” (16). It was the centerpiece of the exhibition's outreach, which included the exhibition’s vernissage featuring new music and performance artists in concert, a costume and prop workshop for a late night public choir procession, and a series of artist talks. This was, arguably, a defining characteristic of the work, underscoring how the work of art, in this case the piano itself, its abjection illustrated by the perfunctory means typically used to dispose of them, is augmented or gains value through its social construction, over-and-above any that is originally ascribed to it. As Bourriaud writes, any kind of production takes on a social form which no longer has anything to do with its original usefulness. It acquires exchange value that partly covers and shrouds its primary “nature”. The fact is that a work of art has no a priori useful function—not that it is socially useless, but because it is available and flexible, and has an “infinite tendency”. (42) In the Dunlop’s press release, curator Blair Fornwald also confers a supplemental value ascribed to the reframed material object. She describes how the public space in front of the library, as a place of social interaction and cultural identification—of “being seen”—is augmented by the red piano: its presence in an unfamiliar setting underscores the multitude of creative and performative possibilities inherent within it, possibilities that may extend far beyond playing a simple melody. By extension, its presence asserts that the every day is a social, cultural, and physical environment rich with potentiality and promise. (Fornwald) Juxtaposed with the first red piano, the second was dramatically staged within the Dunlop gallery. The room, painted black, formally replicated the framing and focusing conventions of the theatre: its intention to propose other ways of “being seen” and to suggest the blurring of lines between “on stage and off,” and by extension, “on line and off.” A camera embedded in the front of the piano and a large projection screen in the space provided a celebrity moment for anyone approaching the instrument and implied, arguably, the ubiquitous surveillance associated with public space. Indeed, a plausible way of reading the red piano in the darkened gallery was as a provocation to think about how the digital and physical are increasingly enmeshed in our daily lives (Jurgenson). Lit by a chandelier and staged on a circular red carpet, this piano was also available to be played. Unlike the one outside of the building, it was augmented by speakers, a microphone, and a webcam. Through a custom-built digital system (using MaxMSP software), it recorded and played back the sound and image of everyone who sat down to perform, then repeated and superimposed these over similar previously captured material. Enhanced by the unusual stark acoustics of the gallery, the sound filled the reverberant space. Affixed to the gallery’s back wall was the projection screen made up of sheet music (Bach, Debussy and Mozart) taken from the Irwin family’s piano bench, a veritable time capsule from the 1950s. Image 2: Piano installed inside Dunlop Gallery (photo credit: Jeff Morton) In addition to the centrally placed piano, a miniature red piano was situated near the gallery entrance. It and a single red chair placed near the screen, repeated the vivid colour and drew the eye into and around the space underscoring its theatrical quality. The toy piano functioned as a lighthearted invitation, as well as a serious citation of other artists—Eikoh Sudoh, Margaret Leng Tan, John Cage, and Charles M. Schulz’s “Schroeder”—who have employed the miniature instrument to great advantage. It was intended as an illustration of the infinite resonances that material objects may provide and the diverse ways they may signify contingent on the viewer. Considered in a historical context, in the golden age of the upright and at the turn of the twentieth-century, piano lessons signified for many, the formation of a modern citizen schooled in European culture and values. Owning one of these intricately engineered and often beautiful machines, as one in five households did, reflected the social aspirations of its owners and marked their upward economic mobility (Canadian Encyclopedia). One hundred years later, pianos are often relegated to the basement or dump. Irretrievably out of tune, their currency as musical instruments largely devalued. Nonetheless, their cultural and social value persists, no longer the pervasive marker of status, but through the ways they are mediated by artists who prepare, deconstruct, and leave them to deteriorate in beautiful ways. They seem to retain their hold on us through the natural impulse to engage them kinetically, ergonomically, and metaphorically. Built to be an extension of the human hand, body, and imagination, they are a sublime human-scale augmentation of a precise musical system of notation, and a mechanism evolved over centuries through physical augmentations meant to increase the expressivity of both instrument and player. In PLAY, the use of the pianos referenced both their traditional role in public life, and our current relationship with forms of digital media that have replaced these instruments as our primary means of being linked, informed, and entertained—an affirmation of the positive attributes of technology and a reminder of what we may have lost. Indeed, while this was not necessarily clear from the written responses in the Gallery’s guest book (Gorgeous!: Neat!; Too, too cool!; etc.), we surmised that memory might have played a key role in the experience of the installation, set in motion by the precise arrangement of the few material objects – red piano, the piano bench, red chair, and toy piano, each object designed to fit the shape of the body and hold the memory of physical contact. These were designed to trigger a chain of recollections, each chasing the next; each actively participating in what follows. In the Gallery’s annual exhibition catalogue, Ellen Moffat suggests that the relationship the piano builds with the player is important: “the piano plays and is played by the performer. Performing the piano assigns a posture for the performer in relation to the keyboard physically and figuratively” (Moffat 80). Technically, the piano is the sum of many parts, understandable finally as a discrete mechanical system, but unbounded in imagination and limited only by our capacity to play it. Functionally, it acts as an affective repository of memory and feeling, a tool to control the variables of physical and expressive interaction. In PLAY, the digital system in the gallery piano captured, delayed and displayed audio and video clips according to a rubric of cause and effect. Controlled by computer software designed by Morton, the installation captured musical phrases played randomly by individuals and augmented these notes by playing them back at variable speeds and superimposing one over another—musical phrases iterated and reiterated. The effect was fugue-like—an indeterminate composition with a determinant structure, achieved by intertwining physical and digital systems with musical content supplied by participants. The camera hidden in the front of the piano recorded individuals as they sat at the instrument and, immediately, they saw themselves projected in extreme close up onto the screen behind. As the individual struck a note, their image faded and the screen was filled again with the image of a previous participant abstracted and in slow motion. The effect, we suggest, was dreamlike—an echo or a fleeting fragment of something barely remembered. Like the infinite variations the piano permits, the software was also capable of expressing immense variety—each sound and image adding to an expanding archive in an ever-changing improvised composition developed through iterative call and response. Drawing on elements of relational aesthetics, scenographic representation, and digital technology, in PLAY we attempted to cross disciplines in ways that distinguished it from the other piano projects seen over the past several years. Indeed, the image of the upright piano has resonated in the zeitgeist of the international art scene with colourful uprights placed in public places in urban centers across Europe and North America. Wherever they are, individuals engage enthusiastically with them and they, in turn, become the centre of attention: this is part of their appeal. The pianos seem to evoke a utopian sense of community, however temporary, providing opportunities to rediscover old neighbours and make new friends. In PLAY, we posed two different social and aesthetic encounters—one analogue, real, “off-line” and one digital, theatrical, and “on-line,” illustrating less a false binary between two possible realities that ascribes more value to one than the other, than a world where the digital and the physical comingle. Working within a public library, this was a germane train of thought considering how these institutes struggle to stay relevant in the age of Google search and the promise of technological augmentation. The piano also represents a dichotomy: both a failure to represent and an excess of meaning. For decades replete with social signification, they have now become an encumbrance, fit only for the bone yard. As these monumental relics come to the end of their mechanical life, there is more money made in their disposal than in musical production, and more value in their recycled metals, solid wooden bodies, and ivory keys then in their tone and function. The industry that supported their commodification collapsed years ago, as has the market for their sale and the popular music publishing industry that accompanied it. Of course, pianos will be with us for a long time in one form or another, but their history, as a culturally potent object, has diverged. The assumption could easily follow that they have been rendered useless as an aesthetic, generative, and social object. What this installation offered was the possibility of an alternative ending to the story of this erstwhile entertainment console even as we seek our amusement by other means and through other devices. Not surprisingly, the title of the installation suggests that the consideration of “play,” as social and recuperative engagement, is significant. In his seminal work, Homo Ludens, Johan Huizinga discusses the importance of play, suggesting that it is primary to and a necessary condition of the making of culture. He writes, “In play there is something in play, which transcends the immediate needs of life and imparts meaning to the action” (Huizinga 97). According to games theorist Mary Flanagan, playing may serve as a way of creating something beautiful, offering frameworks for new ways of thinking, exploring divergent logic, or for imaging what is possible. She writes, “Games, both digital and analog, offer a space to explore creativity, agency, representation and emergent behaviour” (Flanagan 2010). In reaching out to Regina’s downtown community, the Dunlop Art Gallery dispersed some of the playfulness of PLAY in planned and accidental ways, as the outdoor piano became a daily destination for individuals who live rough or in the city’s hostels, some of whom who have enviable musical skills and considerable stage presence. One man came daily with sheet music in hand to practice on the indoor piano—ignoring the inevitable echo and repeat that the software triggered. Another young woman appeared regularly to perform at the outdoor piano, her umbrella raised against sun and rain, wedged under her arm to keep both hands free. Children invariably drew parents to it as they entered or exited the library—for some it may have been the first time they had touched such an instrument. Overall, in press, blogs, and the visitors’s book, responses to the pianos were enthusiastic and positive. One blogger wrote in response to an online publication, Art, Music, News (Beatty), chapeau June 13, 2013 at 11:51am this is most definitely up and running, and it would be interesting to see/hear all that will go on with that red piano. my two-and-a-half year old daughter and i jammed a bit yesterday morning, while a stranger watched and listened, then insisted that i play the same mostly crappy c-blues again while he sang! so i did, and he did, and my daughter and i learned a bit about what he feels about his dog via his singing. it was the highlight of the day for us—I mean really, jamming outside on a very red upright piano with strangers—good times! (Simpson) As evidence of public approbation, for the better part of the summer it stood unprotected on the sidewalk in front of the library encountering only one minor incident of defacement—a rather fragile tag in white spray paint, someone’s name in proper cursive writing. Once repaired and retuned, it became a dynamic focus for the annual Folk Festival that takes over the area for a week in August. In these ways, PLAY fulfilled the Library’s aim of encouraging literacy and reinforcing a sense of community—a social augmentation, in a manner of speaking. As Moffat writes, it encourages the social dimension of participation through community-engagement and dialogic practices. It blurs distinctions between spectator and participant, professional and amateur. It generates relationships between people or social actions. (Moffat 76) Finally, PLAY toyed with the overtones of the word itself—as verb, noun, and adjective—signifier, and metaphor. The title illustrated its obvious current potential and evoked the piano’s past, referencing the glittering world of the stage. While many may have more memories of seeing pianos in disrepair than in the concert hall, its iconic stage setting is never far from the imagination, although this too changes as people from other cultures and backgrounds recognize little cultural capital in such activity. In current vernacular, the word “play” also implies the re-imagination of ourselves in the digital overlays of the future. So we ask, what will be the fate of the piano and its meme in the 22nd century? Will the augmentation of reality enhance our experience of the world in inverse proportion to a loss of social interaction? Conclusion In her essay, Moffat notes that as digital technology replaces the analog piano, a surplus of second-hand uprights has become available. Citing artists Luke Jerram, Monica Yunus, and Camille Zamora (among others), she argues that the use of them as public art coincides with their disappearance, suggesting a farewell or memorial to a collective cultural icon (Moffat 76). What is there in this piece of furniture that speaks to us in art practice? The answer, it would seem, is potential. In a curatorial interview, Irwin suggested the possibility that beyond the artist’s initial meaning, there is always something more—an augmentation. The pleasure of discovering this supplement is part of the pleasure of the subjective experience of the spectator. Similarly, the aleatoric in music composition, refers to the pursuit of chance as a formal determinant and its openness to individual interpretation at the moment of reception. For Morton, the randomness of memory and affect are key components in composition. They cannot be predicted, controlled or quantified; nor can they be denied. There is no correct interpretation or response to music or, indeed, to relational art practice. Moffat concludes, as a multi-faceted media installation, PLAY proposed “a suite, chorus or a polyphony of things” (Moffat 76). Depending on your point of reference, the installation provided a dynamic venue for considering our relationships with material objects, with each other and with new technologies asking how they may or may not augment our reality in ways that supplement real-time, person-to-person interaction. References Beatty, Gregory. “Exciting Goings-On at Central Library.” Prairie Dog Blog 11 June 2013. Bourriaud, Nicolas. Relational Aesthetics. Trans. Simon Pleasance and Fronza Woods. Paris: Les Presses du Réel, 1998. Canadian Encyclopedia. “Piano Building.” ‹http://www.thecanadianencyclopedia.com/en/article/piano-building-emc/›. Chapeau [David Simpson]. “One Response to ‘Exciting Goings-On at Central Library.’” Prairie Dog Blog 13 June 2013. Fornwald, Blair. PLAY. Regina, Saskatchewan: Dunlop Art Gallery. 2013. Flanagan, Mary. “Creating Critical Play.” In Ruth Catlow, Marc Garret and Corrado Morgana, eds., Artists Rethinking Games. Liverpool: Liverpool University Press, 2010. 49-53. Huizinga, Johan. Homo Ludens. Boston: Beacon Press, 1995. Jerram, Luke. Play Me, I’m Yours. Site-Specific Piano Installation. Multiple Venues. 2008-2013. Jurgenson, Nathan. “Digital Dualism versus Augmented Reality.” Cybergology: The Society Pages 24 Feb. 2011. 1 Dec. 2013 ‹http://thesocietypages.org/cyborgology/2011/02/24/digital-dualism-versus-augmented-reality/›. Moffat, Ellen. “Stages and Players” in DAG 2 (2013). Regina: Dunlop Art Gallery, 2013. 75-87. Walkin, Daniel J. “For More Pianos, Last Note Is Thud in the Dump.” New York Times 29 June 2012. Yunus, Monica, and Camille Zamora. Sing for Hope Pianos. Site-Specific Piano Installation and Performance. New York City. 2013.

In the age of digital transformations of images, communications, and storytelling, Marshall McLuhan's insight that "the medium is the message" can be augmented with the corollary that the media is the mix. Digital forms of narrative are not only characterized by their mixed, hybrid forms and content, but their recombinations 1 draw the spectator into the mix in unforeseen ways. By mixing varying degrees of non-linearity and interactivity in what are ultimately animations, digital narratives create new kinds of digital spectatorship. The examples I'll explore here are three films, Conceiving Ada, Gamer, and Time Code. In different yet interconnected ways, each privileges the mix of media over content, or rather, foregrounds the mix as content. These digital narratives divert the reader/spectator/ participant from the traditional ways of making meaning--or at least sense--of narrative. One way to illuminate the examples is to explore how they mix linearity and interactivity. In teaching The Theory and Practice of Digital Narrative, my students and I developed a model for analyzing how different works create new modes of storytelling, and fresh relations of looking at and within the frame.2 Extending some of the terms that Janet Murray develops in Hamlet on the Holodeck: The Future of Narrative in Cyberspace, the diagram posits an x axis of linearity and a y axis of agency.3 Click here for an animated version of this graph. The linearity axis spans narratives from most linear (one plot line that progresses chronologically through time and space to one outcome through a series of cause and effect occurrences represented from a singular point of view) to non-linear (narratives that could be circular, rhizome-shaped, achronological, synchronic, multiple plotlines, multiple points of view, multidimensional).4 The agency axis spans media that call for very little active participation from the reader/spectator/listener to media that demands a high level of interaction. Of course, all "reading" is in some way active, both in the physical acts of seeing and turning pages/clicking a mouse 5 and in comprehending, imagining, remembering, and making meaning. Nevertheless, there is a distinction between page-turning and making choices in a hypermedia work, and Murray uses the term agency to suggest an "active creation of belief" (as opposed to Coleridge's "willing suspension of belief"). Digital environments require that belief be created and reinforced; as William Gibson posited in Neuromancer, the imagined place of cyberspace is a "consensual hallucination," a social agreement to act as if the places in cyberspace exist. The willing creation of belief is a social agreement that relies on a like-minded community. The more self-reflexive and unconventional the narrative, the more it calls for the will to believe. These examples of digital cinema "interpellate," or hail, their spectators as willing agents in the common project of the creation of belief. The subjectivity that these films seek to create for their viewers is one of being an active, technologically-savvy spectator. Instead of encouraging participation in, to use Guy Debord's phrase, the society of the spectacle, these digital narratives perform a Brechtian function in a distinctly technological manner that derives from the mix of media that is digital cinema. The term "digital cinema" has acquired many meanings, ranging from movies shot on digital video in a manner we associate with film (i.e.: single camera) to digital exhibition of media in digitally-equipped movie theaters and streaming video on the web. Lev Manovich defines digital cinema as "a particular case of animation which uses live action footage as one of its many elements."6 Manovich's historical argument suggests commonalities between the earliest moving images and current developments in digital media; animation was marginalized in the development of cinema, and although some of its techniques were adopted by the avant garde, only now animation returns at the very center of digital cinema.7 Lynn Hershman Leeson's film Conceiving Ada explores digital media in both form and content. In the film Emmy, a contemporary computer software engineer, uses technology to make contact with Ada Lovelace, who invented the first computer language in Victorian England. The narrative moves between Emmy in the present day and Ada in the past as Emmy figures out how to send a software agent into the past to retrieve information. Although the plot doesn't really make sense scientifically, it resonates emotionally; despite the 150 years separating their lives, Emmy and Ada face some of the same issues as women working with technology. Instead of building sets, Leeson developed a technique of blending live action footage shot against a blue screen with digitized photographs of Victorian inns. As she explains in the technical notes of the DVD of the film (and also on the Conceiving Ada website): I felt it important to use the technology Ada pioneered. Virtual sets and digital sound . . . provided environments in which she moves freely through time, becomes liberated and, ultimately, moves into visibility. The actors and filmmaker collaborated in what amounts to a consensual hallucination: On the set, these images were maneuvered through several computers where mattes were added and images were put into perspective or enlarged. They were then laid onto digital videotape while the actors were performing. Actors could reference their location through a monitor that showed them their "virtual" environment. . . . The immediacy of shooting live action while simultaneously manipulating digitized backgrounds in real time was, remarkably, exhilarating. By mixing past and present, fact and fiction, personal and professional, digital and analog, live action and animation, Conceiving Ada tells its powerful story in both form and content. Although it is not immediately obvious that the sets are virtual rather than actual, much of the story takes place in front of and inside Emmy's computer equipment. There are many shots of Emmy gazing into the computer, trying to make her programs work, and when she does make contact with Ada, she can see her memories and talk with her on the computer screen. The film frame often encompasses the computer screen, so we too see the graphic interfaces Emmy designs and animates. By mixing some of the techniques of video art with film style and the malleability of the digital image, Leeson extends her trailblazing career in many directions at once in a work that is a mix about mix. A representation of intense digital interactivity, Conceiving Ada's heroines, creators, and spectators use technology, and specifically digital imaging technology, to create agency. Like Conceiving Ada, Time Code is also an example of digital cinema that calls attention to the digital techniques used in its production, and enlists its spectators as accomplices in creating the narrative. Time Code is itself the product of improvisation, and uses digital technology to capture "real time." Director Mike Figgis divides the screen into four frames; each quadrant contains a continuous 90-minute take of an unscripted, improvised performance, all shot simultaneously with four Sony Dvcam DSR-130s; the film was blocked out on music paper. Figgis describes his movie as a "black comedy about a 90-minute slice of life in Hollywood," and it takes filmmaking as one of its subjects along with jealousy, infidelity, and a fin-de-siecle philosophical and artistic exhaustion. The audio mix of the theatrical release of the film shifts its emphasis between the quadrants, thus directing the spectator's attention to a certain quadrant. Choosing to focus on a quadrant is a kind of spectator-editing. Looking at the entire frame means seeing a new kind of animation, created by multiple screens, encompassing multiple points of view. (See Time Code clip here.) The film folds in on itself self-reflexively. The plot centers around the intersecting lives of four characters involved personally and/or professionally with Red Mullet, Inc., a movie studio (which is the real-life name of Figgis's production company; see www.red-mullet.com), and becomes increasingly Brechtian as the movie builds to the climactic scene in which a hot young independent filmmaker pitches a movie that, like the one we are watching, splits the screen and follows the interactions between four characters. In what could be taken as a manifesto for digital cinema that counters the "chastity" of DOGME 95 with the passionate embrace of technology, the filmmaker announces, "Montage has created a fake reality. . . . Technology has arrived, digital video has arrived, and is demanding new expressions, new sensations. . . . It's time to say again: Art, Technology: a new union." She shows the studio executives storyboards of how four cameras will follow four characters, who are really four aspects of the same character at different points in their lives. But the filmmaker's passionate and theoretical speech is undercut by the context of black comedy that infuses the film: artistic praxis clashes with business practice, and all the plotlines peak as Stellan Skarsgard's Alex bursts into laughter and exclaims, "This is the most pretentious crap I've ever heard. . . . Do you think anybody around this table has a clue about what you're talking about?" Figgis assumes his audience does, that Bauhaus, Soviet montage, and Guy Debord are not as foreign to his spectators as they are to the executives; the filmmakers both within and outside the frames show their theoretical orientation. Time Code is the first major work of digital cinema. It creates a new kind of animation based on subjectivity and point of view, and calls for an active creation of belief from the spectator-editor who takes in and ultimately creates the narrative of the film. The DVD extends the spectator's agency even further. Among its special features is a documentary about how the film was shot 15 times, all four cameras operating simultaneously around the actors' improvisation. Figgis chose the fifteenth version for the theatrical release, but also includes the first version on the DVD. Because the making of the film, and how it uses digital technology, is so central to the spectator's experience, the director's commentary and interviews with the cast reinforce the spectator as an active creator of belief and meaning in the film. Moreover, by including a special audio mixing feature, the DVD gives the medium a new level of interactivity. Using the remote control of a DVD player, the spectator/participant can switch between the audio of the different quadrants. Because the audio is a major aspect of directing the spectator's attention (in addition to visual elements such as movement and stasis), being able to make choices in the audio mix is, to use the music metaphor that the film is based on conceptually, to become the conductor of the film.8 The medium is the mix. Like Time Code and Conceiving Ada, the new French film Gamer is also a particularly digital mix. Gamer moves between live action and digital environments as its main character Tony gets the idea for, designs, and then is swindled out of a computer game. The first time the film environment switches to the game environment is when the hero is in a car chase and his car morphs into a game graphic of a car. The shift between a reality created by conventional film style and the unconventional use of game graphics style reveals the main character's subjectivity, for his reality (and other characters' as well) are constructed through their interaction in the playing and making of games. Gamer is aesthetically and viscerally ambitious in the range of live action and computer graphic interfaces it moves between. The adrenalized state of game-playing mixes with a fictionalized account of game design. Gamer succeeds in creating an immersive text about two differently immersive mediums, film and computer games. When the film depicts how live action can be made digital, and both shows and denies the indexicality of the digital image, it explores the nature of digital cinema in a way that is complementary to the projects of Conceiving Ada and Time Code. In addition to fostering new relations of looking, these digital narratives make forays into nonlinearity. Conceiving Ada and Gamer both have discursive plots that revise the conventions of the linear plot, moving between nested narrative frames in time, space, and subjectivity. Time Code is in one way relentlessly linear, but its synchronic depiction of multiple physical and emotional points of view ruptures the cinematic conventions of time and space constructed by the dominant style of continuity editing. Taken separately, these three films hover around the center point of my diagram, raising the issues of agency and linearity that will continue to be at the center of digital narrative. Taken together the films offer a model of digital transformation that points the way to what is bound to be a medium that increasingly involves its audience in thinking about and then participating in increasingly immersive, nonlinear, and interactive experiences. Notes 1. In the “renew” issue of M/C, David Marshall suggests that one of the areas Cultural Studies can look to is how the culture industries that produce “recombinant culture,” make efforts “to incorporate new technologies into different forms in order to reconstitute audiences in ways that in their distinctiveness produce value that is exchangeable as capital.” This essay is part of a larger work that attempts to open up some of the avenues suggested by Marshall. P. David Marshall. "Renewing Cultural Studies." M/C: A Journal of Media and Culture 3.6 (2000). [22 February 2001] . 2. This diagram is a work in progress. My students and I have had much discussion about what to term the bottom of the agency axis. “Passive” seems too simple, yet other terms we’ve come up with such as structured, controlled, limited, voyeuristic, or (my favorite) enslaved don’t seem to work. Special thanks to my students Elliott Davis, Tom Mannino, and John O’Connell, for such discussions. 3. Janet H. Murray, Hamlet on the Holodeck: The Future of Narrative in Cyberspace, MIT Press, 1999, esp. 128. 4. For an interesting discussion and diagramming of story shapes, see Katherine Phelps, “Story Shapes for Digital Media.” [3/7/01] < http://www.glasswings.com.au/modern/shapes/>. Steven Johnson’s assertion that the hyperlink is the “first significant new form of punctuation to emerge in centuries” is an intriguing one for thinking about the connections possible in hypermedia. (Interface Culture: How New Technology Transforms the Way We Create and Communicate (San Francisco: HarperEdge, 1997), 110-111). 6. Lev Manovich, “What Is Digital Cinema?” http://jupiter.ucsd.edu/~manovich/text/digital-cinema.html 7. In a footnote, Manovich makes an interesting point on avant-garde strategies such as collage, painting on film, combining print with animation and live action footage, and combining many images in a single frame: “what used to be exceptions for traditional cinema became the normal, intended techniques of digital filmmaking, embedded in technology design itself.” Innovative forays into the mix of digital media like my examples illuminate not only the emergence of exceptional techniques but also of innovative narrative and spectatorial strategies. 8. Figgis is using the quadrant method once again in a new film, Hotel, which recently finished production in Florence. During the filming, there was a brilliant site that every day had a new page with the four-quadrant split. In addition to quicktimes of footage of the shoot and the actors in the hotel where the film is set, there were some of the cleverest animations I have ever seen on the web. Unfortunately the site shut down after the shoot finished, but it will be up again in May, most likely at www.filmfour.com/hotel, but check www.red-mullet.com as well.