Museum Virtual Worlds

The Exploratorium’s team of Second Life staff and volunteers put on another mixed-reality webcast viewing event in SL on June 5th, 2012, extending the museum and online event of the Exploratorium’s live webcast of the 2012 Transit of Venus into the virtual world. Avatar-scientist Patio Plasma hosted the in-world event which featured the 6.5 hr long live webcast of telescopic images of the sun and the rare planetary transit. Volunteers in SL helped avatars see the stream. For this event, we tried a different method of putting the stream into SL–using the HTML on a prim feature of the SL viewer. This method proved to be workable, though not as easy to use for SL residents as the traditional approach of replacing a prim texture with a Quicktime stream. With each event, we learn more about the best ways to use the multimedia features of SL and how to engage avatars with live webcasts.

Yesterday I went to a presentation by Australian artist, Lynette Wallworth, an artist-in-residence at the Exploratorium, who showed us her recent works in which she focuses on a rare astronomical event, the Transit of Venus, as a metaphor for awareness about the rising of sea temperatures in bio-diverse ocean environments–an indicator of global climate change.

A transit of Venus occurs when the planet Venus is aligned between the earth and the Sun.  These alignments occur in pairs that are eight years apart.  The time between transit pairs is over a hundred years.  In our lifetime, the transit of Venus occurred in 2004 and the second transit in this cycle will occur in June of 2012.

In Rekindling Venus, a multi-media work commissioned by the Adelaide Film festival, Lynette examines the history of scientists and explorers around the globe exchanging observations of the transit of Venus in 1761 and in 1769.  She connects the global exchange of information and ideas about the transit in the past to the current exchange of information among scientists and the general public about the issues of climate change facing us now.  The first part of the work, In Plain Sight, is an augmented reality work that uses the mobile AR platform Junaio, to allow people to peer into a 3D virtual world with endangered corals.  She’s collaborating with scientists who study corals in ocean areas where coral bleaching occurs due to elevated sea temperatures and examines coral species that fluoresce under different wavelengths of light.   She discussed how scientists are observing that some species of corals that were not known to fluoresce have been shown to now be exhibiting this phenomena, possibly as an adaptation mechanism to changing climate conditions.  The Junaio AR channel for the work, recognizes posters of endangered coal species and uses this image-based 2D marker approach to show 3D models of corals in their fluorescent state. An element of note in this work are the hotspots on the 3D models which, when activated by touch, display a window with recent elevated sea temperature alerts from NOAA, gathered from locations across the planet. This points to great potential for linking mobile AR experiences to live data sources, and helps connect the viewer to something happening right now.

The Rekindling Venus website counts down to the 2012 transit of Venus, which, along with the installation and mobile exploration of the AR elements, offers a call to action for viewers by engaging them in awareness, investigation, and discussion of critical climate change issues as we witness the upcoming astronomical alignment and understand the global context for shared wonder in natural phenomena and how that can inspire us to combat the climate change issues we face.

Over 700 museum professionals convening last week in Philadelphia for the Museums and the Web 2011 conference saw applications and content expanding from the desktops of museum visitors onto networked exhibits inside of museums and onto a growing collection of mobile devices. The annual gathering, which brings together museum exhibit developers, producers, user-experience designers, artists, curators, software developers, digital media directors, and vendors of content creation tools and services, showcases the latest digital media work in the international museums community. It provides a great opportunity for people to share work, ideas, and strategies for using digital media in it’s many evolving forms to engage visitors both in the museum through networked exhibits and online through websites and mobile applications.

As part of a conference session on augmented reality, I presented a paper which discusses the initial investigations of AR that I’ve been doing here at the Exploratorium. This includes a science inquiry activity about weather in the San Francisco bay which will be part of our Science in the City video program series, and the playful Step Into a Virtual View art installations at February’s After Dark: Get Surreal event. The paper presents details about how these exhibit prototypes were developed, considerations for preparing 3D content for use in mobile AR applications, interface design challenges, and some initial observations of how museum visitors interact with mobile AR exhibits. Read the full paper, “Mixing Realities to Connect People, Places, and Exhibits Using Mobile Augmented-Reality Applications” here.

Planning an Exploratorium AR exhibit prototype- Golden Gate Bridge Fog Altimeter

Also presenting about their work with AR in this session were two museums from the Netherlands. Margriet Schavemaker, from the Stedelijk Museum in Amsterdam, discussed that museum’s work with mobile AR using the Layar platform. Their ARTours project, currently one year into a two year project, investigates how their museum visitors interact with AR art and architecture points of interest and exhibits in Amsterdam. Read the paper, “Augmented Reality and the Museum Experience”, co-written with her colleagues Hein Wils from the Stedelijk and with Paul Stork and Ebelien Pondaag from Amsterdam-based design studio Fabrique.

Cutting-edge work was also shown by Ingeborg Veldman and Tanja van der Woude from Science LinX, the science center of the Faculty of Mathematics and Natural Sciences of the University of Groningen in the Netherlands. Science LinX is focused on engaging teenagers in STEM disciplines and they experiment with exhibit development methods and tools that can be used to communicate hard to explain phenomena. I was captivated by Ingeborg’s presentation about their project, MIGHT-y, an exhibit and game which uses 2D markers on the faces of cubes to let visitors explore concepts about scale from the film by Charles and Ray Eames, Powers of Ten. It’s a mixed-reality exhibit in which visitors don AR glasses to see “into” the cubes and manipulate 3D animated objects. Use of AR glasses seem new for museum exhibits and still in an early stage of application. Also, it’s expensive. The glasses used in MIGHT-y, Wrap920 video eyewear from Vuzix, are a consumer product geared toward gamers that can be adapted for use in exhibits to provide an immersive virtual reality experience. I tried a portable version of MIGHT-y with the eyeware, and although it was a bit jarring in terms of head tracking lag, it does provide a compelling augmented display. It’s encouraging to see experimentation with different forms of AR in museums. Read their paper, “Science LinX: the neXt level in augmenting science center Xperiences” which was co-written with Bart van de Laar, also of the University of Groningen

MW2011 was a great conference and it was a special honor for the Exploratorium’s website to be recognized with the Best Long-Lived Site Award by the Museums and the Web community!

Visitors to the Exploratorium’s monthly evening program series for adults, After Dark, experienced bizarre interactions with their sense of reality at After Dark: Get Surreal on Feb 3, 2010–an event infused with surrealist themes in art, music, and science.

Takuji Narumi, a Ph.D. candidate in the Graduate School of Engineering at the University of Tokyo, and Takashi Kajinami, a master course student in the Graduate School of Information Science and Technology at the University of Tokyo tantalized visitors with their interactive experience, “Meta Cookie.” The installation allows participants to don a head-mounted display while holding a cookie that has a 2D marker image burned onto it. A webcam attached to the headgear detects the marker on the cookie while the AR magicians change settings which cause the image of the cookie the person sees through the display to change to a different flavored cookie. Before your eyes, the cookie morphs visually from a butter cookie to a chocolate cookie to a strawberry cookie to a maple cookie to a lemon cookie. Did I mention that tubes attached to the headgear aimed at the adventurous person’s nose deliver scents of the different cookie flavors as the display changes? This remarkable exhibit really pulls at your sense of reality as you nibble on the cookie when it appears to be one flavor and then again as the image and smells change, sensing the flavor as entirely different!

Meta Cookie at the Exploratorium’s After Dark: Get Surreal event on Feb 3, 2010

A few months ago, I was introduced to Helen Papagiannis, an artist, designer, and researcher working with the emerging technology Augmented Reality (AR). I was captivated by the way her playful AR exhibits and installations drew people in through touch, video, and sound. She’d recently exhibited her work at the Ontario Science Center and was exploring ways to engage museum audiences with their sense of discovery and wonder. I caught up with her recently and asked her some questions about the ideas behind her work.

The Amazing Cinemagician: New Media Meets Victorian Wonder” exhibition by Helen Papagiannis at the Ontario Science Center, May-Sept., 2010, Toronto, Canada. Photos: Pippin Lee

When did you start experimenting with augmented reality?

I began experimenting with AR in September 2005. When I saw AR for the first time, I was so entranced I think I entered a permanent state of wonder with the technology. And it was all very simple: a bare bone 3D virtual cube seemingly appearing in my physical space. It was completely astonishing! I went into mad scientist mode from there tinkering, prototyping, and dreaming of the creative possibilities for AR. Five and a half years later, and I’m still riveted.

What are some of the challenges that you’re exploring in your AR work?

When I began working with AR, the challenges were largely around the technical constraints. It has been important for me to work with what is at hand, right now, not tomorrow, or 6 months from now. I always ask, ‘How can we realize this now and make it compelling within the parameters?’ My process has entailed allowing the constraints to guide the work, then finding ways to push beyond those boundaries to create something new.

I strongly believe AR is emerging as a new medium and it will come to play a large role in entertainment and information sharing. The challenge at hand is to continue to investigate how best to apply the medium and really elevate it creatively, and to do this as a community of artists, engineers and industry. We need to identify what is truly unique about this new form, and how we can best leverage these characteristics to tell new stories and create engaging experiences that are unlike anything we’ve ever seen before.

How can museum audiences experience AR?

We’re beginning to see more AR applications in museums, which is very exciting. AR is becoming more accessible and affordable, including the use of personal devices that museum visitors may already have at hand, such as smart phones. AR can be used to provide additional information about objects in a museum’s collection and to enable experiential learning through discovery and play. Wonderment, as discussed in my TEDx talk, is an important part of my work in AR. For me, AR fosters a great sense of wonder as a looking glass into another world and can be used to further ignite curiosity and inquiry in a museum setting.

Museum audiences can experience AR through various viewing devices including tablets equipped with cameras, large wall-mounted screens or kiosks, and mobile devices. Here are a few contemporary examples:

Tablets:
Natural History Museum’s interactive film “Who Do You Think You Really Are?” as well as Metaio and Louvre-DNP collaboration

Screens:
Examples include my Wonder Turner installation at the Ontario Science Centre, and Total Immersion’s environmental kiosk

Mobile devices:
An unofficial exhibit at the MoMA using Layar and one of my recent projects “AR Hanging Mobile” using ARToolkit for iPhone.

What limitations of current ways that people can experience AR art works and installations would you like to get beyond?

I’d like to see more work move beyond the single viewer experience in AR and engage larger audiences in a simultaneous viewing and even collaborative interactive experience. I think this is particularly relevant for museums in designing and producing AR experiences. My early work in AR started out with books and other small hand-held objects creating an intimate experience for a single user at a given time and then expanded to a more collaborative exploratory setting with multiple users engaging in an act of play and discovery together. There’s a great opportunity for enabling a larger group dynamic at work in AR with multi-users. This can combine visitors both on and off-site.

Let’s also make the work so wondrous people forget they are looking at a screen or using a device! I’m continually exploring ways to heighten “presence”, a term used in AR to describe the illusion of non-mediation.

***
Helen Papagiannis is an artist, designer, and researcher specializing in Augmented Reality (AR). She is presently completing her Ph.D. at York University in Toronto, Canada and is a Senior Research Associate at the Augmented Reality Lab (Department of Film, Faculty of Fine Arts). Helen’s mixed reality art installations were recently featured in a solo exhibition at the Ontario Science Centre, and at TEDx, where she was also an invited speaker. Prior to her augmented life, Helen was a member of the internationally renowned Bruce Mau Design studio, where she was project lead on “Massive Change: The Future of Global Design”, a touring exhibition and book published by Phaidon Press.

Recently, Paul Doherty and I met in SL with New York Hall of Science (NYHOS) curriculum developer (and Museum Virtual Worlds contributor) Ray Ferrer, along with some adventurous high school Explainers.

The Hall is working with their first cohort of high school Explainers to envision, design, and facilitate the virtual space that will be the new Virtual Hall of Science (VHOS). The meeting/tour participants had a look at what the Exploratorium has been doing with exhibit development in virtual environments and got an introduction to some of the environment and object building processes in Second Life. We played with different exhibits and chatted about things the Exploratorium has learned in developing exhibits there, including the interaction benefits of putting the avatar into the exhibit as much as possible and of moving the avatar as part of the exhibit experience. I’m looking forward to seeing how the new VHOS develops!

Visitors from NYHOS check out some exhibits on 'Sploland in SL

Trying out Dance Floor Color Mixer

NYHOS Explainer avatars being moved in an exhibit