Museum Virtual Worlds

A few months ago, I was introduced to Helen Papagiannis, an artist, designer, and researcher working with the emerging technology Augmented Reality (AR). I was captivated by the way her playful AR exhibits and installations drew people in through touch, video, and sound. She’d recently exhibited her work at the Ontario Science Center and was exploring ways to engage museum audiences with their sense of discovery and wonder. I caught up with her recently and asked her some questions about the ideas behind her work.

The Amazing Cinemagician: New Media Meets Victorian Wonder” exhibition by Helen Papagiannis at the Ontario Science Center, May-Sept., 2010, Toronto, Canada. Photos: Pippin Lee

When did you start experimenting with augmented reality?

I began experimenting with AR in September 2005. When I saw AR for the first time, I was so entranced I think I entered a permanent state of wonder with the technology. And it was all very simple: a bare bone 3D virtual cube seemingly appearing in my physical space. It was completely astonishing! I went into mad scientist mode from there tinkering, prototyping, and dreaming of the creative possibilities for AR. Five and a half years later, and I’m still riveted.

What are some of the challenges that you’re exploring in your AR work?

When I began working with AR, the challenges were largely around the technical constraints. It has been important for me to work with what is at hand, right now, not tomorrow, or 6 months from now. I always ask, ‘How can we realize this now and make it compelling within the parameters?’ My process has entailed allowing the constraints to guide the work, then finding ways to push beyond those boundaries to create something new.

I strongly believe AR is emerging as a new medium and it will come to play a large role in entertainment and information sharing. The challenge at hand is to continue to investigate how best to apply the medium and really elevate it creatively, and to do this as a community of artists, engineers and industry. We need to identify what is truly unique about this new form, and how we can best leverage these characteristics to tell new stories and create engaging experiences that are unlike anything we’ve ever seen before.

How can museum audiences experience AR?

We’re beginning to see more AR applications in museums, which is very exciting. AR is becoming more accessible and affordable, including the use of personal devices that museum visitors may already have at hand, such as smart phones. AR can be used to provide additional information about objects in a museum’s collection and to enable experiential learning through discovery and play. Wonderment, as discussed in my TEDx talk, is an important part of my work in AR. For me, AR fosters a great sense of wonder as a looking glass into another world and can be used to further ignite curiosity and inquiry in a museum setting.

Museum audiences can experience AR through various viewing devices including tablets equipped with cameras, large wall-mounted screens or kiosks, and mobile devices. Here are a few contemporary examples:

Tablets:
Natural History Museum’s interactive film “Who Do You Think You Really Are?” as well as Metaio and Louvre-DNP collaboration

Screens:
Examples include my Wonder Turner installation at the Ontario Science Centre, and Total Immersion’s environmental kiosk

Mobile devices:
An unofficial exhibit at the MoMA using Layar and one of my recent projects “AR Hanging Mobile” using ARToolkit for iPhone.

What limitations of current ways that people can experience AR art works and installations would you like to get beyond?

I’d like to see more work move beyond the single viewer experience in AR and engage larger audiences in a simultaneous viewing and even collaborative interactive experience. I think this is particularly relevant for museums in designing and producing AR experiences. My early work in AR started out with books and other small hand-held objects creating an intimate experience for a single user at a given time and then expanded to a more collaborative exploratory setting with multiple users engaging in an act of play and discovery together. There’s a great opportunity for enabling a larger group dynamic at work in AR with multi-users. This can combine visitors both on and off-site.

Let’s also make the work so wondrous people forget they are looking at a screen or using a device! I’m continually exploring ways to heighten “presence”, a term used in AR to describe the illusion of non-mediation.

***
Helen Papagiannis is an artist, designer, and researcher specializing in Augmented Reality (AR). She is presently completing her Ph.D. at York University in Toronto, Canada and is a Senior Research Associate at the Augmented Reality Lab (Department of Film, Faculty of Fine Arts). Helen’s mixed reality art installations were recently featured in a solo exhibition at the Ontario Science Centre, and at TEDx, where she was also an invited speaker. Prior to her augmented life, Helen was a member of the internationally renowned Bruce Mau Design studio, where she was project lead on “Massive Change: The Future of Global Design”, a touring exhibition and book published by Phaidon Press.

I haven’t posted about any new developments with virtual worlds that the Exploratorium is developing in a few months. Fortunately, my lack of writing about it doesn’t equate to nothing going on in that realm. Exploratorium Island and ‘Sploland island in Second Life continue to thrive with several hundred visitors per week each, guided tours of exhibits, and occasional building parties and building tutorials. We’re thinking about what events would be good to stage there in the coming year. Many new exhibits have been added to Exploratorium Island recently and the space is undergoing a needed make-over. More on these developments soon!

Now I’m going to begin a little detour from immersive 3D virtual worlds and introduce some new experiments with mobile augmented reality (AR) –an emerging technology and practice that allows you to overlay a view of a physical environment or object with virtual content. We did an experiment here at the Exploratorium last year with AR, as part of our kick-off for the celebration of our 40th anniversary. It presented a real-time rendered view of an overlaid interactive 3D model onto a 2D marker that was embedded in the cover design of our quarterly publication, Explore. This was inspired by others beginning to use the print medium as an entre to the technology. I first became fascinated with the idea of location-based augmented reality when I read descriptions of pieces that “locative” artists made in William Gibson’s book Pattern Recognition. In his post-cyberpunk story, Gibson describes several large-scale 3D virtual objects that are placed at specific locations meaningful to the story (via geo-referencing). A giant octopus, a field of moving flowers, a re-created scene, etc. Viewers needed to don cumbersome headgear to see these things, which were rendered in high resolution 3D stereo and composited compellingly into a view of a physical environment. It reminded me of things I’d tried previously with slide and video projections at specific locations. In a digitally-enhanced world, with the ubiquity of the Global Positioning System and the mass-market access to it through hand-held devices including smartphones, augmented reality is now a technology that many can experience, and one that museums are experimenting with.

For me, augmented reality represents a slice of a virtual world that people can experience on the go, in their day-to-day lives. It seems like a fluid extension of immersive 3D virtual worlds in that virtual and physical world realities are mixed. There are possibilities for blending communication (social aspects) and user-generation of content in the augemented views. AR is also being heavily influenced by the commercial gaming space, with haptics and full-body gestural interfaces promising more natural-feeling interaction with augmented views. Products like the Wii and Kinect are being used as interfaces to non-gaming content. Open-source toolkits to use those devices are now available. More on that topic soon.

AR technology is in an early stage. Hardware and software platforms are rapidly emerging, open and proprietary systems are vying for developer and consumer attention, business models are being invented, content developers are exploring different uses. Museums are beginning to develop models for use for AR, designed to engage visitors in discovery of “extra” information about objects, exhibits, and places. The New Media Consortium 2010 Horizon Report: Museum Edition, tracks the time-to-adoption for AR as two to three years. So, it’s an exciting time to see this new technology grow and to work with it yourself to see how you can use it to engage your audiences.

For those not familiar with the VHOS project, it is essentially a virtual space within the Active Worlds Universe in which the New York Hall of Science intends to create explorable/interactive exhibits through a collaborative process involving the contributions of Hall staff, Hall Explainers, participants of the Hall’s camp programs and finally (and ideally) casual visitors. The first phase of the VHOS project was simple enough– train a group of 18-23 year olds to use Active Worlds to a point in which they are comfortable creating things as well as showing others how to create things in-world. The second phase was a reminder that no design can be efficient without prototyping; middle schoolers have knack for showing you that the way you think they think is wrong and so anything designed for them will likely have to be revised on the fly. The third phase of the VHOS project was an interesting reminder for myself about how the process of designing something that actually meets needs is iterative. So while I was thinking that I could have veteran participants take a hand in delivering basic skills to newer participants, they just weren’t interested in being teachers. As a solution to this we introduced the “Easter Egg“. As new participants acquainted themselves with the basic navigation and building skills, veteran participants were given a “mission”; first, create an easter egg containing some scripting skills considered advanced for the newbies, then secretly place that egg somewhere on a newbies virtual property. So here we have veterans showing off there skill in a way that newbies can glean important skills from. Some veterans went as far as to create portals that will take you to a secret location containing your personalized easter egg.

Unlike the second phase, the third phase was focused on one content area. Participants designed and developed virtual exhibits dealing only with the phases of matter. During phase two of the VHOS project it appears that participants were a bit overwhelmed by the option of selecting any STEM topic of their choice. Too much time was spent narrowing down the focus of their designs and not enough designing. The effects of this can be seen when contrasting a phase two exhibit, which often illustrates a broad concept, with a phase three exhibit illustrating some characteristic feature of a substance transitioning from one phase of matter to another.

As we continue to run camps the VHOS becomes richer with educational experiences which will inevitably lead to the issue of categorizing the exhibits and directing the user/casual visitor in a way that facilitates learning. I’m excited to see where this is leading as there is already a feel of being in a place where someone has been before you, giving the space and how you experience that space siginificant thought.

Hello World! My name is Ray Ferrer. I’m a Digital Learning Curriculum Developer at the New York Hall of Science currently incorporating virtual worlds into the learning experiences here at the Hall. As the Hall’s first endeavor using 3D virtual environments to facilitate learning, I’m excited to report that our first run was promising as an indicator of the type of learning experiences that can be had.

Using an Active Worlds space graciously donated by Cornell University, participants of the VHOS project went through a four-day camp learning how to navigate and build in the environment, research a STEM topic of their choice, learn exhibit design from and expert, and finally design their own exhibits in-world. But that’s not where it ends– in fact that’s not even how it began. Prior to the camp, a team of Explainers (the Hall’s equivalent of a docent) went through a series of AW trainings in order to help  camp participants realize their designs. At the conclusion of the camp participants completed a draft of their exhibit designs. The images included below are samples.

During the week of April 14th-17th, new participants will begin the process of populating the VHOS space with their own exhibit designs while returning participants work on reiterations of their designs as well as help teach new participants the fundamentals (and obstacles) of designing in the AW environment.

The aprenticeship model that we are using has been succesful for the Hall in past programs and I trust that it will be as effcective in virtual environments. I’m eager to see the designs that come out of this project and will keep the readers of this blog posted.

People often ask me what is there ‘there’ that keep people so fixated on their screens. I briefly wonder whether they are about to launch into a tirade about how pathetic it is to see someone locked into their favorite sitcom; or how come their kids seem to run straight to instant message their friends from home after spending the whole day in school with exactly those same friends. But these days it seems that that the puzzlement is not directed at why anyone would want to keep abreast of the real, online, or onscreen lives of others, but more often than not, it’s about why anyone should want to spend their time in an immersive world such as Second Life. I suppose that in a way it’s probably a combination of both. Worlds like Second Life (SL) tend to offer both of the above; and a lot more. Not only can you track the lives of others at study, work and play, but you can also maintain a very active life of your very own in the persistent world – a world that doesn’t go away when you log off but continues exist, and even thrives in your absence. After many years of augmenting my face to face conversations with friends, family and colleagues through email, instant messaging services, image and video-sharing platforms and social network interfaces (and, no, I will not be naming names here nor will I digress to other platforms) many of these contacts have now taken up residence in Second Life. And so the conversations continue to flow – through badly spelt text chats, or earnest voice conversations in world where I am finding new ways to augment my mediated presence and catch ups with friends in the delightfully creative and resourceful world of Second Life.This Slog (SL log – as opposed to ‘blog’, the abbreviation of term or ‘web log’) will invite you to join me in my modest, ocean view home at Del Mar, will take you around the wonderful attractions that magically evolve overnight; will invite you to a front row seat to conferences and seminars that take place in world, and will prod and nudge you to move into the shoes of your own avatar and experience it all for yourself. So don’t believe that well worn chestnut by Gertrude Stein. In my modest opinion there is a lot of stuff going on ‘there’ and you are welcome to join me on these little jaunts into Second Life – to watch, listen and to celebrate what is currently going on and to discover for yourselves that there is actually something there, there.

Four conferences and a wedding

Over recent months I have been involved with in four in-world conferences. These have offered me all kinds of new possibilities; things that were not previously do-able before the Second Life platform had become robust enough for real time voice conversations, and very persuasive exchanges. Two of the conferences brought Second Life to audiences who had not yet really had an opportunity to explore what it meant to go to an island, even though they had heard about things going on there through the media, and from friends and colleagues. Other occasions were more pragmatic – acting as meetings that in essence allowed me to ‘travel’ to conferences without actually having to get onto a plane. All of the events brought together the professional museum community to a shared place and time, and allowed for meaningful exchange of ideas and experiences in and about their own practice in Second Life. Each event, in their own way, shifted the benchmark a little; all, in some way effecting how people think about presenting, and exchanging ideas, even though not all of us were actually present in the room at the time. At the EVA/MINERVA Conference on the Digitisation of Culture held in Jerusalem at the Hebrew University in November 2007 the meting was held simultaneously in Second Life and in the air-conditioned auditorium in Jerusalem . This year the annual conference hosted close to three hundred people who had come to hear how advanced technologies are currently being developed and applied in the cultural area, with an impressive range of European Commission supported projects presented during the two-day meeting. The hour and a half long session that took place in Second Life brought the physical audience together with the in-world panel of presenters, and offered an exciting glimpse about what was is currently possible in-world.

The Conference Platform, EVA/MINERVA 2008

The session took place at a round table perched on a platform 300 meters in the sky, where the in-world presenters met for an avatar-to avatar discussion about the permanency of Second Life. Each presenter in turn ‘took’ the in-auditorium visitors to their own builds; a fascinating selection of museums and culture centers, located on different Islands around the grid. The first of the projects showcased was Fred Bos from Tressis, The Netherlands (RL) / Milan Brynner’s (SL) Virtual Starry Night – Vincent’s Second Life; a gorgeous build that presents the works of the Dutch painter, gathered from all around the ‘real’ world. The project also includes a growing number of 3-dimensional paintings where visitors can literally ‘step into’ a Van Gogh masterpiece. Virtual Starry Night has become one of the more popular builds in Second Life, and has created waves far beyond the simulated world.

SL presenters meeting in the gallery at Virtual Starry Night

Aaron Collins (RL) / Xander Ruttan (SL) hosted the panel and the audience at his (SL) development, where he has attracted an international group of artists and art admirers to the Cetus Gallery District; a cultural island he created through his company, Ruttan Development. Cetus mimics real life urban arts communities such as those which often arise through the adaptive re-use of historical industrial areas, such as New York’s Chelsea, and the Pearl District in Portland, Oregon where Collins/Ruttan lived for 20 years. In Cetus residents have many opportunities to exhibit their art in galleries, coffee houses, and loft residences. The community is fostered through the collaboration of the Cetus Gallery District Association; an highly dynamic association which provides communication and marketing support for the artists and gallery owners in the district. Cetus is always abuzz with activities such as art openings, lectures, workshops, live concerts, social events, fashion shows, and community meetings. Behind the Xander Ruttan avatar is Aaron Collins, who is networked in the U.S. art world as a co-founder of a California-based nonprofit arts organization, a freelance arts & culture writer, and former associate director of a prominent contemporary art gallery.

Ruttan, Ruttan hosting his guests in Cetus Gallery District during the conference

The next two visits took place in Swedish builds, where Olle Wästberg (RL) / Olle Ivory (SL) introduced us to The Second House of Sweden where the Swedish Institute’s director, Stefan Geens (RL) / Belmeloro DiPrima (SL), the Manager of Sweden’s virtual embassy in Second Life and his team have created Sweden’s official representation in Second Life. It is a project conceived, directed and funded by the Swedish Institute, a government agency with the mandate to “share Sweden with the world”. The design of the building in Second Life was adapted from the real-life House of Sweden in Washington, DC, designed by the Swedish architect firm Wingårdh AB. Second House of Sweden was built by Electric Sheep Company, with design by Söderhavet. The project was originally conceived in January 2007 and inaugurated in May. It exists to showcase Sweden for Second Life inhabitants, but above all it allows their staff to experiment with the new immersive medium, allowing both them, and their many visitors to discover what works and what doesn’t. Second House of Sweden houses a number of permanent exhibits, including paintings from Stockholm’s National Museum, an exhibit on Linnaeus, and a reproduction of Swedish diplomat Raoul Wallenberg’s office. To date, special events have included Swedish lessons, gallery talks and inauguration parties. A short film festival for Swedish films is planned. .

The Second House of Sweden

The second Swedish trip was organised by Marie Rytk?l? (RL) / Kaja Lurra (SL) who invited the audience to her build; Stockholm‘s Old City, Gamla Stan. Kaja is a SL terra-firma who specialises in sims that create specific cultural ambiance. From the auditorium at the Jerusalem located conference, Kaja took us to the pre-launch of Stockholm’s Old City, Gamla Stan, that attracts Swedes to listen to music, dance together, wander around the gorgeous sim, get a massage, order pizza or beer or simply hang out together by the port and the magnificent ships moored there.

Kaja Lurra (SL) build; Stockholm‘s Old City, Gamla Stan

The last panelist invited visitors to Exploratorium Island, Sploland, and Midnight City’s The ‘Splo where Rob Rothfarb (RL) / Pepto Majestic (SL), Director of Web Development at the Center for Learning and Teaching at the Exploratorium, San Francisco introduced the audience to their official presence in SL on Exploratorium Island, located adjacent to Sploland. This multi-purpose space features both indoor and outdoor exhibit areas, a large amphitheatre for Webcast and other programs, as well as teaching spaces for different audiences. Both Sploland and the Exploratorium Island are part of an archipelago of science-themed educational places called SciLands, and, as this Slog’s host recognises, the team at the Exploratorium have been one of the leaders in the museum community to use collaborative multi-user environments, such as SL to further engage its audiences and to extend more of its programming and educational resources into highly interactive, networked places.

Exploratorium Island, Sploland, and Midnight City

The Second conference in this Slog took everyone, speakers and participants alike to the New Media Consortium Conference Center (NMC) on March 8, 2008. This pioneering conference was organised by The Virtual Worlds: Libraries, Education and Museums Conference, and was held exclusively in Second Life. The day long conference, thoughtfully encompassing as many times zones as humanly possibly, took place in several locations on the conference center. As most sims have obvious traffic limitation, the cut off for this conference seem to have been around 80 attendees and there were clearly many more people interested in attending the conference that was possible in spite of the $8000 Lindens (US $30) registration fee.

Keynote address at the New Media Consortium Conference Center (NMC)

The demand was so great on the day that, in fact, some people coordinated their leaving the sim with others so that their friends could get access. Barbara Galik (RL) /Puglet Dancer (SL) and Kitty Pope (RL) / Kitty Phillip (SL) presented the keynote discussion: “Virtual Worlds: Libraries, Education, Museums, and More” in the main auditorium and the breakout parallel sessions took place at number of locations around the build. The animated auditorium was packed with avatars, all watching the slides on the stage and listening to the presenters over voice. The audience was a mixed pack – with not all attendees recognisably human, but just as engrossed in the proceedings as their human-shaped participants seemed to be. I have to admit that I was engrossed with my own direct chat with another participant during the keynote; luckily, and unlike traditional conferences, nobody noticed out intense chatting on our keyboards.

Conference attendees at New Media Consortium Conference Center (NMC)

My own presentation, Persistent Worlds: Will They Ever Go Away? took place in voice at the Muriel Cooper Coliseum and argued that these kinds of worlds are far from being a fringe fantasy land for pure escapists and have long since developed into a persistent world for play, commerce, creativity and exploration. My own focus, as always, was on cultural institutions, museums and historical simulations. I was thrilled at the lively conversation that took place as my slides came into focus (I have since learned to pre-load them as one would when caching a website you want to present) and the audience contributed almost as much as I did. At some point, during my hour long presentation, I did suffer a brief lapse in my own suspension of disbelief (in the Mel Slater term), when my projection of me was jerked into the reality of my own living room when the (very real) family popped in for visit – it was a normal Saturday afternoon after all. In spite of this very tempting distraction, I was able to refocus – much like one would for a phone call – and continue my presentation to the avatars attentively sitting in the Coliseum, waiting for my next slide – I don’t think they noticed the kids trouping in for tea … All the presentations are online on the NMC website.

Just before the breakaway session at New Media Consortium Conference Center (NMC)

The next in-world meeting was the Crit Room at the (very real) Museums in the Web 2008 conference, which took place in Montreal, Quebec, Canada, April 2008. Unable to travel to the conference the Chairs and organizers Jennifer Trant and David Bearman agreed for me to present the critique of the three museum websites from an in-world location, the Exploratorium amphitheatre; the successful location of previous MW events. Setting up the extra screen and audio feeds proved to be worth the effort, as the session went smoothly, with the participants in the hall watching to Jennifer Freund (my avatar) and listening to my voice in real time. By now I had learned to preload the slides, and the three sets of screen shots moved smoothly throughout the session. While I could hear some of the discussion going on in the hall, specifically those voices located to the microphone feeding into the SL platform, the key comments were being faithfully relayed to me by chat by my colleague, Rob Rothfarb who was physically present in the room at the time, allowing me to be able to respond to questions and comments. The email responses I received after the conference indicated that what was important not that the whole thing took place embodied by a moving animated character in a synthetic amphitheatre, rather that the content and message was pertinent and evidently useful.

Crit Room at Museums and the Web 2008

And now the wedding

When people come together in Second Life, whether in a professional conference, or round a table for a quick chat, it is usually driven by a desire to remain connected. Second Life is a highly social place, where commerce, learning or play takes place in a shared space. Friendships, as in real life, often blossom into long term commitments and what better a way to consolidate that commitment in the synthetic world than with a real time, Second Life wedding. Over the coming months this Slog will, introduce you to the community of Del Mar; the island where I have built my own modest house. In spite of the fact that many of my neighbors are on different time zones, and we don’t actually meet up as often as we might like, we do look forward to catching up with the island gossip and keeping up to date with ongoing lives of its residents. As in real life, couples come together; and split up from time to time in Second Life, pronouncing their ‘couplehood’ after the nuptials in their online profile.

Over the months that I have ‘lived’ there, I have been invited to a number of weddings; some highly extravagant, while others more modest; all driven by the basic human desire to ‘perform’ watershed moments of ones’ life as a collectively celebrated spectacle. In order to protect my neighbor’s RL identities, I will not be using either their RL or SL names, neither will I be sharing the wedding photos here in my Slog, but needless to say, this was one of the more spectacular of weddings that was celebrated in Second Life in recent months.

The wedding took place in a specially constructed wedding chapel, floating high up in the clouds over our Island. The couple took their vows in the presence of all their friends; some of whom I knew from Del Mar, while others teleporting in from other more distant islands; but all there to support their friends in their special moment and to witness the changing status of the wedding couple.At the crucial moment the groom took his windlit bride in his simulated arms and embraced her lovingly – the especially written script flowing out from the blue and pink balls on the stage. At the auspicious moment a flock of white doves flew out behind them, and the delighted guests sitting on the pews greeting them with cheers and blessings of their own. I have to admit that I didn’t even mention that I had just attended a SL wedding to my own family; they would, no doubt scoff at the very idea. But those guests and friends who attending this particular wedding, knew something that others, outside of the persistent world knew. They had all come together to be there at this specific moment, to support their friends; and they were there because they recognized that after all, there really is something there, there.

Ahhh yes – I had promised you a fourth conference in this first of my Slogs. Well you will have to come back for that one – it is due to take place on May 18, 2008, when ICOM celebrates International Museum Day. This year ICOM is celebrating their special day as Museums: agents of social change and development, and what better place to celebrate than on The Tech Island in Second Life. But make sure that you register before hand, because this sim can only hold 50 avatars at a time! … and if you are a little curious about what a SL wedding looks like I would suggest you Google ‘wedding+Second Life’ you would amazed at how many results you get … and don’t forget to Google the images too…

Susan Hazan / Jennifer Freund