Pressing start

So here we are, eight days until our LCC exhibition and showcase and we have pretty much everything in place for our demo/vertical slice. I think Simon and I have both been so focused on getting here that we haven’t really had a chance to stop and go – wow – we’ve done it!

But when I saw our logo and start screen on Sunday, I finally arrived at that moment.

So here we are folks, we’re ready to go!

 

A milestone

Yesterday we rehearsed and then recorded our final V.O for Fractus, which felt like such a milestone! Huzzah!

We worked with a wonderful performance director, Deirdre Daly and two brilliant actors. Neath Champion-Weeks, as the voice of Melody and Elroy ‘Spoonface’ Powell as the voice of Fractus.

Pivot(al) Conversations Part 3

Yesterday Simon and I met with Ana Tudor who is the Head of MA VR. It was a really brilliantly helpful meeting, and another one of those important conversations which once again made us rethink the direction we’ve been exploring.

We showed Ana the visuals through this video capture:

And also some of the very rough story development on Twine (this link must be copy & pasted to work): file:///Users/larabarbier/Dropbox/Fractus/Story/Scripts/A.L.I.A.S_draft1.html

Her biggest piece of feedback was that although she liked exploring the concept of A.I by using live actors, this isn’t something that UAL could do in an exhibition space, and that it would be difficult for us to produce in the way we wanted to. She recommended we refocus back on the Fractals and use a voice over to tell the story.

My big take away from this is that I think I’m very interested in location-based VR, and the inter-play between actors and VR technology and that this is something I should continue to explore, but that it’s not possible for this project!

For Simon, I think his concern now is that we’re running low on time and we are going into a third story phase.

My gut feeling is that we possibly don’t need a narrative for this project – at least not in a traditional sense. The fractals are interesting in themselves. I wonder how it might be if we just allow our users to manipulate them along with some very lovely ambient sound. Do we need voices?

However, Simon does really want this narrative element, and it’s been one of the major challenges in our project to find it.

So…it’s my job now is to come up with a story that uses the fractals, still explores our relationship to AI in an interesting way and offers a different interaction other than speaking. Back to the drawing board…

Storyboarding and storytelling

People often ask Simon and I how we manage being a husband and wife team. It’s true that there are obviously additional challenges mixing such personal and professional relationships, but on days like today it just makes me really appreciate how well we work together.

As a screenwriter, I am very rarely given the opportunity to be part of any storyboarding work. So it was great fun to take my rough script which I’d mocked up on Twine, and then work with Simon to translate that to visuals. It was incredibly helpful for me, as we were able to discuss why the script wouldn’t work in places and where it would. How visuals could underline certain pieces of info, and how to use colour to produce emotions. Also it really showed me how little dialogue is needed. Obviously, this is a screenwriter’s first commandment (show, don’t tell), but I still tend to overwrite things.

Simon has a background in animation, and has worked for games studios and so it’s always a pleasure to work with him – even though we have different skills and experiences – we have a shared language for visual storytelling, and that really is a great strength in this partnership.

Raindance Immersive 2019

Venus created in VR

Copyright: Raindance Immersive

I was lucky enough to attend the Raindance Immersive Summit last week as well as nab tickets for the amazing XR experience Cosmos Within Us and VR experiences Ayahuasca and Heart of Darkness.

For me it was such a fantastic event to attend because:

1) it was a privilege to listen and learn from those people forging ahead on this XR path and

2) it was such a generous and supportive environment.

Pretty much every speaker just kept reiterating that “there is no right way” yet and that everyone is experimenting and making things up and just to get stuck into these emerging technologies and have fun!

Notes from the summit

The theme of the festival is female creatives and Brexit.

Raindance Immersive has only been running since 2016 (Instigated and now curated by Mária Rakušanová)

Trends in 2019:

Tilt brush has become THE TOOL to use for VR animation and design.

Acting talent is being drawn into the medium, as are other high profile mainstream names (i.e David Attenborough). XR is no longer a “fringe medium” although still a long way to go.

The rise of AR and mixed reality such as Rise of Animals (using magic leap).

Gaming is also colliding with VR for example Doctor Who: Edge of Time and The Infinite Hotel.

Branching narrative is being pushed, such as After Life.

Computer generated experiences are becoming ever bolder: Ayahuasca  and Heart of Darkness.

How you navigate in XR is still experimental. New technological developments emerging are the use of blinking and eye tracking as a narrative mechanic for example in Look again and Drip Drop.

Live performance meets VR in realtime with Box in the Desert and Cosmos Within Us.

Sound design is finally being recognised as a pivotal part of the experience, and awards are being created for this.

TV IP is moving into immersive content.

Rise of Animals

This is a mixed reality experience. Users wear “glasses” which allow the creators to put animals “into the room.” The glasses/headset traces the room in infrared and responds to eye tracking. It also has directional speakers. It took 4 years for them to get to this stage.

Any new medium comes with new lessons. The four big challenges for our creators:

  1. The space (real and visual)
  2. The UX
  3. The UI
  4. Tech limits

Can this be used at home? They’re not yet sure. For now they have created a curated space for people to visit and wear the glasses.

This is a new UX design: the controller is your head and eyes and hands (removing controllers).

They used a footpad anchor which allowed users to return there to trigger the next animal. To guide users gaze they implemented a particle affect and ambisonic audio.

Their decision between Rift versus Quest came down to the fact they needed it to run on mobile. To do this they had to significantly reduce their polycount.

They discovered that lighting was absolutely key, and it was difficult to balance real versus virtual light.

They did a lot of user testing which unearthed some not so obvious problems. So far they have tested on 150 people.

Moving forward plans include multiplayer options and using real assets.

The main thing is to embrace the change! Media is converging between TV, Film, AR & VR. Be multidisciplinary.

Anonymous

This was created using a very small budget (£2000). They shot on an instapro 360. It’s a 12 minute monologue based on the creators personal experience with an alcoholic parent.

Over the 12 minutes she moves between 5 different chairs representing the 5 stages of grief. They did also add in some graphics to give it a child-like theme. This was an issues based/awareness raising piece.

Playing God

This was another small budget looking at immigration. A branching narrative piece that has to conclude in 1-2 minutes. Live action experience and basically asks “how much of an arse are you?” The point of the experience is you’re not supposed to know that you’re making choices about real people. But they found that until people understood it was about real people – they didn’t really engage. It was tricky to know how much to give away – because you do want people to care and have stakes. This was used as a catalyst for conversation.

After Life

The story is composed of 45 scenes shot in live-action VR.

There are 29 branching paths and 5000 possible interactions and multiple outcomes.

After Life Branching map

They designed gate-controlled paths. The branches happen when you follow a character. You don’t consciously “choose” it’s more organic.

After Life, example scene outline

Discovered that space became a character, and the house its self was a character. Even if you have multiple branches in a story, you still structure your story according to three act structure. There are no templates or “rules” about what a VR shot list or outline looks like, and so they combined knowledge from standard filming with what they learned they needed in order to be able to film in 360 degrees.

Panel discussion: Location-based Entertainment

  •  A Box in a Desert | Nanna Gunnar, Director, and Owen Hindley, Director Huldufugl (UK)

  •  Cosmos Within Us | Tupac Martir, Director (UK)

  • Alexia Kyriakopoulos, Arvore (Brasil, Greece)

  • Panel moderated by Nina Salomons

Box in the Desert, by Huldufgl is an interactive digital fable. You are in a box, and a character appears outside of the box (played by an actor) telling you you’re safe as long as you stay inside the box. But you can hear a third person speaking to you telling you not to trust the external character. You have to decide who you will trust.

Location-based VR immerses all your senses. In Box in the desert you are interacting with people on a set. It’s a role playing theatre experience. So far they have found that because they don’t feel there is an audience or a person there that people will come out fo themselves a bit more. The screen gives they a mask and avatar.

Box in the Desert set up

In Cosmos within Us the interacter becomes important because they frame the shots for the audience. Tupac (the director) can control the heat, volume and sound, so can be very responsive. Sometimes people start talking back! There are also smells in Cosmos within us which is very powerful for directing people in the space.

In Box in the desert you do sometimes get an interacter that is a challenge or tries to push the limits. One question they ask is “Where do you want to go? Space?” And the interacter did not want to be lead so said “No, and asked to go under the sea.” They couldn’t do that because they hadn’t created that for the experience, so the compromise was taking them to a waterfall.

Extra sensory elements like smell and touch add to the feeling. Your brain does add a lot too. Some people are convinced they smelled something in Cosmos within us when in fact they didn’t.

Funding is always an issue. Box in the desert just did it for fun and in spare time. Because it only requires one actor that’s made it doable. But the challenge for making it profitable is that you can only have one audience member. So to sell 350 tickets you have to do 350 shows.

Cosmos within us got most of its money from Luxembourg. They have also been scaling up audience. In Venice they only had 4 audience members. At Raindance they have 10 and hopefully in the future they can have 40. So if they take over a theatre the distribution model changes – but also brings new challenges – do I need extra screens for the audience?

In video games people watch other people play games like on Twitch. Theatre’s should start taking on VR projects, because people will pay to watch others experience something, if they can also get something from it.

Could they also join experiences up? So you buy one ticket and can see three performances back to back over three hours (maybe with an interval)?

Glimpse

The story is about Herbie (a panda) and his break up with girlfriend Rice (a deer).  The illustrations are Herby working through the breakup.

Funding was a massive hurdle for this project. They recorded the script early on. So essentially had a Radio Play and some illustrations but couldn’t do much more.

Talking through funding

They learned that there is a new language forming around VR creation. The film process is so well-laid out, but that isn’t the case for VR. For example, How do you storyboard in VR? This was a problem for funding – how to explain the production process, when they are still learning what the “correct” way to do this is themselves!

Immersive Games Panel

Think about why you’re creating your game for VR: is it emotional or is it immersive? Otherwise why do it in VR?

Can you allow people to speak to a character? Unfortunately that’s asking a huge amount. Think about how to trigger responses that feel unique to the action from the player. For example, how you open a door impacts on how a character responds. If you open it slowly or fast may indicate how the player feels and so illicit a different response from a character.

Movement is a CHRONIC issue in VR games. It’s incredibly difficult to port over games like Zelda because “running” around a world is just impossible for now.

Most traditional game narratives are 3rd person. You may not “be” that person, but you may have a relationship with that person.

In traditional games as a first person shooter, you tend to be silent and holding a gun until you get to a cut scene.

In VR you can’t speak, but you really want to!

So you are seen but not heard. Dialogue in VR is just very complex.  You can’t spark a conversation. Essentially you can say “yes” or “no.”

But you CAN use action instead.

Usually story is told through your ability to mess with the world i.e how much can you break.

What happens if you poke a character or slap etc.

Being 1st person you can’t really drive a scene. You also can’t have cut scenes. One solution is to have an AI who talks to you.

You can use subtitles BUT think about where they sit, and also do they further the narrative.

Punch drunk works well because you walk through it as an invisible person and can pick up the story at different places and it doesn’t matter.

In VR action carries all the meaning – when you move something should happen.

But the key for VR is that how you feel is more important than how you interact.

If you have a story really think about what medium it fits.

Pivot(al) Conversations Part 2

One of my LCC Screenwriting lecturer’s once said to me that often as a screenwriter you won’t be clear on what your theme is until several drafts in to a script. However, she also said that you’ll find that no matter how much you think you’re writing a unique story each time, writers often have one or two themes they spend their whole lives writing about, and you can see it crop up in their work time and time again.

It makes sense to me – part of why I think many of us do creative work is about trying to understand ourselves and the human condition. For some artists their themes may always be about abandonment, others about injustice and so forth. I think it goes to the heart of what your own “character wound” is – what is that key event or maybe several events that created that deep dark point in your soul that you’re constantly trying to unravel. The question you shout out to the universe, and keep searching for an answer to which you somehow believe will heal that existential crises in you.

So when Avril asked us the other week – what is the story you want to tell? I was at first a bit of a rabbit caught in the headlights. It wasn’t until after that conversation that I sat down to think about my work and what I tend to write about and what interests me at a subconscious level. Once I’d had time to reflect I found that the kind of themes that crop up time and again in my own work are often about death and loss. I tend to focus on human stories, on the intimate reality of the everyday in the extraordinary, to be fascinated by how others live and love and die. Part of why I became a writer was so I could live 1000 lives (plus time travel)!

For Simon, his interests tend lean more to science fiction and he enjoys in-depth world building, in particular explorations of alien “other” like Ian M. Banks “Culture.” He isn’t scared of the future – but believes that technology and machines will save us from ourselves, as such his art work and stories often tend to lean towards these areas. His fascination with creating 3D fractals is a huge motif that has been driving this project: He loves strange and alien worlds, and the 3D fractal landscapes he’s producing are a wonderful reflection of this.

Copyright: KnowYourMeme

But as we’ve continued to talk through August and September about “what we’re trying to say,” it has felt really difficult at times to hit on what the right mix that reflects both of us – how to marry a “human” story to such an alien visual.

Finally Simon proposed that a good “repeating” theme to explore exactly this juxtaposition between us was maybe “humans being replaced by machines.” It’s a very topical subject, and so we began to think about how we could create a story in VR exploring this.

One of the other ideas we had was tying voices into the Fractal movements. We then started to ask ourselves, could an interaction be that our VR user “speaks” into our fractal world and that’s affects the fractal movement? What would voice/fractal integration look like? What if our Fractal landscape answered back? What would it say?

This also lead to a conversation around the difficulty of having a “silent protagonist” in VR, where you are both extremely intimate and embedded in the world, and yet the primary interaction you would anticipate having (conversation) is still not yet technologically possible.

So then we asked ourselves…could we use VR and an actor to allow for this kind of interaction? Ambitious yes, but something we felt we wanted to explore. After all, this is what this residency is about – exploring the possibilities of storytelling in VR.

And so, to this idea:

A.L.I.A.S – Artificial Lifeform (AL) – Intelligence Articulation System

Theme: Humans replaced by technology, 

Concept: Over centuries humans have been replaced by machines and technology. eventually we will replace humans with digital versions of themselves. But what happens when the digital version is replaced for an upgrade? 

Players find themselves inside the I.A.S, conversing with an Artificial Lifeform (played by a live actor, hidden from view) nicknamed AL, and must answer a series of questions to prove their worth, otherwise you will be deleted for your upgrade. 

Some considerations we’re already mulling – how to ensure this isn’t just a “machines bad/humans good” paradime. To include justifiable arguments from AL as to why humans were replaced in the first place. Some of which will involve offering historical context i.e Gutenberg Printing Press democratised reading by replacing the creation of books from being a hand craft by a few to mass produced by a machine. Another consideration is the environmental impact of humans. Finally this also touches on ideas around phenomenology – a huge research topic to dig into.

I’m already slightly nervous about how to structure/ write a template for what will essentially be a live improvised performance, but I’m curious to have ago and see how I get on…

Pivot(al) Conversations Part 1

Over the weekend Simon and I had our first mentoring meeting with Avril Furness.

For me, it was really great to have a conversation with someone who is intimate with writing and storytelling in 360 degrees. It was a crucial chance to be able to air some of my concerns and struggles with finding a compelling narrative that would work with our broad theme of repeating patterns in history, linking this diaspora of voices and the abstract Fractal imagery and technology that Simon is working on.

What are you trying to say?

One thing became clear quite quickly – that Avril wasn’t just a brilliant 360 Director, but had extensive experience with being a creative director. She was quick to triage where I was coming unstuck – that our theme was just WAY too large and ambitious, and that we needed to simplify things down and focus on one simple idea to pursue.

We bounced some possibilities around over the conversation, but her advice was for us to step back from the fractals and fractal theme, and just do some concept work and blue sky thinking, and just see what stories we wanted to tell.

She also suggested we take some time out to look around at many different kinds of art and look for inspiration to see what really got us excited – which is something I’ve heard before, and I’ve used in the past, but frankly you do need a reminder when your eyeballs deep in your own project.

What are others doing with Fractals?

Avril also suggested we explore how others have used Fractals to illustrate a point or tell stories, and found some great examples.

What is already possible/being done with fractals in VR and 3D?

The top video “Like in a dream” was a huge inspiration for Simon from the start with our project and pursuing this Fractal imagery. But it was great to see where others had been forging ahead in the VR and 3D world with fractals:

How and why are fractals important? 

So, I added this as it’s something that I’ve sort of been digging into on the side anyway as part of this project. It probably deserves it’s only post at some point, so for now I’ll just pop this video Avril shared, here which is a nice summary for those curious about the story of fractals: