Pressing start

So here we are, eight days until our LCC exhibition and showcase and we have pretty much everything in place for our demo/vertical slice. I think Simon and I have both been so focused on getting here that we haven’t really had a chance to stop and go – wow – we’ve done it!

But when I saw our logo and start screen on Sunday, I finally arrived at that moment.

So here we are folks, we’re ready to go!

 

A milestone

Yesterday we rehearsed and then recorded our final V.O for Fractus, which felt like such a milestone! Huzzah!

We worked with a wonderful performance director, Deirdre Daly and two brilliant actors. Neath Champion-Weeks, as the voice of Melody and Elroy ‘Spoonface’ Powell as the voice of Fractus.

Pivot(al) Conversations Part 3

Yesterday Simon and I met with Ana Tudor who is the Head of MA VR. It was a really brilliantly helpful meeting, and another one of those important conversations which once again made us rethink the direction we’ve been exploring.

We showed Ana the visuals through this video capture:

And also some of the very rough story development on Twine (this link must be copy & pasted to work): file:///Users/larabarbier/Dropbox/Fractus/Story/Scripts/A.L.I.A.S_draft1.html

Her biggest piece of feedback was that although she liked exploring the concept of A.I by using live actors, this isn’t something that UAL could do in an exhibition space, and that it would be difficult for us to produce in the way we wanted to. She recommended we refocus back on the Fractals and use a voice over to tell the story.

My big take away from this is that I think I’m very interested in location-based VR, and the inter-play between actors and VR technology and that this is something I should continue to explore, but that it’s not possible for this project!

For Simon, I think his concern now is that we’re running low on time and we are going into a third story phase.

My gut feeling is that we possibly don’t need a narrative for this project – at least not in a traditional sense. The fractals are interesting in themselves. I wonder how it might be if we just allow our users to manipulate them along with some very lovely ambient sound. Do we need voices?

However, Simon does really want this narrative element, and it’s been one of the major challenges in our project to find it.

So…it’s my job now is to come up with a story that uses the fractals, still explores our relationship to AI in an interesting way and offers a different interaction other than speaking. Back to the drawing board…

Raindance Immersive 2019

Venus created in VR

Copyright: Raindance Immersive

I was lucky enough to attend the Raindance Immersive Summit last week as well as nab tickets for the amazing XR experience Cosmos Within Us and VR experiences Ayahuasca and Heart of Darkness.

For me it was such a fantastic event to attend because:

1) it was a privilege to listen and learn from those people forging ahead on this XR path and

2) it was such a generous and supportive environment.

Pretty much every speaker just kept reiterating that “there is no right way” yet and that everyone is experimenting and making things up and just to get stuck into these emerging technologies and have fun!

Notes from the summit

The theme of the festival is female creatives and Brexit.

Raindance Immersive has only been running since 2016 (Instigated and now curated by Mária Rakušanová)

Trends in 2019:

Tilt brush has become THE TOOL to use for VR animation and design.

Acting talent is being drawn into the medium, as are other high profile mainstream names (i.e David Attenborough). XR is no longer a “fringe medium” although still a long way to go.

The rise of AR and mixed reality such as Rise of Animals (using magic leap).

Gaming is also colliding with VR for example Doctor Who: Edge of Time and The Infinite Hotel.

Branching narrative is being pushed, such as After Life.

Computer generated experiences are becoming ever bolder: Ayahuasca  and Heart of Darkness.

How you navigate in XR is still experimental. New technological developments emerging are the use of blinking and eye tracking as a narrative mechanic for example in Look again and Drip Drop.

Live performance meets VR in realtime with Box in the Desert and Cosmos Within Us.

Sound design is finally being recognised as a pivotal part of the experience, and awards are being created for this.

TV IP is moving into immersive content.

Rise of Animals

This is a mixed reality experience. Users wear “glasses” which allow the creators to put animals “into the room.” The glasses/headset traces the room in infrared and responds to eye tracking. It also has directional speakers. It took 4 years for them to get to this stage.

Any new medium comes with new lessons. The four big challenges for our creators:

  1. The space (real and visual)
  2. The UX
  3. The UI
  4. Tech limits

Can this be used at home? They’re not yet sure. For now they have created a curated space for people to visit and wear the glasses.

This is a new UX design: the controller is your head and eyes and hands (removing controllers).

They used a footpad anchor which allowed users to return there to trigger the next animal. To guide users gaze they implemented a particle affect and ambisonic audio.

Their decision between Rift versus Quest came down to the fact they needed it to run on mobile. To do this they had to significantly reduce their polycount.

They discovered that lighting was absolutely key, and it was difficult to balance real versus virtual light.

They did a lot of user testing which unearthed some not so obvious problems. So far they have tested on 150 people.

Moving forward plans include multiplayer options and using real assets.

The main thing is to embrace the change! Media is converging between TV, Film, AR & VR. Be multidisciplinary.

Anonymous

This was created using a very small budget (£2000). They shot on an instapro 360. It’s a 12 minute monologue based on the creators personal experience with an alcoholic parent.

Over the 12 minutes she moves between 5 different chairs representing the 5 stages of grief. They did also add in some graphics to give it a child-like theme. This was an issues based/awareness raising piece.

Playing God

This was another small budget looking at immigration. A branching narrative piece that has to conclude in 1-2 minutes. Live action experience and basically asks “how much of an arse are you?” The point of the experience is you’re not supposed to know that you’re making choices about real people. But they found that until people understood it was about real people – they didn’t really engage. It was tricky to know how much to give away – because you do want people to care and have stakes. This was used as a catalyst for conversation.

After Life

The story is composed of 45 scenes shot in live-action VR.

There are 29 branching paths and 5000 possible interactions and multiple outcomes.

After Life Branching map

They designed gate-controlled paths. The branches happen when you follow a character. You don’t consciously “choose” it’s more organic.

After Life, example scene outline

Discovered that space became a character, and the house its self was a character. Even if you have multiple branches in a story, you still structure your story according to three act structure. There are no templates or “rules” about what a VR shot list or outline looks like, and so they combined knowledge from standard filming with what they learned they needed in order to be able to film in 360 degrees.

Panel discussion: Location-based Entertainment

  •  A Box in a Desert | Nanna Gunnar, Director, and Owen Hindley, Director Huldufugl (UK)

  •  Cosmos Within Us | Tupac Martir, Director (UK)

  • Alexia Kyriakopoulos, Arvore (Brasil, Greece)

  • Panel moderated by Nina Salomons

Box in the Desert, by Huldufgl is an interactive digital fable. You are in a box, and a character appears outside of the box (played by an actor) telling you you’re safe as long as you stay inside the box. But you can hear a third person speaking to you telling you not to trust the external character. You have to decide who you will trust.

Location-based VR immerses all your senses. In Box in the desert you are interacting with people on a set. It’s a role playing theatre experience. So far they have found that because they don’t feel there is an audience or a person there that people will come out fo themselves a bit more. The screen gives they a mask and avatar.

Box in the Desert set up

In Cosmos within Us the interacter becomes important because they frame the shots for the audience. Tupac (the director) can control the heat, volume and sound, so can be very responsive. Sometimes people start talking back! There are also smells in Cosmos within us which is very powerful for directing people in the space.

In Box in the desert you do sometimes get an interacter that is a challenge or tries to push the limits. One question they ask is “Where do you want to go? Space?” And the interacter did not want to be lead so said “No, and asked to go under the sea.” They couldn’t do that because they hadn’t created that for the experience, so the compromise was taking them to a waterfall.

Extra sensory elements like smell and touch add to the feeling. Your brain does add a lot too. Some people are convinced they smelled something in Cosmos within us when in fact they didn’t.

Funding is always an issue. Box in the desert just did it for fun and in spare time. Because it only requires one actor that’s made it doable. But the challenge for making it profitable is that you can only have one audience member. So to sell 350 tickets you have to do 350 shows.

Cosmos within us got most of its money from Luxembourg. They have also been scaling up audience. In Venice they only had 4 audience members. At Raindance they have 10 and hopefully in the future they can have 40. So if they take over a theatre the distribution model changes – but also brings new challenges – do I need extra screens for the audience?

In video games people watch other people play games like on Twitch. Theatre’s should start taking on VR projects, because people will pay to watch others experience something, if they can also get something from it.

Could they also join experiences up? So you buy one ticket and can see three performances back to back over three hours (maybe with an interval)?

Glimpse

The story is about Herbie (a panda) and his break up with girlfriend Rice (a deer).  The illustrations are Herby working through the breakup.

Funding was a massive hurdle for this project. They recorded the script early on. So essentially had a Radio Play and some illustrations but couldn’t do much more.

Talking through funding

They learned that there is a new language forming around VR creation. The film process is so well-laid out, but that isn’t the case for VR. For example, How do you storyboard in VR? This was a problem for funding – how to explain the production process, when they are still learning what the “correct” way to do this is themselves!

Immersive Games Panel

Think about why you’re creating your game for VR: is it emotional or is it immersive? Otherwise why do it in VR?

Can you allow people to speak to a character? Unfortunately that’s asking a huge amount. Think about how to trigger responses that feel unique to the action from the player. For example, how you open a door impacts on how a character responds. If you open it slowly or fast may indicate how the player feels and so illicit a different response from a character.

Movement is a CHRONIC issue in VR games. It’s incredibly difficult to port over games like Zelda because “running” around a world is just impossible for now.

Most traditional game narratives are 3rd person. You may not “be” that person, but you may have a relationship with that person.

In traditional games as a first person shooter, you tend to be silent and holding a gun until you get to a cut scene.

In VR you can’t speak, but you really want to!

So you are seen but not heard. Dialogue in VR is just very complex.  You can’t spark a conversation. Essentially you can say “yes” or “no.”

But you CAN use action instead.

Usually story is told through your ability to mess with the world i.e how much can you break.

What happens if you poke a character or slap etc.

Being 1st person you can’t really drive a scene. You also can’t have cut scenes. One solution is to have an AI who talks to you.

You can use subtitles BUT think about where they sit, and also do they further the narrative.

Punch drunk works well because you walk through it as an invisible person and can pick up the story at different places and it doesn’t matter.

In VR action carries all the meaning – when you move something should happen.

But the key for VR is that how you feel is more important than how you interact.

If you have a story really think about what medium it fits.

AI: Friend or Foe?

Idea recap

A.L.I.A.S – Artificial Lifeform – Intelligence Articulation System

Theme: Humans replaced by technology

Concept:

In the very distant future human consciousness has been uploaded into a machine overseen by a “benevolent AI” which is used like a hive mind to power everything. Inside the machine you’re living out your perfect paradise. However, this AI’s key operative is to use resources effectively and always look for increasing optimisation.

You are rudely “woken” from your digital paradise by A.L.I.A.S (A.L) for a nightmare “job interview” in which you must argue why your digital self must not be deleted and replaced by a more efficient operating process. In this scenario you must answer a series of questions to prove your worth. However, as you speak with A.L you are really presented with the facts that the world has become a much better place since human’s stopped running it. Your final choice really becomes about asking yourself “Just how altruistic am I?”

The final twist is that our audience will not realise they are conversing with a human who “plays” A.L.

Research

It’s been interesting to drill down and imagine how you would argue the case for humans. Simon and I carried out a few interviews and a recurring argument that humans make in this “scenario” is that we are uniquely creative. But is that true?

Some stand out quotes and food for thought from others exploring this argument:

“Humans are not original. We only reinvent, make connections between things we have seen. While humans can only build on what we have learned and what others have done before us, machines can create from scratch…Ultimately, humans are mere biological machines, and conversely, a thinking, dreaming computer could be considered a silicon life-form. If we can be creative, why not computers? Computers may even become more creative than us, as Klingemann proposes. By trawling the web, they will have access, potentially, to all knowledge. Our human brains are too limited to imagine how powerful machine creativity may become.” – The Guardian, Can Machines be more creative than humans?

Marcus Du Sautoy has written an entire book, The Creativity Code on this subject exploring the nature of creativity and how long it may be before machines even overtake humans in their imagination. I’m currently about half way through but can definitely recommend it!

 

The other argument often used against machines is that we are inherently empathetic. But again, this appears to be a false argument:

“These things are already better at empathy than we are,” he says. “AI is able to recognise a false smile as opposed to a genuine smile better than a human can.” – The Guardian, Could robots make us better humans?

As I’ve been researching this idea, I’ve found myself asking myself just how much do humans deserve to exist these days? How would I justify myself? How do I quantify my worth?

This has lead me down quite a millennial train of thought by tying my worth to my productivity. This seems to be a unique quantifier for my generation and brings up it’s own issues.

“Many millennials feel starved of meaning, lacking purpose, and desperate for some sense of identity that can ground them.” Inspiring Interns, The Rise of Hustle Culture

“Welcome to hustle culture. It is obsessed with striving, relentlessly positive, devoid of humor, and — once you notice it — impossible to escape. “Rise and Grind”…This is toil glamour” – New York Times, Why are young people pretending to love work?

“What made millennials the way they are? Why are they so burned out? Why are they having fewer kids? Why are they getting married later? Why are they obsessed with efficiency and technology…Our entire lives are framed around becoming cheaper and more efficient economic instruments for capital. That, taken to an extreme, has pretty corrosive effects on society, particularly young people… “ Vox, Why are millennials burned out? Capitalism

What I’ve realised is that, just as my main character A.L is obsessed with optimisation, conversely my own generation is just as obsessed! Productivity and optimisation has become the be all and end all of life! We seem to be working harder than ever before, and yet, like proverbial hamsters on a wheel, we never seem to reach an end point where we stop running and find satisfaction.

I also wonder if there’s something here about the fear of death in the digital age, as a society. I think it’s why we all secretly want to publish a book (which feels solid, like it may last down the generations). A desire within ourselves to leave a mark on this world – and perhaps a fear that as things have become more digitised, everything feels less permanent.

We have stone tablets with the first writing from thousands of years ago in the British Library, but there is no guarantee that any of our cleverly wrought blog posts/ tweets/digital media will last into the future. Digital in its very essence is fleeting. And yet…we are on this endless treadmill of creating output, as though sheer volume will ensure we are never forgotten…

So how have other generations rated their worth in society? What markets have they used? In general this search lead me to a lot of religious debate about human “intrinsic worth” until I stumbled across the rather controversial George Bernard Shaw and found some fascinating quotes from his writing:

“You must all know half a dozen people at least who are no use in this world, who are more trouble than they are worth. Just put them there and say Sir, or Madam, now will you be kind enough to justify your existence? If you can’t justify your existence, if you’re not pulling your weight in the social boat, if you’re not producing as much as you consume or perhaps a little more, then, clearly, we cannot use the organizations of our society for the purpose of keeping you alive, because your life does not benefit us and it can’t be of very much use to yourself.” ― George Bernard Shaw

George Bernard Shaw’s play The Simpleton of the Unexpected Isles (1930’s) is fairly disturbing, and the ideas behind it, even as a satire are quite controversial. However it had a brilliant quote which applies quite well to our A.L.I.A.S concept, as my closing thought:

“The lives which have no use, no meaning, no purpose, will fade out. You will have to justify your existence or perish.” ― George Bernard Shaw

Pivot(al) Conversations Part 2

One of my LCC Screenwriting lecturer’s once said to me that often as a screenwriter you won’t be clear on what your theme is until several drafts in to a script. However, she also said that you’ll find that no matter how much you think you’re writing a unique story each time, writers often have one or two themes they spend their whole lives writing about, and you can see it crop up in their work time and time again.

It makes sense to me – part of why I think many of us do creative work is about trying to understand ourselves and the human condition. For some artists their themes may always be about abandonment, others about injustice and so forth. I think it goes to the heart of what your own “character wound” is – what is that key event or maybe several events that created that deep dark point in your soul that you’re constantly trying to unravel. The question you shout out to the universe, and keep searching for an answer to which you somehow believe will heal that existential crises in you.

So when Avril asked us the other week – what is the story you want to tell? I was at first a bit of a rabbit caught in the headlights. It wasn’t until after that conversation that I sat down to think about my work and what I tend to write about and what interests me at a subconscious level. Once I’d had time to reflect I found that the kind of themes that crop up time and again in my own work are often about death and loss. I tend to focus on human stories, on the intimate reality of the everyday in the extraordinary, to be fascinated by how others live and love and die. Part of why I became a writer was so I could live 1000 lives (plus time travel)!

For Simon, his interests tend lean more to science fiction and he enjoys in-depth world building, in particular explorations of alien “other” like Ian M. Banks “Culture.” He isn’t scared of the future – but believes that technology and machines will save us from ourselves, as such his art work and stories often tend to lean towards these areas. His fascination with creating 3D fractals is a huge motif that has been driving this project: He loves strange and alien worlds, and the 3D fractal landscapes he’s producing are a wonderful reflection of this.

Copyright: KnowYourMeme

But as we’ve continued to talk through August and September about “what we’re trying to say,” it has felt really difficult at times to hit on what the right mix that reflects both of us – how to marry a “human” story to such an alien visual.

Finally Simon proposed that a good “repeating” theme to explore exactly this juxtaposition between us was maybe “humans being replaced by machines.” It’s a very topical subject, and so we began to think about how we could create a story in VR exploring this.

One of the other ideas we had was tying voices into the Fractal movements. We then started to ask ourselves, could an interaction be that our VR user “speaks” into our fractal world and that’s affects the fractal movement? What would voice/fractal integration look like? What if our Fractal landscape answered back? What would it say?

This also lead to a conversation around the difficulty of having a “silent protagonist” in VR, where you are both extremely intimate and embedded in the world, and yet the primary interaction you would anticipate having (conversation) is still not yet technologically possible.

So then we asked ourselves…could we use VR and an actor to allow for this kind of interaction? Ambitious yes, but something we felt we wanted to explore. After all, this is what this residency is about – exploring the possibilities of storytelling in VR.

And so, to this idea:

A.L.I.A.S – Artificial Lifeform (AL) – Intelligence Articulation System

Theme: Humans replaced by technology, 

Concept: Over centuries humans have been replaced by machines and technology. eventually we will replace humans with digital versions of themselves. But what happens when the digital version is replaced for an upgrade? 

Players find themselves inside the I.A.S, conversing with an Artificial Lifeform (played by a live actor, hidden from view) nicknamed AL, and must answer a series of questions to prove their worth, otherwise you will be deleted for your upgrade. 

Some considerations we’re already mulling – how to ensure this isn’t just a “machines bad/humans good” paradime. To include justifiable arguments from AL as to why humans were replaced in the first place. Some of which will involve offering historical context i.e Gutenberg Printing Press democratised reading by replacing the creation of books from being a hand craft by a few to mass produced by a machine. Another consideration is the environmental impact of humans. Finally this also touches on ideas around phenomenology – a huge research topic to dig into.

I’m already slightly nervous about how to structure/ write a template for what will essentially be a live improvised performance, but I’m curious to have ago and see how I get on…

Art, The Universe and Everything

You don’t make a photograph just with a camera. You bring to the act of photography all the pictures you have seen, the books you have read, the music you have heard, the people you have loved.

– Ansel Adams

I love this quote as it’s such a good reminder that your own art is enriched by everything you experience – which is why you should go out and experience things! Sounds simple enough. However, that can easily slip by the wayside when you’re head down pursuing a creative career.

First of all, it’s very easy to get stuck in a tunnel just working on your craft or on a project. There have legitimately been months where all I’ve done is work the day job and work evenings and weekends on the creative job. I don’t see friends or family. I turn down opportunities to travel or go out to the cinema, shows or galleries. All in the pursuit of my writing career – because it takes up all of my spare time outside paying bills. But this really isn’t brilliant or sustainable long term for your mental and physical health. But just as importantly it can also start to drain your creativity. It’s an old, old adage that life inspires art inspires life. Well, if you don’t get to have any kind of experiences, then pretty soon your inspiration starts to wear a little thin.

Secondly, It’s very easy when you specialise in one medium to just focus down on that. I write predominantly TV scripts, and so I watch a LOT of TV shows. But it’s so important to cross-pollinate your art forms. If you write novels, spend a few hours listening to opera; if you paint go try a VR experience; if you write TV, read books and so forth. We can learn a lot and be inspired by each other – you just never know what may spark that next great idea. Sometimes it may even involve a mash up of different mediums.

This is all to say that sometimes you need someone to tap you on the shoulder and remind you that it’s OK to step away from the drawing board/laptop and go outside – literally just go outside. Even if it’s just for a walk around the park (that’s where I get some of my best ideas at least).

When our mentor Avril gave us homework along the lines of “go see things, go play, come back inspired,” I could absolutely recognise the wisdom in her words.

So that’s what Simon and I have been doing at the advice of Avril Furness, our mentor.

Because we both work day jobs around our creative projects (either 9-5 (Simon) or freelance (me)) and struggle to fit everything in, we decided to split some of our time over the last month between some exhibits as well as going to some together.

Walking through Art

Olafur Eliasson: In Real Life

I was lucky enough to visit the Olafur Eliasson exhibition at Tate Modern twice, and I much preferred my second time, (when I went during the week, without crowds). It was fascinating to see the perfect paper and metal scale models for his installation pieces, and see him using fractals and geometric shapes to great effect.

It was also interesting to see his study of light in so much of his work. Taking sometimes really simple concepts (like a candle reflected in a mirror) and using that to make us reflect on the beauty of this. One of my favourite pieces was aptly titled “Beauty” and consisted of a fine sheet of water in the darkness, lit by light to show the rainbow refraction. I could have stood for hours just observing that.

Impossible to capture with my terrible iPhone was an immersive piece which involved walking through a corridor filled with smoke that was lit by ambient light. As you walked you moved from yellow to blue.

I also enjoyed seeing his research at the end – in fact I was so engrossed with reading his research and watching his films about his process I entirely forgot to take a photo!

Takis

Full disclosure – Takis deserves another visit, as I was only able to pop in briefly to see the exhibit. I will go back soon.

However, what I saw was pretty fascinating, and I enjoyed looking at Takis’s studies into electromagnetic and “movement” in the art pieces.

Walking through History

We were lucky enough to get away for a long weekend to Normandy in August.

I had never been to Normandy before, and the name conjured up vague primary school history lessons around D-day landings and my one viewing of Saving Private Ryan. I was astounded to find Normandy not only actively remembered the D-Day landings but celebrated them, with flyers and posters of the names of officers, and the flags of all the Allies who fought to liberate France. It was humbling and moving to walk through small and large towns and see this displayed everywhere, the lessons of a war never forgotten – when it feels like we in Britain have completely forgotten what was so hard fought-for and at such great cost.

My most striking moment of realisation that I walked in history’s footsteps – and a history that is not far gone – was seeing the rusting, giant carcasses of the Landing Bridges. Rather than “clean them up” they have been left as permanent monuments jutting out of the golden – and now idyllic – sands of Normandy’s beaches. No one walking there can fail to understand that war tore across Europe a mere 70 years ago, and the peace we take for granted today cost thousands of lives.

That visceral feeling, of walking amongst the monuments of history (a history so close I could still touch its recognisably modern remnants) is something I want for those who step into Fractus.

Walking through Mountains

In September we disappeared to the Lake District. Something that I’ve been finding fascinating about Fractals is how they appear everywhere in nature: in leaves, in river tributaries, in the shells of snails. Nature and fractals are bound up together in our perceptions of beauty.

Walking in nature I was reminded that our vision for Fractus was always about creating something breathtaking, visually sumptuous. That the Fractals and our Fractal landscapes should be enchanting and captivating. For me, a lot of that will be down to the use of light and colour to soften what can be perceived as quite brutal shapes – it turns out that 3D fractals are quite a different beast on the eye compared to their 2D counterparts, somehow losing some of that innate wonder.

We need to ensure that we add that back in when it comes to our final experience.