How do you direct VR?
Hello, friend, and welcome back to episode number 19 of my 31 Day Challenge, where I answer your questions every single day about creating a career or business that you love. In today's episode I'm going to be speaking about something that doesn't really necessarily relate to career or business, but it's a question I get asked so often I thought I might as well bash one out in podcast form.
The question is: How do you direct a VR film, A.K.A a 360 video or experience? I'm going to be diving into that!
This is a full transcript of the podcast. Instead you can listen here:
If you've got a question, then please ask away. You can ask me on Instagram or Twitter, @alexmakesVR is my handle, and you can ask a longer question on email at firstname.lastname@example.org. Every single day when I put these episodes live, I send out a newsletter to remind you, and I also give you a recap of what I talked about in that episode. If you want to be reminded then sign up to the newsletter at alexmakesvr.com.
So how do you direct a piece of VR? Disclaimer right at the top; this is my personal opinion and every director you talk to is going to have different opinions. One of the things about VR is it's so new that even the language we use to describe the processes behind it keeps changing, so they might even have different terminology to communicate with you. Do bear in mind these are the things that work well for me when I direct experiences, but also things I pick up on as an audience member of VR when it’s done very well.
The three key areas that I use to direct a VR piece are choreography, editing and sound design. Those three are the big ones. Sound design is a little bit of a luxury because you have to be working on a production with enough budget to be working with a spatial audio designer, so I'm not going to focus as much on sound. Some of you listening might want me to- I do think sound is ultimately one of the biggest ways for you to direct attention in VR, it's so powerful. There was a piece actually this year in Tribeca. It's a piece where, for maybe the first moments of the piece, you’re in a one shot where you’re following this characters shadow. It was a design piece for a game, but the filmmaker had used a combination of shadows and sound design to really cleverly disorientate the audience, and it was absolutely amazing. So the use of sound can be really masterful - but to some extent, I have to be aware most people who work on their own solo projects with smaller budgets might not have the luxury of working with spatial audio. If you can, I would highly recommend it. The number one piece of advice I’d give you as a VR Director is to think about the way you naturally engage with things in the world. How does someone or something get your attention? Sound is a huge part of it, and when you start to really take note of that, of when you’re walking down the street and what sounds draw your attention from right to left from right in front of you. That’s how positional sound can focus your attention.
The other important thing is choreography. When it comes to directing a piece, if I want someone to look left and I don’t have a sound cue to do that, I will physically want to choreograph it so that the person or object in the VR moves over to the left side of the sphere (frame). I might have them walk off centre to make you follow them and discover a new detail in the frame. That’s one really easy way of doing it. You can not only direct attention with choreography but incite curiosity.
For example, if you're having a conversation with someone in real life, and the person you're looking at is making eye contact with you but then all of a sudden they look to the side, and your instinct is to look at what they're looking at. You want to check what they're looking at. It must be like some kind of hardwired, cave person instinct of taking cues on potential dangers nearby or something! But that's what you want from a VR experience. You're trying to instigate curiosity in the audience, but you want to direct them to certain points of interest, to the things you want them to be curious about, if that makes sense. I can't tell you how many times I've watched a VR piece and the direction is just all over the place, because the person has this attitude of wanting to capitalise on this entire 360 degree frame. But as humans, it’s not a comfortable experience to feel exposed in an open space like that, like when you’re at an event, you don’t really want to stand in the middle of the room where there’s stuff going on around us, it can get pretty stressful. We naturally gravitate so that our backs are protected as humans. It could be an absolute unintentional mistake by the artist to do that to someone, but make sure if you’re going to you don’t make the unintentional mistake of overwhelming the audience. It’s common for people who are new to VR to just put everything, everywhere and have loads going on, but often audience feedback is that they didn’t get massive FOMO because they had no idea where they were supposed to be looking, and that is the opposite of good direction.
So you can also use choreography outside of actual physical movement of a character or an object to pull attention around, but also to direct their eye line. Maybe they make reference to or look to a particular part of the location, and that then draws the person's attention. If you wanted to make sure someone doesn't miss something, you have a person call out. For example, if you were doing more of a corporate type experience, let's say an induction video for new recruits, and you've got a presenter, and you want that person to look left, but you don't want your presenter to walk over there. Maybe you could literally have them say, ‘Look to your left now,’ and you'll see the fire exit or you'll see X, Y, Z. So you could play with that in your scripting, there's lots of different ways within choreography that you can do that.
The second big one is editing. This is my absolute personal opinion, but I hate it when people edit within the same scene in a VR piece, because it’s as if you're treating me as if I'm a camera. That cut made no sense, it was jarring and annoying, and now I’m taken out of the piece. There's something about allowing your user to be fully in that space and have the action move around them which adds to the experience. If you're in a static 360 piece, if you've got more of a room scale piece where they can move around the experience then you can have them do that like if that is what you want to go for. But try not to cut and have them in a different place in the same room in the same scene.
People try to do the equivalent of a wide shot and a close up in 360 by moving the camera; that doesn't work in my opinion. I'm very happy to have a discussion with anyone that disagrees with that, but I've, I've shown VR to enough audiences to know generally the audiences do not like that. It brings them out of experience, and it makes them more likely to say, ‘I'm not really sure why that was a VR piece because it could have very easily been a normal film.’
So how can you use editing to your benefit? There is this principle- if you've not heard this theory before, then google Jessica Brillhart, she was the principal VR filmmaker for Google for a long time. She wrote this theory about editing in 360 and described the process as jumping between worlds. So if you imagine you’re in one scene and facing a particular way, and you roughly know where that person is going to be looking because you've intentionally directed their attention there. When you cut and edit to that next scene, you’re jumping worlds, and so what you want to do is line up a point of interest and there is a real intention to change and not just move the camera position.
I’m not suggesting all VR pieces should be like one shot or done in one location, because that’s not what I’m suggesting.
So, for example, while you’re listening to this, you’re looking straight ahead. Imagine there’s a cup of tea in front of you (I hope there is because there’s nothing more delightful than a podcast, a cup of tea and a biscuit on a Sunday). Cut scene. Now you’re in a diner with a pizza in front of you. Roughly speaking, I haven’t put any other action in this scene, but I know there’s a good chance you’re going to be looking forward to your cup of tea and biscuit because that’s the point of interest. When I cut to the diner, the pizza will be in the same place as the tea, because I’ve got a really good shot lined up there and that's the equivalent of a match cut. That’s a pretty educated guess that’ll be where you’re looking, and it’s where I want your attention now. Another example is that I want you to be paying attention to your best friend in the first scene with the cup of tea and biscuit between you, but to the left hand side of you is another friend sitting there and chatting to you about Alex Ruhl’s amazing podcast (!) and they’re looking at you, and ask, “are you paying attention to me?” Now I can guess hypothetically that you're either going to be looking at the cup of tea, or looking at your friend that's just asked you if you're paying attention to them. There’s a high probability that you're looking at them. So when I cut to the next scene, you answer, “yeah, I'm paying attention,” and we cut there. Maybe I have the waitress, waiter or the manager of the diner, but I'm going to line them up exactly where your friend best friend just was. In doing so, when I've jumped to that next world, when I've edited to move the story on, I've managed to line up your attention. And then maybe, because I'm guessing that most of you will have been looking at the best friends, you’ll be looking at that waitress, and I'm going to have them walk away once they’ve asked you if everything's alright with your meal, that kind of thing. I'm going to have them walk off towards your right. So then if you weren't looking at the pizza, now, you've just discovered the pizza, because you've followed them as they've walked off. And that choreography and editing has allowed you to discover the fact that you're now in a diner with a pizza in front of you. That's obviously a terrible example off the top of my head. But I'm hoping that whilst you're listening you can imagine that playing out as I've described it, in a very clumsy way!
Now, another thing to think about, and this applies to both choreography and editing, is this idea that we’re trying to blend film, theatre and game. I like to add a little splash of dreaming. When you dream it feels real, it feels like you're in control, but then you wake up and it wasn't. But dreams are in first person point of view. So take note of all of these things; when you're watching TV; when you're like watching movies, theatre. But when you're dreaming and you wake up, think about what it looked like, what the framing almost was. The equivalent of a close-up or a wide shot in VR- this totally applies purely to 360. Because if you're doing a VR piece at room scale, or the user has more freedom to walk around, then they can dictate their own distance to things- is the personal space and the space you give between characters, or objects in a frame.
Let's go back to our scene where we are looking at our cup of tea, or our best friend who was sat to the left of us. If we want that scene to have the feel of a wide shot, and we want it to see everything in the frame as an establishing kind of shot, maybe we have the cup of tea and the best friend far enough away that you can see them, and the best friend to your left far enough so that it doesn't feel very intimate. It's just casual. It's a setup shot where you’re establishing where you are, who you are, who this person is, why you've got a cup of tea and a biscuit. Now let's say we want to totally change the tone of that scene and we want your best friends to come over to have a really serious chat with you, and things are not great. They've made you a cup of tea and a biscuit because you're actually really upset. So I'm going to have the best friend way closer to you, or maybe I want them slightly out of shot so that you're either looking at them or the cup of tea and the biscuit, but you're getting that personal space. You've got less personal space as they are much closer to you, so you get the idea that this person is quite close to you, you get the feeling instantly that there is an intimacy there. The way the actor performs in that scene will dictate that as well, but you have to remember the camera is the person's head when they've got a headset on, and that would give you a good idea of how that person is going to feel when they put the headset on and experience your piece. So the way you play with space, especially if there's multiple people or specific objects in a scene, you can really tell a lot about what you're trying to do based on how far or close that object or person is to the camera.
Because we are completely developing this new storytelling language, instead of thinking about scenes as wide shot or close-up, I think space defines a lot of the cinematic language of VR.
If that’s not a sound bite I don’t know what is! Instead of the close up and the wide shot and everything, we use the space. Whether that’s the actual physical location, the feeling of being outside, out and open and free or being in a small room crowded with lots of things and feeling quite claustrophobic, so much about VR storytelling and directing is about feeling. One of the superpowers of VR is the fact you can give someone a sense of atmosphere without even really doing anything. Often people will say that locations and space within VR become their own characters and for a long time I didn't really agree with that, but now I haven't been in it for years, I tend to agree. You want your user or audience member experiencing it the way that you want them to feel it, whether that’s about a scene, character or action going on.
So much of that will be framed instantly by the location that you've chosen, or by the lighting of that location. We can get into kind of the weeds of using lighting design to direct attention, which is very true, but I'm trying to boil it down to the key things that I would be thinking about, especially if I was new to VR storytelling or directing.
So let’s recap. Choreography is your number one friend; it's the easiest way to direct attention, it will have the biggest impact and it's the least technical because you can set up the camera and physically move the props and people in the scene to create the direction you want without having to do anything nice and techy.
Your second friend is editing; getting through that story, thinking about how you want your users attention to change throughout time. This will change if you're doing interactive pieces- if you've not seen The Line, which is a piece I've talked about in a previous episode. It won at Venice Film Festival, the VR Selection, last year or the year before, and that is a beautiful masterclass in interactive VR storytelling, it's not like a branch narrative. It's not like you can do different things and that will change the story, but the way you interact with the scene propels the story on and your movements and actions you take definitely change throughout the story and become their own story arc. There's a piece in The Line where you've been trained and are helping this character that is like a godlike figure. I won't spoil it too much in case you haven't seen it, but essentially, it's set upon like, a vintage kind of table top.
Imagine a train set, but it's like got all these lights and characters that go along the same line every single day. It’s a big metaphor for life and is absolutely stunning for lots of different reasons.
But what I'm trying to say is it's all very interactive, but it is still linear, you're still pushing the narrative forward. It's not like you're changing the narrative. But there's definitely moments in it where you can't intervene but you want to, or you do intervene, and it backfires a bit. Your actions in themselves become a plotline, and I found it fascinating that it worked on all these different levels. It's beautifully designed, it's very simple, but it's so masterful on so many different levels. And that is because ultimately, the director has considered the direction from the point of view of the audience member; what do I want my audience to feel?; where do I want their attention to be?; how do I want them to feel for the characters, for themselves? They hit on those emotionally impactful notes at the right time. So much of that is to do with how they use the space and how they use choreography, but there's no editing in that piece. Because you are editing as you're walking around different parts of the tabletop, because it's a room scale experience.
The sound design is really beautiful but they're not using sounds massively to draw your attention. It's just a beautiful, beautiful kind of example, of using space and your audience's actions and the way that they interact with the scene as a way of guiding the story. So I do think it's really important. And as a director of VR, you have to assume that to a certain extent you are relinquishing control, because your audience can theoretically look anywhere, but a good director in VR will have faith and will have done enough work on things like the choreography, editing and the sound (mainly the first two) to know you will most likely be looking the way that they want you to, and that's a sign of a good director in VR.
I use this analogy quite a lot when I do talks on VR storytelling, so I would think about this. I think you should think about studying, or at least giving some thought to magicians, and the fact magicians give you the illusion of free will, but it's all misdirection. They want you to be looking at a specific hand, because over on the other hand they're doing something sneaky that sets up for the next shot, and I think there's so much to be learned from that and can be applied to VR. It's the art of misdirection. You think you can look anywhere, but you're not going to because I've thought through every single thing in this scene, and every way you're looking is by design. That is the ultimate holy grail of directing in VR.
Hopefully that gives you a good overview of ways to think about directing a VR piece going forward. If you've got any follow up, I would love to hear from you. I've had some of the most incredible messages over the last couple of days from you guys. And honestly, this is why I want to do this. This is why I want to keep putting out this information. This is like one of the reasons why I'm so psyched to be doing this challenge, because when I hear from you guys it just makes my day, it makes my year and it makes it all worth it.
If you've got follow up, then please message me on social media. You can get me on Instagram and Twitter @alexmakesVR, or you can send a slightly longer question, if you've got a question for future episodes, to alexmakesVR@gmail.com. If you want to be reminded every single day when these episodes go live with a bit of a recap of what I'm talking about in the episode, you can sign up at Alexmakesvr.com to the newsletter. I hear that it's actually quite important, regardless of what platform you're listening to me on, to please, please, please take a second to make sure that you are subscribed or following. It really, really helps apparently, and also will be another way of knowing when a new episode comes out.
So that would mean the world to me if you would do that, if you're getting value from this podcast. If you can leave a review, if that's an option on the platform you're listening on, that would mean the world as well. No pressure, though, because I realise that it takes a bit of time and they've not made it the most easiest process in the world, but if you're getting value from this, and you fancied saying a couple of nice things then maybe pop it in a review! But again, no pressure.
That's it from me today. Enjoy your Sunday, I hope you have a delightful day! If you're listening to this anytime in the future, I hope you're having a wonderful day too. I hope that the world is slowly reassembling itself from the chaos that it has been in for the last few months.
That's all I have to say. Speak to you tomorrow!
Listen to the Alex Makes VR podcast here
Subscribe to the newsletter here
Follow Alex on Instagram here