Thesis: Perception

Mapping and Re-focusing

This week, I drew a map of the subjects of interest to my thesis and how they relate to each other. It seems as though the idea of perspective or viewpoint shifting is not what I’m most interested in. I thought back to a starting point of my project: the idea of turning a clever visual comparison into an interactive experience, and the idea of representing the wonder I felt at watching animated credit sequences in the context of a game.

Previously, I was focusing a lot on the idea that the shifts were switching between different spatial relationships, but I wasn’t doing a lot of research on how this was being accomplished.

I want to understand what happens when one is continually experiencing transitions that make use of the fact that the game is a 3D space represented in 2D. (Transitions that may require one to understand the space as 3D, then as 2D, then as 3D again.) How does this change the interpretation of the space and the general experience of the player? What is the difference between achieving this in a 2D medium vs. an interactive 3D one?

Shigeo Fukuda

Recent research on the history of minimal poster art led me to Shigeo Fukuda. He was an artist known for creating posters emphasizing anti-war, using illusion and simplicity (Heller, 1). Fukuda’s posters have a lot of visual strength. They are simple and evocative. They remind me of my fascination of visual minimalism in posters and comics and how they can create a very strong sense of a world. (My next post will be about how story may or may not fit in with the project)

shigeo-fukuda-victoryshigeo-fukuda-1975shigeo-fukuda-amnesty

Left: Fukuda, Shigeo. Victory 1945. 1971. N.p. Winner of the Warsaw Poster Contest
Middle: Fukuda, Shigeo. Exhibition Keio Department Store. 1975.
Right: Cooper, Clay . End gun violence initiative for the AIGA. 2013. N.p.

Visual Perception

Visual perception is a large area of study. There are a few books from the OCAD library I’m reading to get an idea of the key theorists and to find the most relevant sub-areas. From my research so far,  I can guess that these areas are:

  • Perception of 3-dimensional shape
  • Perception of space
  • Visual illusion and cognitive biases
  • Ambiguity vs. specificity of visual information

One of the books is called 3D Shape : Its Unique Place in Visual Perception, written by Pizlo Zygmunt, a professor from Purdue University (Indiana). Zygmunt includes a history in the theoretical understanding of the perception of shape, including key theorists such as Hermann Von Helmholtz.

neckercube

A Necker cube is a good example of a multi-stable image (I’ve written more about multi-stable perception in a previous post), and is often used when talking about visual perception. I found a relevant paper by psychologist Richard Gregory on perceptual illusions. Around page 172, Gregory differentiates between optical/sensory illusions and perceptual illusions, which he explains come from “misinterpretation by the brain of sensory information”. He includes images of the Necker cube by Swiss crystallographer and geographer Louis Albert Necker, who described it in a letter as “a rhomboid (which) reverses in depth, sometimes one face appearing the nearer, sometimes another.” (172).

In addition to this research, I am taking a class on cognitive science, where a lot of these ideas are discussed.

Games

Have people been researching or theorizing about how ideas in visual perception apply to games or interactive 3D experiences?

Lev Manovich’s The Language of New Media has a chapter called Illusion, Narrative, and Interactivity where he compares different types of media and the types of illusion they create. Manovich explains that in 3D game worlds, elements such as the GUI appearing after cut-scenes and  “level of detail” (where objects gain lower poly models based on their proximity to the player character) constantly reveal the machine behind the illusion (206). Manovich explains that “as the user navigates through space, the objects switch back and forth between pale blueprints and fully fleshed out illusions. The immobility of a subject guarantees a complete illusion; the slightest movement destroys it” (206). Questions about suspension of disbelief could connect theories of visual perception with game design. Challenging the way a player perceives the game world, a designer may wonder how their relationship to it differs from other games, and what affect this may have on things like engagement or spatial presence.

Manovich’s book is also a link between games and film. For example, he goes on to compare the experience of illusion in interactive media with that of traditional film, which he claims “aims at all cost to maintain the continuity of the illusion for the duration of the performance” (207).

I’m still looking for more research that could link visual perception ideas with game theory or design ideas.

Works Cited

Gregory, Richard  “Perceptual illusions and brain models” Proc. Royal Society B 171 179-296. Published to web January 3, 2008. Web. January 18, 2016.
Heller, Steven. “Shigeo Fukuda, Graphic Designer, Dies at 76.” The New York Times. The New York Times, 19 Jan. 2009. Web. 18 Jan. 2017.
Manovich, Lev. The language of new media. Cambridge, MA: MIT Press, 2002. Print.
Pizlo, Zygmunt. 3D shape: its unique place in visual perception. Cambridge, MA: MIT Press, 2010. Print.

Thesis: Shrinking

(Updated to include videos)

In the prototype for the Dec. 1st critique, I tested two different kinds of transitions where:

  • It seems as though you are changing (in this case, shrinking) relative to a space
  • It seems as though the environment is changing (in this case, growing).

From here, I can observe how peoples reactions may differ. I’m especially interested in their:

  • Sense of control
  • Orientation/disorientation
  • Spatial understanding/confusion

 

soupstoryboard.png

The first transition was technical improvement on the lake shift. It took a frustratinghallstoryboard.png amount of time (and lessons from friends) to figure out the Unity side of this. The transition still isn’t completely smooth. For example, sometimes you will see the edge of a platform appear beneath you. However, I now have a better understanding of how to affect the camera script, which has unlocked possibilities for other approaches to scene changes.

I also added a script making the hall and sky change colour as the player walks forward, essentially giving them control over the time of day. The next step to strengthening the shift between the school and the mystery book world is sound and shadows.

 

 

hexroomstoryboard
The white sections are invisible meshes Unity uses to know where the player isn’t looking. (Something tricky will always happen behind them)

Creating the second scene also required learning a new tool. Instead of using triggers to see where the player is looking and standing, I’m using meshes to see if the player is not looking at a spot (a spot where something needs to happen). This method is more straightforward and does not rely on a specific screen size. In this scene, the space changes shape and grows larger as the player walks and looks around inside. It uses 7 different room models, positioned in the same spot.

Later, I may add in a small time delay to ensure a gradual transition that prevents changes from happening all at once. Delineating the walls is another issue. I used a light for this demo, which does not achieve the flat shaded look in the other scenes.With the shape of these 7 rooms, it is impossible to follow a dark-light-dark-light pattern of “artificially” lighting the walls. A solution may be to add features such as windows or decorations to each wall.

 

 

 

 

 

 

Thesis: Making Spaces

Screen Shot 2016-11-24 at 1.02.39 AM.png

Here is some furniture I created for the inside of the cracker/raft house. The objects are less informed by research and more informed by playing around and seeing what looks good. Sometimes weird proportions can make the item seem oddly cute and handmade to suit the needs of the person that uses it. I’m finding it really enjoyable to try to make objects that bring up questions and generate story.

Screen Shot 2016-11-24 at 5.37.05 PM.png

It’s easy to get attached to certain pieces that don’t fit in with the other objects. I was conflicted between some more traditional looking pieces and some more modern ones.

Screen Shot 2016-11-24 at 7.57.13 PM.png

I did a lot of research on specific historical interior design styles. Using this research, I’ve been modeling and getting feedback from others to try to create settings that evoke the feeling of being in a distinct era. It’s harder to find references for, say, Europe in the 20s, that show the homes of the less wealthy, but the OCAD library is a good resource for books that at least write about the home-lives of general populations.

Week 10: Prototype

For week 10, I prototyped different perspective shifting techniques and tested a few small technical and aesthetic things.

Screen Shot 2016-11-14 at 6.18.05 PM.png

I played with a new type of transition that captures and records the positional relationship between the player’s camera and the object at which they are looking. It then cuts to a different scene, positions the camera in a different direction (this is the perspective shift) and positions the object using that same relationship. What happens is, the player has the same view the whole time, from the first scene to the next. The object can then be locked to the environment, and the camera control can be returned to the player, so that they realize they are not looking in one direction anymore, but another. The technical problem with this is gaining enough control over the camera with a pre-made Unity fps controller. I have achieved this effect only for the second before something in the pre-written code automatically snaps the camera somewhere else.

This effect can be used for any perspective shift based on direction, but is best for up to down or down to up shifts because of its reliance on views that don’t have anything in them (eg.: of a flat sky) or of two views that look the same (eg.: a reflection of the sky and the sky)

Screen Shot 2016-11-14 at 6.18.32 PM.png

Another thing I prototyped was a space that grew as you turned and walked around inside it. I thought it would be as simple as making a lot of location and view triggers, but I didn’t realize that these triggers would intersect with each other and need to be very carefully positioned. It does give a cool effect of be in a confined space and then suddenly be in an open space. If I were to make this prototype again, I would use triggers not to see where you were looking but where you weren’t looking.

Screen Shot 2016-11-14 at 6.18.45 PM.png

I also tried moving the player through code toward a building to achieve a “far to close” perspective shift, but the effect wasn’t that interesting, and the sudden lack of control over movement would probably be annoying to the player.

Screen Shot 2016-11-14 at 6.17.39 PM.png

I added on to my last prototype (the one I made with the classroom and hallway). Now, as the player enters their room, they will have the opportunity to pour some soup from a flask and make some tea. The food will calm them enough to be distracted by their imagination and their book again.

Screen Shot 2016-11-14 at 6.16.08 PM.png

If the player puts their curser over the kettle, its material switches to a shader with an outline and some text will pop-up, asking the player to engage with it. If they press a key, the text will change. I put a view trigger over the soup and a location trigger in front of it. If both of these triggers are triggered, the player camera’s depth of field will decrease until they have a zoomed-in view of the bowl. The scene will then change to one where the player is now very small, looking out a blimp floating above a round lake with a raft in it. They can walk around the blimp and look out the windows to see the rest of the lake. They can also jump out and see a house take form on the raft as it is viewed from the side rather than the top (where it blends into the raft, looking like a cracker).

This sequence is a perspective shift that goes from close to far and from being big to small.

Screen Shot 2016-11-14 at 6.16.33 PM.png

Finally, I played with limited colour palettes and gradients to try to make beautiful, visually interesting environments. For this purple mountain environment, I added the gradient with texture mapping in Blender. It fades to dark purple that matches the purple of the ground plane that the mountain plane grows out of. That dark purple is also the sky colour I set in Unity, so the environment seems to move out infinitely. I wanted to create the effect of feeling isolated. The main effect I got out of it was motion sickness, because of how slowly it felt to climb the mountains (because their texture changes so gradually, it doesn’t feel like you’re going anywhere).

The mountains look better from far away. I may address this motion sickness issue by making the mountains a non-explorable aspect of the environment (e.g. a view from a tower).

 

Thesis: Week 10 Feedback

My short week 10 presentation slides can be viewed here through Google Drive.

This post is mostly about the very informative feedback I got after the presentation and what it means to my project.

It was clear that I need to build a lot more story and context around these perspective shifts in order to gage whether or not they have meaning. It was expressed that interactions made through text can be amusing but it might be better to show them through animations. I find short text interactions really fun (pretty simple underlying logic, but surprisingly satisfying to interact with),  but they may conflict with the theme of daydreaming. I find animations pretty easy to make, but figuring out how to implement them in my newer version of Unity is one of my next steps.

People found it important to think about the settings and daydreaming quality I’m trying to evoke when considering whether or not to use text or narration. Text may conflict with the idea of dreaming, as it’s not typically something you can do when you’re dreaming. Voice-over can play to the idea of daydreaming and thinking to oneself. It can also be a way of adding the personal element of your own voice. My response: I find voice-over narration detracts from interactive environments, and that I skip over long blocks of text, but I know people who really appreciate things such as interactive text adventures. I think that the ways people are comfortable interacting with stories really varies and may be dependant on the story telling forms they grew up with.

Wes Anderson was mentioned as a reference for his use of still shots and timing. These aspects can set the scene and make the viewer anticipate what will happen next. I appreciate Wes Anderson’s creation of stylized but consistent environments, partly through the use of specific colour palettes. I’d also like to take inspiration from film makers such as him to understand how scenes can be initially presented to “set the scene”.

Alice in Wonderland was also mentioned because of its shifts between the main characters point of view and omniscient narration. Narrative shifting may be considered as a perspective shift; eg.: shifting between exploration (player POV) and narration in the game. But really consider what parts of the story should be exploration vs narrated. Alice in Wonderland also plays with scale, and there are numerous noticeable transitions to look at, such as falling backward and getting smaller. This can be tied into the idea of playing either as a the main character at school or as the meta-fictional character that this character embodies. The playing through the first character’s view could reveal story in a more exploratory way and playing through the second character’s view could have more elements/aesthetics taken from novels, e.g. the traditional story telling method of a 3rd person omniscient narrator.

Another feedback point: The narration/storytelling style should probably match (so no text unless it is subtle) the subtlety of the environment that has already been made.

My response: Subtle elements can be overshadowed by non-subtle elements. If you have all subtle elements and no overt story telling devices that people are used to in games and film, then the player gets into a more critical or thoughtful or searching mode of analyzation for meaning. You can have a scene with minimal objects in it that has meaning, but maybe you need to give the player time and focus to question what these objects mean.

I also asked for tips on the technical side of making transitions. People suggested locking the camera until the new scene loads. It was also suggested to use Unity to see if the player is looking in the right place by finding the dot product of the camera’s normal vector and the target object’s normal vector. This can be used for both the lake transitions and in situations where things happen outside of the player’s view.  There is a function in unity called dot product.

I’ve been using raycasting and colliders to do this sort of thing already, but the method that was mentioned may be more accurate, and wouldn’t require the positioning of colliders on objects. I will write more on my latest technical adventures with perspective shifts in a following post.

Feedback thanks to:

Parth Soni

Dani Jones

Chris Jadoo

Muddassir Iqbal

Kevin Back

Rosalind Chapman

Emma Westecott

Joshua Martin Salvacion Salvador

Anth Rodi

And also thanks to everyone else, who listened and said small things that weren’t caught on record.

 

Thesis: Academic Research

I devoted a lot of my attention in the past two weeks to finding and documenting a lot of academic sources, looking specifically at:

Game Studies:

  • Perspective and space
  • Relating emotions, metaphors and mechanics

Film Studies:

  • Cuts and sequences
  • Disorientation and Subjectivity

There’s a lot written about subjectivity in game and film studies, and it’s important to define it before going forward. Usually in game studies, using subjectivity to refer to the unique experience of the player. I’m more interested in how the player is given a view of a world that is filtered through the mind of a character, and especially how abstractions of a game environment can communicate ideas about a character’s interpretation of what’s around them. (Relating to bias, unreliable narration and emotional perspective.) I’m focusing my written thesis more on visual/spatial perspective, but my project is about creating meaning through spatial perspective shifts, using subjectivity to create this meaning.

I have spun myself into a sticky web of concepts, but a surprising connection was a group of sources that are about both subjectivity in film and spatial disorientation. These sources are a bit more difficult to read about, because they deal with film theory.

Perspective needed to be a more important part of my project, and I’ve been thinking of different ideas that perspective can suggest in a story.

Close to far – the transition of time, forgetting

Far to close – inspection, understanding

Inside to outside – isolation or confinement to exclusion or freedom

Outside to inside – curiosity, being wrapped in something

One side to another side of a reflexion/photograph/painting -imagination, memory

Looking up to to looking down – hope to despair, a power dynamic shift between two people, the process of growing up, flying upward or moving beyond something

As was pointed out after my last presentation, I’ve been working on a lot of things in my project that relate to Russian director and screenwriter Vselvolod Pudovkin’s 5 methods of editing. These methods were really interesting to read about because they’re kind of common or at least relatable to similar concepts in art, music and literature, and I like connecting my somewhat diverse interests in different established rhetorics/forms of expression.

I read about Pudovkin’s ideas on editing and montage in Film Technique and Film Acting: The Cinema Writings of V.I. Pudovkin, in the chapter on Relational Editing. The main methods of editing (with the intention of creating impression) he listed are:

  • Contrast
  • Parallelism
  • Symbolism
  • Simultaneity
  • Leit-motif

Contrast

I’m using contrast by comparing the situations and reactions of two different characters. I also like to switch between drastically different scenes sometimes to create a sense of displacement and escape.

Parallelism

Pudovkin writes that parallelism this is similar to contrast. I see parallelism in the game as the player switches back and forth between the worlds of these two characters, and both of them are facing isolation and dissatisfaction in their own ways. The transitions that rely on 2 similar looking things may also be considered parallelistic.

Symbolism

Symbolism is another way of creating another level of meaning that can draw connections between drastically different environments. This game has the potential to feel really disjointed and disorientating, and I’m already interested overlaying different themes/patterns/emotional arcs into stories, so a lot of these methods are very relevant.

Simultaneity

Two things are happening at once! Pudovkin writes that this is over-used in the ends of movies to create a sense of tension. It may not be something I get to by the time I’ve finished my thesis, but I had imagined a series of rapid scene cuts where the player keeps looking around and finding themselves somewhere new, as the focal character is overwhelmed/confused/distressed. I’d like to test out what kind of effects the frequency of transitions can have on a player- whether frequent transitions make them nauseated, upset or confused or if this can be used to put forward a point while still keeping the player engaged. I don’t want to make the player stressed out. I also don’t want to rely on frequent transitions to create a sense of urgency- if I continue to make a linear narrative, each transition should say something about the character’s state in the story and be supported by other story telling methods and by context. (I have similar attitudes towards to the use of music)

Leit-motif

This is my favourite of Pudovkin’s methods. In music, leitmotifs are short musical phrases associated with certain people, places or ideas. Re-occuring themes with varied tempos, instruments and keys kind of blow my mind. Themes in film can be used as non-conventional methods of conveying meaning. In certain films and novels, you can follow a theme through the piece and think about everything that’s happening in relation to that theme, instead of thinking about the story in the traditional sense: according the the successes and failures of the main character in fulfilling their main goal.

The intentional and thoughtful creation of themes can help create a story that is well structured. I don’t think that’s the only key to a good story, but it adds strength /underlying logic.

Works Cited

Pudovkin, Vsevolod Illarionovich, Ivor Goldsmid Samuel Montagu, and Lewis Jacobs. Film Technique and Film Acting: The Cinema Writings of V.I. Pudovkin. New York: Bonanza, 1949. Print.

Thesis: Prototype Lessons

I’ve returned from the fall break after doing a lot of academic research and work on my prototype.

View my latest progress presentation

Gradual Transition

screen-shot-2016-10-14-at-1-05-03-pm
Garbage can turning into an urn with a dead plant in it… how odd

I tried out gradual transitioning in my prototype, which wasn’t overly successful. It ended up looking like a hallway with a bunch of random items in it rather than a hallway where you realize 2/3rds through that you’re somewhere dramatically different. To make it clear that the character is moving between reality (the school house mid-day) and projecting themselves into a work of fiction (a novel they are reading, where the protagonist is escaping an abandoned castle in the middle of the night), here are several solutions:

-using a longer hallway

-having only 2 distinct architectural styles (this requires a lot more thought and research than I had anticipated, but this type of research can make a story world more tangible and consistent and interesting)

-showing the view from the windows at the start and end of the transition

-lighting and scale shifts

-shadows with an angle change, becoming more dramatic

-periodically changing the scene behind you to match the scene in front of you as you move forward

-a character that addresses you as the fictional character, or some other sign of the fictional character that the main character is projecting themselves into

One of the things noted during my presentation was that movement attracts attention. Having silhouettes moving past the windows may help guide the player forward, but could also distract the player from going to the end of the hallway.

Links between scenes, foreshadowing, visual storytelling

screen-shot-2016-10-14-at-12-26-53-pmscreen-shot-2016-10-18-at-4-23-35-pm

Environmental storytelling takes a lot of thought! A lot of things that are really easy to describe in words are really hard to describe through environments. Certain stories are easier to tell through spaces than others, and I’ve chosen one that involves a lot of inner character thought and somewhat mundane/low stakes events. One of the things I wanted to do through my prototype was get the player to ask what the new setting that they were walking into was, and then have that question answered by showing them a book cover. Then they would get the idea that the main character was pretending to be in one of the fictional settings of the novel they had brought to school.

Further avenues into storytelling:

-testing the game with some text or voice-over narration

-keys with a room number, that links to a number on the mail sorter, that links to the number on their room door

-un-packed luggage

-musical themes

Tech

screen-shot-2016-10-17-at-8-47-56-pm

The transition I used at the end of my prototype worked by changing the scene behind the player. This effect was explored in a previous tech demo, but I hadn’t tried it before with such a large scene. Grouping objects together in Unity under an empty parent, naming things consistently and using Blender-linked prefabs (3D model assets that you can still make changes to in Blender) was essential in keeping code smaller, staying organized and making later changes easier. These things can be a hassle when you’re working quickly, but are really worth the effort.

I now know how to implement a lot more elements into Unity, including location based sound and video play-back. I also taught myself a little about animation in Adobe Animate, which can be used in a future scene (eg.: showing landscape passing by train window)