Thesis: Reflections

Because I presented my thesis to the class this week, I thought I would use this post to reflect on my experiences and write about future steps.

I am happy that I’ve learnt a lot through research and experimentation and talks with professors. I am also happy that I worked hard and made something that generates interesting questions. If I were to do this thesis again, I would change my process. The original process evolved as I was learning what was more important to work on, what my project was about and what my project could be about. I was also learning about my work habits and my interests. I got the chance to explore fields including perceptual psychology, narratology, film and animation. I also found that I enjoy reading academic papers.

In my first thesis course, I debated whether to do a thesis on surrealism or on creativity in games. I was worried I wouldn’t be as attached to the former idea. It was not as familiar to me. However, I think I chose it because I knew there was something really interesting about that first transition I tried out. Through these past three semesters, I discovered a tornado of fascinating interrelated subjects that connected to this transition.

Either project would have been an exploration of a more experimental, unconventional type of game. These types of explorations are really difficult, because they involve a lot of unknowns. The process of making this project was not constantly gratifying. It took a long time to make scenes that were somewhat close to what I had imagined in my head. I also had to focus on a lot of technical issues before I could implement visual details and I kept getting story ideas that wouldn’t work in the game. It often felt like I was holding my breath.

One of my favourite moments was watching classmates play, and walk into objects because they assumed something unordinary would happen. These people were using expectations built from hearing me talk about my project many times. I really want to know if I can build up these expectations in the game itself. I want to encourage this weird form of play. I want to create a game where players take on a unique mode of interpreting space, where players anticipate some of the abstraction that takes place and factor this into their own actions, so that an illogical space has an alternate kind of logic.

I’m going to keep working on this project at least until the graduate exhibition at the start of May. I want to polish the experience, making it more intuitive to navigate and adding more aesthetic consideration. I also have a lot of ideas I didn’t have time to explore. One of them is to blur figure-ground relationships. Another has to do with kaleidoscopes and 3D space.

Thesis: Animation

After playtesting, I decided my game needed something at the beginning to motivate people to look around. It needed something that the players’ observations could build onto.

I figured a good way to introduce the story might be an animated credit sequence. It was surprisingly intuitive to move from 3D animation to 2.5D animation in Blender. The process of putting together an animated sequence and figuring out how to move the camera has been challenging, but exciting.

One of the challenges is managing a project with really small models and really large models.  I wanted the animated sequence to have a dramatic zoom-out effect, similar to one of the playable scenes in the game. Unfortunately, once models in Blender get large enough, there are navigation and clipping plane issues.

Managing a lot of parts in general is another challenge, which I have also had to deal with in Unity. If you have an animation or a game with cuts or levels, your objects are divided into more manageable chunks. (Chunks also make sense for limited processing, at least for games.) Now say you want to gradually transition between a kitchen and then a forest and then a beach. You need all of the kitchen furniture, all of the trees and all of the beach stuff in the same scene. This can get annoying.

Animation is exciting because I’m learning a set of techniques that are allowing me to express some of the imagery I’ve imagined more easily. There are fewer barriers between what I picture in my head and what I can materialize in animation, compared to what I can materialize in game form. In this way, it is more immediately gratifying. I sometimes feel like I’m translating my ideas more directly.

Starting the game with an animated sequence feels fitting, because animated credit sequences were a significant point of inspiration for my project. I often wondered what it would be like if someone could explore a game version of these sequences.

I spend a lot of time looking at 2D art in general and wondering if it would be possible for it to exist in (simulated) 3D. There’s a book called “Dream Worlds” where animator Hans Bacher writes some interesting stuff about applying traditional art styles to computer generated animation. Describing his work on the project Fraidy Cat, he states:

“What I tried to do was a combination of 2-D and 3-D. Everything was supposed to be built 3-dimensionally, but would look flat in the end. Only a moving camera would reveal that there was an optical illusion.” (166) (see pictures)

It seems like the freedom of camera in most 3D exploratory games almost always gives away efforts to hide underlying geometric structures, the sharp edges, the planes. I do remember seeing some Tilt Brush worlds that really felt like convincing manifestations of paintings. I don’t know how feasible it would ever be for that kind of tech to be used in games, even if it is vector based.

Work Cited

Bacher, Hans P. Dream worlds: production design in animation. Burlington, MA: Focal Press, 2008. Print.

Thesis: Making Meaning in Games and Film

I wrote another draft of my thesis, which took a very long time. I was not happy with it when I finished, but it was important that finish a complete thing. Before it was a collection of bits and pieces. Writing it out completely was a not fun but it made me aware of logical holes and places for improvement. I found some unknown-unknowns. For example, I was writing about the possibility of making meaning, when I hadn’t really researched how games made meaning or how this compared to film.

(It really easy to discover a new area of research such as “spatial continuity” or “suspension of disbelief” and think it is the one thing my paper has been missing all this time. But this process never ends. There are so many relevant topics.)

Aesthetic Illusion

Finding the terms aesthetic illusion and naturalization led me to the field of narratology. Jocelyn Cammack’s essay “Aesthetic Illusion and the Breaking of Illusion in Ambiguous Film Sequences”, is particularly relevant because she writes about how films have created unique perceptual reactions, including Man with the Movie Camera (Dziga Vertov, 1929), where “visual information momentarily conflicts with understanding and a double-take reaction is evoked” (306) or Code Inconnu (Michael Haneke, 2000) where aesthetic illusion is split into two interpretations (303). These effects are similar to what I am interested in creating in my game. The latter effect can be related to a multi-stable image, which I’ve written about previously.

Cammack writes: “the work of filmmakers who experiment with these perceptual concerns often demands a different kind of engagement in the process of viewing”, where the viewer wants immersion and enjoys being overtly manipulated (312). Could this type of engagement be created in a game?

A big difference between how information is interpreted in a game rather than in a film is that this information is used not only to generate understanding but to understand how one can act within a system. Looking at an Escher lithograph is disorientating in a fun way, but maybe this is because you’d never actually have to find your way inside the piece. Cynthia Freeland writes in “Continuity, narrative, and cross-modal cuing of attention” that the main way visual experiences of film are different from real-life ones is that “vision serves human agency” (34). Freeland then writes about how separate parts of the brain are used to process “spatiotemporal information” and information on how an object can be used (35) (although the latter is more related to bodily movement). Freeland suggests that this “agentive vision” may be more important to video games than it is to most films (37-8).

Christian Wessely’s “Columns of Figures as Sources of Aesthetic Illusion” also points out that the experience of games differ from “traditional illusionist media” because of players’ interactive involvement (341). Games are not only a means of expression but tools where players constantly re-frame their perspectives as interactivity changes, in order to be more efficient users (Wessely, 343-4). This constant shifting brings attention to the fact of the game (Wessely, 343-4), whereas one might forget that they were watching a film. However, he also writes that being aware of the relationship of oneself and the computer isn’t detrimental to aesthetic illusion, because the medium  of the computer is not seen as separate to the experience of illusion (356). This relates to Lev Manovich’s The Language of New Media, where he writes about how games are constantly breaking illusion by making their underlying system overt.

Because I am making a game that defies certain real-world physical rules, I am wondering to what degree the convention of mimicking these rules is arbitrary. How much do players rely on these rules in order to comprehend and willingly engage with a game? Wessely writes that to create immersion, games create reliability through an “analogous experience of direction and movement” as well as other physical experiences (342). He points out that while it isn’t necessary to create a completely analogous game environment, “games with such features form a commercial insignificant minority because consumers obviously favour a certain familiarity with the environment.” (343). Although I agree that some real-world or convention-based familiarity is important to easily understanding how one can engage with and understand the meaning of a space, I do not think commercial availability of certain types of games is sufficient to understanding what consumers or people in general enjoy. Wessely’s points do bring up a question I have been facing during game design: what is the right balance of familiarity and surprise? Can a game compensate for, say, reversing gravity with extra attention to realistic sense of movement in order to remain engaging or comprehensible?

Convention

Besides differences in how visual information in film and games is perceived, I was also curious about why certain representational conventions exist in games. How arbitrary are they?

Daniel Schäbler writes in “Unnatural Games” that “modern, graphically advanced games strive to simulate cinematographic realism, not mimetic extrafictional realism”, quoting theorist Stephen Schwingeler to suggest that it is because there is still a credibility to photographic realism.

This suggests that games use representational conventions from film to communicate more easily. But what about the structure of the represented content? In”Game Design as Narrative Architecture”, Henry Jenkins writes that games suffer from an overly literal application of film theory, and more specifically, “traditional narrative structure” (118-9). Jenkins explains that games tell stories best through space, and that “choices about the design and organization of game spaces have narratological consequences” (130). He also writes that games often use narrative conventions from literary genres such as science fiction because world-building has more priority (123).

Continuity

Cynthia Freeland’s paper not only suggests a way in which the perception of game information may differ from that of film information, but also explains how films use continuity editing to create meaning. Some of my previous research focused on editing conventions. I was wondering if these conventions could be used to create meaning in my game. The transitions in my project often create a visual continuity (one scene blending smoothly into the next) while breaking logical spatial continuity (transporting the player to another place). Typically, within a scene, a film breaks visual continuity while adhering to certain conventions to create the sense of spatial continuity. However, a match-cut can link two spatially disparate scenes in a more visually continuous way.

Freeland writes that editing is about allowing a viewer to “shift their attention to the audiovisual details currently relevant to them and the narrative” (38). These details are determined by a process of guessing and expectation for the ultimate purpose of  what Noël Carroll calls “narrative closure” (Freeland, 38). Basically, cuts can be unseen because they respond to narrative-related expectations. It is interesting to think about how this relates to games, where the player typically manipulates their own view in order to answer questions. Could a game transition be less jarring or disorientating if it creates “narrative closure”? How do you create “narrative closure” in games where the designer has less control over the presentation of the story?

Final Remarks

Maybe instead of asking whether I can create meaning in a certain way (because the answer will almost always be “yeah, sure”) I should ask to what degree I can create a comprehensible or engaging experience.

Works Cited

Cammack, Jocelyn. “Aesthetic Illusion and the Breaking of Illusion in Ambiguous Film Sequences.” Immersion and Distance. Amsterdam, NL: Editions Rodopi, 2013. 295-312.ProQuest ebrary. Web. 12 Mar. 2017.

Freeland, Cynthia. “Continuity, narrative, and cross-modal cuing of attention.” Projections:The Journal for Movies and Mind, vol. 6, no. 1, 2012, p. 34+. Academic OneFile. Web. 11 Mar. 2017.

Jenkins, Henry. “Game Design As Narrative Architecture.” First Person: New Media as Story, Performance, and Game. Cambridge, MA: MIT Press, 2004. 118-30. Electronic Book Review. Web. 12 Mar. 2017.

Schäbler, Daniel. “Unnatural Games? Innovation and Generification of Natural and Unnatural Visual Effects in Dead Space and Alien: Isolation.” Storyworlds: A Journal of Narrative Studies 7.1 (2015): 21-38. Project Muse. Web. 12 Mar. 2017.

Wessely, Christian. “Columns of Figures as Sources of Aesthetic Illusion: Browser-Based Multiplayer Online Games” Immersion and Distance. Amsterdam, NL: Editions Rodopi, 2013. 339-362.ProQuest ebrary. Web. 12 Mar. 2017.

Thesis: Structure and Story

I have been working intensely on my prototype for the past couple of weeks leading up to the midterm presentation.  I thought I would get especially helpful information out of the creation process and the playtesting feedback if I created a longer experience. I had a lot of questions related to how things like expectation and meaning were going to be built up over multiple scenes. 

The creation process was really helpful and even if time constraints made me settle on certain ideas, it was important that I materialize them rather than spend too much time trying to think of better ones. In addition to giving me a better understanding of the project, the creation process also helped me to generate new and possibly more feasible ideas. Moving forward, I plan to jump more quickly into prototyping. As I’ve learnt, this is especially important for a project with a lot of weirder, frustratingly intangible story and mechanic ideas.

It seemed as though every story idea I had was good in some ways but didn’t fit in other ways, because I had a massive list of criteria, and I was really afraid of the story not working.

I eventually stuck to a loose story structured around ideas of isolation, loneliness and fear of the unknown. The character goes from homesickness to curiosity to fear, and finally to empowerment. I charted out the steps of this journey, which move through Reality, Memory, Fiction, Dream and Imagination. I noted important transition moments- eg.: moments where the character is losing control require more automatic transitions. I also made note of important visual elements to draw attention to and potential colour schemes. 

So far, I’ve touched on Reality, Memory and Fiction in my prototype. The last scenes have been planned out. Visuals will become more abstract and the world will shift around the player without regard for what the player is doing. However, a simple perception-based puzzle will let the player regain control and return to reality. These remaining scenes will probably only take up less than a third of the game.

I’m not sure whether players will ever get a clear idea of which world they’re in, but they will each have their own distinct characteristics

For example, a characteristic of Fiction will be a lot of intentional framing of important elements that can only be experienced from a certain view and horizontal symmetry. I want to hint at the idea of an illusion or constructed fictional world not only through  what’s being represented (eg.: magic or dragons or archetypes) but also through how things are represented. It is fun to play with some of the representational conventions of storytelling media that are coming a lot in my academic research.

Experimenting with a more limited colour palette

In contrast, the Memory world will be less stable, and more distorted.

What happens if the transition between worlds is really smooth or gradual? There’s no moment of blackness or fade-out to separate the scenes in the player’s mind.

One thing that could work, at least to differentiate between two different places, is to put a reference to the first place somewhere in the viewable distance of your second place. You thought you were on the raft, but now that you see it far away, you may be somewhere else.

Creating repeating references to objects and giving them a progression (eg.: from floating to sinking) helped to create story. It was certainly fun to create these elements, and fun to imagine their implications, but it was a story that wasn’t influenced by any of the player’s actions. For now, these visuals build more of a contextual story world. More playtesting may be important to figure out how much involvement in the story is important to give the player.

Thesis: Bits and Pieces

In this post, I’ll go into more of the specifics observations and decisions in my prototype-making process.

Because the prototype was more than a couple scenes long, I was able to experiment with motifs. For example, reoccurring train visuals and sounds that support the theme of rootlessness.

I played around with characters and different levels of visual ambiguity. I used collections of simple shapes that relied on motion to be recognized as figures. This was inspired by this page on biological motion (discovered through my Cognitive Science course).

Ideas that seemed simple in my head were often much more complicated or time consuming to make. As you can see below, some transitions take a lot of meshes.

I found inspiration for shifts in mis-interpreted images. Often I’ll glance at a painting and the misinterpretation of it will activate my imagination. Then I will look closely and realize that it’s actually portraying something different and less exciting. This relates to the idea of multistable perception (previously written about here).

The concept can also be used in individual scenes.  I’m not sure if intentionally creating scenes with dual meanings will be interesting or just confusing and annoying. However, it’s an important part of my research to clarify this line through testing.

Below: is that fancy wallpaper or are the walls glass?

I implemented a transition and was unsure about it because it didn’t feel as clever (it wasn’t based on multistable perception, for example). However, it ended up being satisfying to experience. It creates a playfulness that reminded me of the spirit of animated credit sequences, though it doesn’t necessarily make the same kind of statement about perception as the other transitions.

This prototype is making me reconsider my understanding of the relationship between surprise and enjoyment in games.

Thesis: Realism

 

Organizing Ideas

It seemed like the more I research, the more I spiral into hours of reading potentially relevant books, the more related terms I find. I made this diagram to try to understand all these connections I was uncovering and how they relate to my game/piece.

The sentence in black describes what I have been making, and certain words relate more to certain concepts from my research (and ultimately, certain areas of study). For example, the ideas of illusion and perception of space relate more to the element of surprise and the element of space in my project. Basically everything relates to “visual”, so assume that there are a million lines branching off from that word. I left those out for clarity. 

I think it’s ok that the word “affect” is vague, at least for now. Right now it just means interesting reactions, whether they are emotions or unexpected ways a player understands their own perception or something else. A lot of the research I’ve found in design, film and animation theory connects perception to realism. This is something I want to pay more attention to, because the overlap in questions of realism between different mediums has been really interesting.

Animation

I’ve previously mentioned my class on cognitive sciences. I’m also taking a helpful class on animation theory and history. The class has been great for helping me find more concrete/searchable connections to my thesis and understanding their context. Recently, we’ve been looking at how animation was changing after World War II. Animators such as Chuck Jones and those from the UPA were taking inspiration from new art movements and exploring things like flatness, self-referentiality, abstraction and illusion. I plan to read more on why animation is argued by some to be a particularly effective form for exploring these ideas. What are the functional and convention-related differences between animation, games, film and paintings that change how we perceive realism and abstraction? 

 

Realism and Rules

Here’s a potential transition where the two images on each end are representational and relatively realistic. The middle is more abstract. It’s not meant to represent anything except the formal middle point of two scenes. What is the difference between transitions like this and transitions where there is an illogical/surreal shift but there is no point where anything you see is not “referentially realistic” (Prince, 92)? Will the latter feel more or less like a violation of the established reality? My theory is that in a medium where fantasy is common and accepted, maintaining “reality” is less about having real references and more about consistently following rules. A player will accept that flying is possible in the game world as long as they know how flying fits into the logic of the world. In this way, they can create expectations of future events in this world based on a kind of consistency they are reasonably comfortable won’t be violated. 

Research on Realism

I am finding a lot of sources that connect perception, space and realism together, especially in the areas of film and design theory.

I found an essay by Stephen Prince called “True Lies: Perceptual realism, digital images and film theory.” The essay is from 1996 and Prince mostly writes about the rise of photo-realistic imagery and how this should effect film theory. He breaks down the realistic imagery label into “referentially fictional”, “referentially realistic” and “perceptually realistic” (92). Breaking down realism is really important, because it can mean so many different things. In the same book, I found an even older essay by Rudolf Arnheim (film theorist, perceptual psychologist; basically a key theorist on the perception of media) who writes about relationships of spatial continuity (32) and “constant of size” (27) to realism in film. 

Another source is Justin Morris’s “Then one day I got in” (2012), which is specifically about games and realism. It connects some previously mentioned Lev Manovich writing to film theorist André Bazin’s ideas of “cinematic reality” (23).

Work Cited

Arnheim, Rudolf. “Film and Nature.” 1933. Trans. L.M. Sieveking and Ian F.D. Morrow. Film Theory: Critical Concepts in Media and Cultural Studies. N.p.: Taylor & Francis, 2004. 25-39. Print.

Morris, Justin. “”Then one day I got in.” Computer Imaging, Realism Tron.” Cineaction 89.89 (2012): 22-28. Academic OneFile. Web. 28 Jan. 2017.

Prince, Stephen. “True Lies: Perceptual realism, digital images and film theory.” 1996. Film Theory: Critical Concepts in Media and Cultural Studies. N.p.: Taylor & Francis, 2004. 85-97. Print.

 

Thesis: Perception and Story

Is story important to the project?

I’d like to say yes because I think a story can guide that experience in an engaging way, and because I’m interested in weirder ways stories can be told. Adding a story would be an opportunity to explore more figurative or poetic ways of storytelling in games, which I could potentially use past my studies at OCAD. 

However, story may detract or distract from the rest of the project project, as the player may be overly focused on interpreting what is happening in the context of the story. Challenges or goals could also detract from the experience in the same way. I often find myself ignoring all irrelevant (albeit beautiful) aspects of games in order to figure out their system to try to win as quickly as possible. 

I still think there’s something compelling about how a story may or may not fit into this project. I was excited by what could be implied by minimal information in those credit sequences. There’s something about a shape used to communicate two drastically different ideas that triggers my imagination; for a second you’re believing something impossible. 

Themes

I’m also finding a lot of possibilities for storytelling in my research. Olly Moss’s movie posters show how visual similes can convey a key concept from a story. Visual similes create visual metaphor and can communicate a relationship between two ideas, though you could say that about film cuts too, and they don’t usually require a clever transition (see writing on montage), just the juxtaposition of two ideas. I also still think there is something interesting about using a shift between different viewpoints to communicate themes of isolation, distance, or disorientation. 

Disorienting tricks bring to mind ideas of:

• Waking dreams
• Otherworldliness
• Magic
• Involuntary memory
• Visual triggers to imagination

Inception and Paprika touch on the dream thing, using perceptual tricks to portray the plasticity of dream reality. The lack of clear consequence and the  deceptiveness of a world you’re unsure is a dream or reality can be really frustrating. This brings up the ideas of trust and suspension of disbelief, as mentioned in the previous post. 

Additionally, I mentioned in my literature review that Pabst’s Die Dreigroschenoper uses disorientation to show characters’ delusion and create an awareness of spectatorship, as

“the film demonstratively and self-reflexively draws the spectator’s attention to framed reflections and mediated images that disrupt the spectatorial experience in explicit terms by forcing the spectator to become conscious of his or her act of watching. “(Heidt)

Visual Misinterpretation and Misinterpretation as a Theme

In addition to potential story themes, my research has reminded me of the process of reading stories, (and of interpreting information in general). More specifically, of how misinterpretation of phrases can trigger one’s imagination and create interesting understandings of texts that rely on one’s own experiences.

One of my super-old thesis ideas was to make a game environment that reflected a child’s misunderstandings and misinterpretations of the world. One could walk into a room as the child, knowing it’s a room the child could never have entered in real life, and see what they imagine is inside. The world would be distorted/shaped by their interpretation, and one might see how our understanding of the world is shaped by how we need it to be seen.

The idea of misinterpretation or alternate interpretation ties into illusion, and ambiguity and a lot of other things I’ve been looking at. It also relates to the topic of perceptual biases, which I’m learning about in that cognitive science class.

Work Cited

Heidt, Todd. “Double Take: Béla Balázs and the Visual Disorientation of G. W. Pabst’s Dreigroschenoper.” Seminar: A Journal of Germanic Studies, vol. 50 no. 2, 2014, pp. 178-196. Project MUSE. Web. 14 Oct. 2016.