The Familiar Object — part 1
Spatial Augmented Reality for the age of Context.
“Familiar Object” is an interactive spatial AR art installation that was on public display at the SFA Projects gallery in Lower East Side of Manhattan, New York.
At the time of opening, most in-person VR/AR venues and experiences that would have you share a headset with others were understandably closed down due to Covid.
Therefore “Familiar Object” was probably one of the only AR/VR experiences open to the public during the lockdown, because the Spatial Mixed Reality platform used in this installation [called reFrame], does not require viewers to wear any trackers or share a headset — all you need to do is to just walk in front of it and look through the frame at the objects and scenery beyond/behind it — reFrame tracks you in 3D space and as you move, it corrects the image on its optically see-through display to make sure the virtual image registers to what you’re seeing from your point-of-view in space.
From Ego-Centered design to Subject-Centered design
This spatial approach to mixed reality moves away from the ego-centered design paradigm of various head-mounted displays, to a subject-centered interaction design model that does not seek to replace reality all around the user, instead it shares the same world as the user, inhabits the same space, and altering the reality limited to only a certain part of the space with framed boundaries that functions more as a portal, or a window, to another reality.
In the case of “Familiar Object” it could also be thought of as a window to the past, since the 3D point-cloud scan of the “person sitting in a chair” is captured during the installation from the same spot that the wire sculpture is placed. This point-cloud is rendered where it was captured, super-imposed over the wire sculpture that replaces the real chair.
Related Works and Discussions
The main inspiration for this installation is an early work of the surrealist painter René Magritte, titled “Familiar Objects”, which shows five different people, maybe even the same person, each looking through their own unique perception and have formed five categorically different impressions of what they “see through” their cognition.
Joseph Kosuth explores similar themes in his 1965 installation “One and Three Chairs” where 3 different representations of the same “chair” abstraction are displayed next to each other. To use a Semiotic description, there are three “Signifiers” for the same “Signified.”
Both Magritte and Kosuth had to utilize linguistics in their work to represent concepts in their work, be it the word “pipe” or “chair” to point out the abstract idea of the objects they are referring to. Using language also brings with it many undesired complications into how the work is represented, since using any language means that the accessibility of the artwork will be limited to people who can read and comprehend that language, plus a host of unforeseeable issues that would arise in etymology and translation. Kosuth points these issues out in his other work “One and Three Translations.”
“An image is not so wedded to its name,” Magritte later points out in his essays, “that one cannot find another which suits it better.” He demonstrates this idea in his works such as “The Palace of Curtains, III” and more notably in both French and English versions of “The Interpretation of Dreams.”
For these reasons “Familiar Object” stays away from literature and concept representations which are too high-level, and discusses cognition at the level of phenomena, and perception. There are no indications of what the object is that you’re looking at, other than how it relates to human body and the function it can serve.
You can learn to distinguish a phenomenon by observing how another body would interact with it, and indeed if we take words and language away from you in a mime game, you have no choice other than to embody the description of what that object is, and “what the object is” will be determined by how it relates to your body. This is part of the motivation behind using the “sitting body” as the virtual image, to reliably establish an association between the object and a function.
Embodied Cognition through Embodied Interaction
A “chair” is an interesting entity to consider. You cannot define a chair without mentioning the human body and how they relate to each other. It’s something that’s suitable for the human body to sit on, a fundamental and purely ergonomic design that can only be evaluated through the body.
The “seating” potential of objects in the world can only be evaluated by “Looking Through the Body”, so for example the curb of a city sidewalk might look like a perfect place to sit for a 5-year old kid, but it barely even registers as a “step-over obstacle” for a 6-foot tall person who’s also looking for a place to sit. This is a more functional definition of “embodied cognition,” which is sometimes mischaracterized as mere bodily reactions to thoughts and feelings, like a polygraph, implying body as simply a reactive carrying vessel for the mind, rather than “body as the point of view upon the world” as described by phenomenologist Marleau-Ponty. [Phenomenology of Perception, 1945, Routledge Press]
“Familiar Object” is a low-level exploration of visual perception and what it means to see something, not necessarily as the optical notion of rays of light hitting your retina, but seeing as in what an object is and where is it located in reference to where you are located in space. “What the object is” in part depends on how it looks/feels to your sensory instruments, and another part depends on understanding what function it serves to your body; which can be mirrored by seeing how it relates to another body, establishing Visual Perception as a function that’s “not exclusive to the eye or even the brain, but involve the whole organism as it moves about its environment.”
The main interaction that enables this seeing for us as living systems is the embodied interaction of our attention and our moving around in space when we look at and around something. For “Familiar Object” as an interactive experience, where you are and what you’re paying attention to are basically the main two parameters that are tracked as inputs from users to interact with. It’s trying to demonstrate how even these fundamental parameters can have a lot to offer in and of themselves, before having to teach abstract gestures that users have to learn and repeat correctly in order to interact with the installation.
It’s worth mentioning this quote from the 1991 book “The Embodied Mind” which was one of the first to propose the “embodied cognition” approach in cognitive science:
both the environment and first person experience are aspects of embodiment. However, enactive embodiment is not the grasping of an independent, outside world by a brain, a mind, or a self; rather it is the bringing forth of an interdependent world in and through embodied action. [Francisco Varela, et al. The Embodied Mind, 1991, MIT Press]
