Mimic Yourself: Mo-cap Workshop Zurich ZhDK May 30 

On March 30th, Lynda Joy Gerry visited the Innovation Lab at Zürcher Hochschule der Künste (ZhDK) for a workshop entitled “Mimic Yourself.”

This workshop involved collaborations between psychologists, motion-tracking and capture experts, and theater performers. The performers wore the Perception Neuron motion capture suit within an opti-track system. The data from the performer’s motion was tracked onto virtual avatars in real-time. Specifically, the team had used the Structure Sensor depth-field camera to create photogrammetry scans of members of the lab. These scans were then used as the avatar “characters” put into the virtual environment to have the mocap actors’ movements tracked onto. A screen was also programmed into the Unity environment, such that the screen could move around the real world in different angles and three-dimensional planes and show different views and perspectives of the virtual avatar being tracked relative to the human actor’s movements. Two actors playfully danced and moved about while impacting virtual effects with their tracked motion – specifically, animating virtual avatars but also cueing different sound effects and experiences.

Image above: Motion capture body suit worn by human actor and tracked onto a virtual avatar. Multiple avatar “snap shots” can be taken to create visual effects and pictures. Images below: Creating a many-arm shakti pose with avatar screen captures created through mocap.

Image above shows examples of photogrammetry scans taken with the Structure Sensor.

Collaboration meeting @ Virtual Cinema Lab, Aalto University

Our Enactive Avatar team Victor Pardinho, Lynda Joy Gerry, Eeva R Tikka, Tanja Bastamow, and Maija Paavola planning the volumetric video capture of a screen character with a collaborator in Berlin. The team’s work is supported by the Finnish Cultural Foundation, Huhtamäki Fund, and Virtual Cinema Lab (VCL), School of Film, Television and Scenography, Aalto University, and by Pia Tikka’s EU Mobilitas Top Researcher Grant.

Testing behavioral strategies for Enactive Avatar

Testing facial expressions of the viewer driving the behavior of a screen character with the Louise’s Digital Double (under a Creative Commons Attribution Non-Commercial No Derivatives 4.0 license). see Eisko.com

In the image Lynda Joy Gerry, Dr. Ilkka Kosunen and Turcu Gabriel, Erasmus exchange student from the  University of Craiova @Digital Technology Institute, TLU) examining facial expressions of a screen character driven by an AI.

Enactive Virtuality at CNA Sound and Storytelling Conference LA March 22

Pia Tikka and Martin Jaroszewicz gave a joint talk at the CNA Sound and Storytelling Conference.

Image: Dr. Martin Jaroszewicz disucssing his ideas of enactive VR soundscapes at the Chapman University conference, Orange, CA, March 22.

“Enactive Virtuality – a framework for dynamically adaptive soundscapes”. 

The novel notion of enactive virtuality is discussed, drawing from the theories of enactive mind [1], and the concept of enactive cinema [2]. The key attribute of ‘enactive’ refers here to the setting in which the human agent is in continuous feed-back-looped interaction with the surrounding world. Enactive virtuality in turn refers to the story emerging in the agent’s mind in this dynamical setting in order to make sense of the world. This story is based on the agent’s current situation and previous experiences [3], and in relation to others through neurally built-in imitation of their actions [4]. Thus, the concept of ‘enactive virtuality’ extends beyond the common techno-spatial buzzword of ‘virtual reality’. While virtual reality technologies allow platforms where the perception of sound events can be modulated algorithmically, we describe the human agent’s experience of a dynamically adaptive soundscape as an expression of enactive virtuality.Theories and techniques of sound transformations in the spectral domain [5,6,7] for this setting are discussed.

References:

[1] Varela F, Thompson E, Rosch E. 1991. Embodied Mind. Cambridge, MA: MIT Press.

[2] Tikka P. 2008. Enactive Cinema: Simulatorium Eisensteinense. PhD diss. Helsinki: Univ. Art and Design Publ.

[3] Heyes CM, Frith CD. The cultural evolution of mind reading. Science. 2014 Jun 20;344(6190):1243091. doi:10.1126/science.1243091.

[4] Gallese V, Eagle MN, Migone P. 2007. Intentional attunement: mirror neurons (…) J Am Psychoanal Assoc. 55(1):131-76.

[5] Jaroszewicz, M. 2015 Compositional Strategies in Spectral Spatialization. PhD thesis,University of California Riverside,

[6] Jaroszewicz, M. 2017 ”Interfacing Gestural Data from Instrumentalists,” ART – Music Review, vol. 32.

[7] Kim-Boyle. 2008. “Spectral spatialization – an overview,” RILM Abstracts of Music Literature. http://hdl.handle.net/2027/spo.bbp2372.2008.086

A 13 min talk on Neurocinematics, Tallinn University Day 2018

 

1920x1080-13-eng.jpgSee full program here

At 12:15 session Pia Tikka, Research Professor, Baltic Film, Media, Arts and Communication School “Neurocinematics”

My 13 minutes will introduce a multidisciplinary research paradigm of neurocinematics. Combining methods of cinema, enactive media, and virtual screen characters with those of cognitive sciences it allows us to unravel new aspects of the neural basis of storytelling, creative imagination, and narrative comprehension. In addition to contributing to academic research on human mind, neurocinematics contributes to a range of more specifically targeted goals, such as the impact of audiovisual media on its audience for artistic, therapeutic, or commercial implementations, to name few of many.

Enactive Avatar in Time Flies Nordica Spring 2018

You can find an article about prof Pia Tikka and her studies on enactive co-presence between the viewer and screen character in the Nordica in flight magazine “Time Flies”.

You can also read the article in BFM’s blog:
http://media.tlu.ee/imagine-movies-could-read-your-emotions

Common VR/AR grounds with TTÜ and EKA Jan 25

Pia Tikka

Enactive Virtuality team in search for the common grounds with TTÜ and EKA at Ericsson Connectivity room in Mektory House (Raja 15, 12618 Tallinn) 25.01.2018. In order to cultivate successful research environments to study and develop innovations within virtual reality, augmented reality and other related audiovisual fields, collaboration across distinct academic institutions with their specific expertise is essential.

Hosts@TTÜ: Aleksei Tepljakov Research Scientist, Eduard Petlenkov Associate Professor, and Ahmet Kose Junior Researcher at TTU;  Guests: Kristjan Mändmaa, Dean of Design, Ruth Melioranski Design Researcher, Tanel Kärp Interaction Design Program Manager at EKA; Suk-Jae Chang, AR/VR businessman; Pia Tikka, Professor TLU.

Link to Recreation Lab at TTU: https://recreation.ee

TALK by Bruce Sheridan on Creativity: Embodied, Extended, and Enactive 19 Jan 2018

Speaker: Bruce Sheridan (University of Auckland) 
Time: January 19th (Thursday) at 15:00
Place: M-213

Creativity: Embodied, Extended, and Enactive

Cognitivism and computational theories of mind drove approaches to creativity through the 20th century, and formal education across arts and sciences continues to draw on assumptions that are increasingly challenged by developments in neuroscience and artificial intelligence. By emphasizing the physically embodied, socially extended, and inherently enactive nature of human mentation, creativity can be understood as a fundamental, ubiquitous, and highly dynamic capacity central to all human activities. In this presentation I consider ways to conceptualize creativity and learning holistically, and how this approach can both draw from and contribute to work in the biological, physical, and social sciences.

Professor Bruce Sheridan is a filmmaker and educator based in Chicago. He began his filmmaking career in New Zealand, where he produced and directed drama and non-fiction projects for cinema and television, and won the 1999 Best Drama Award for the tele-feature Lawless. From 2001-2017 he was Chair of Cinema Art + Science at Columbia College Chicago, where he led the creation of the Media Production Center, a unique laboratory for modeling and investigating professional collaborative filmmaking practice. Professor Sheridan is Chair of the North American region of CILECT, the international organization of screen education institutions, and a member of the Creativity Research Initiative team at the University of Auckland, New Zealand, where he is currently completing his PhD in philosophy.

With Bruce and Pia in the middle: Vice-Rector for Creative Activities and Cooperation Andres Jõesaar, Head of BFM Katrin Saks, and our EU Mobilitas Plus -project coordinator Maria Hansar.