Enactive Virtuality Research Group, Tallinn University
Activities related to narratives, and narrative systems, art-science, academic and popular events, including related industry engagements
Our Enactive Avatar team Victor Pardinho, Lynda Joy Gerry, Eeva R Tikka, Tanja Bastamow, and Maija Paavola planning the volumetric video capture of a screen character with a collaborator in Berlin. The team’s work is supported by the Finnish Cultural Foundation, Huhtamäki Fund, and Virtual Cinema Lab (VCL), School of Film, Television and Scenography, Aalto University, and by Pia Tikka’s EU Mobilitas Top Researcher Grant.
Testing facial expressions of the viewer driving the behavior of a screen character with the Louise’s Digital Double (under a Creative Commons Attribution Non-Commercial No Derivatives 4.0 license). see Eisko.com
In the image Lynda Joy Gerry, Dr. Ilkka Kosunen and Turcu Gabriel, Erasmus exchange student from the University of Craiova @Digital Technology Institute, TLU) examining facial expressions of a screen character driven by an AI.
News from our international network.
A visiting researcher Ellen Pearlman, hosted by MEDIT and DTI at Tallinn University in November 2017, introduced the Enactive Virtuality team with filmmaker Karen Palmer and her team. Karen spent 6 months at ThoughtWorks as the Artist in Residence in NYC working on her emotionally responsive immersive film RIOT. Our interaction with Karen’s team was focusing on joint interest in the advanced facial recognition technologies that would allow driving the narrative in real time.
Andrew Gordon leads the Narrative Group at the University of Southern California Institute for Creative Technologies where his research is devoted to getting computers to be able to read and generate stories and to use the knowledge in stories in order to become more intelligent. He is also a professor in the USC Department of Computer Science. His project involves reimagining a 70-year-old social science experiment for the digital age.
In 1944, Fritz Heider and Marianne Simmel created a simple animated film depicting the motion of two triangles and a circle as they moved in and around a box that alternated between being opened or closed. Heider and Simmel asked people to describe what they saw. Now a classic work in the field of social psychology, the subjects responded with creative narratives that ascribed human-like goals, plans, beliefs, and emotions to the moving objects. Popular themes included romantic relationships and prison breaks. See also an article by professors of psychology Fritz Heider and Marianne Simmel: “An Experimental Study of Apparent Behavior,” American Journal of Psychology 57.2 (April 1944): 243-59.
ICT’s Andrew Gordon Brings Seminal 1940’s Social Science Experiment Online
Pia Tikka and Martin Jaroszewicz gave a joint talk at the CNA Sound and Storytelling Conference.
Image: Dr. Martin Jaroszewicz disucssing his ideas of enactive VR soundscapes at the Chapman University conference, Orange, CA, March 22.
“Enactive Virtuality – a framework for dynamically adaptive soundscapes”.
The novel notion of enactive virtuality is discussed, drawing from the theories of enactive mind [1], and the concept of enactive cinema [2]. The key attribute of ‘enactive’ refers here to the setting in which the human agent is in continuous feed-back-looped interaction with the surrounding world. Enactive virtuality in turn refers to the story emerging in the agent’s mind in this dynamical setting in order to make sense of the world. This story is based on the agent’s current situation and previous experiences [3], and in relation to others through neurally built-in imitation of their actions [4]. Thus, the concept of ‘enactive virtuality’ extends beyond the common techno-spatial buzzword of ‘virtual reality’. While virtual reality technologies allow platforms where the perception of sound events can be modulated algorithmically, we describe the human agent’s experience of a dynamically adaptive soundscape as an expression of enactive virtuality.Theories and techniques of sound transformations in the spectral domain [5,6,7] for this setting are discussed.
References:
[1] Varela F, Thompson E, Rosch E. 1991. Embodied Mind. Cambridge, MA: MIT Press.
[2] Tikka P. 2008. Enactive Cinema: Simulatorium Eisensteinense. PhD diss. Helsinki: Univ. Art and Design Publ.
[3] Heyes CM, Frith CD. The cultural evolution of mind reading. Science. 2014 Jun 20;344(6190):1243091. doi:10.1126/science.1243091.
[4] Gallese V, Eagle MN, Migone P. 2007. Intentional attunement: mirror neurons (…) J Am Psychoanal Assoc. 55(1):131-76.
[5] Jaroszewicz, M. 2015 Compositional Strategies in Spectral Spatialization. PhD thesis,University of California Riverside,
[6] Jaroszewicz, M. 2017 ”Interfacing Gestural Data from Instrumentalists,” ART – Music Review, vol. 32.
[7] Kim-Boyle. 2008. “Spectral spatialization – an overview,” RILM Abstracts of Music Literature. http://hdl.handle.net/2027/spo.bbp2372.2008.086
Pia Tikka
After discussions with Dr. Jonas Kaplan, Co-Director of the Dana & David Dornsife Cognitive Neuroimaging Center and professor of Psychology at USC’s Brain and Creativity Institute, a stop at the lobby by a trumpet and a painting by legendary Miles Davis.
(Seminar in Finnish language)
Keväällä 2017 Helsingin Sanomain Säätiö järjesti suunnatun haun teemana Totuudenjälkeinen aika. Säätiön hallitus myönsi yli puoli miljoonaa euroa kymmenelle hankkeelle. Mitä hankkeille kuuluu nyt? Mitä ilmiöstä on saatu selville? Miten tästä eteenpäin?
Totuudenjälkeinen aika – nyt -seminaariin maanantaina 19.3. klo 16–18 Päivälehden museoon (Ludviginkatu 2–4).
OHJELMA
Politiikka ja distruptio. Toimitusjohtaja Taru Tujunen, Ellun Kanat
Poliittiset kuplat ja media. Dosentti, yliopisto-opettaja Arttu Saarinen, Turun yliopisto
Miten media selviää polarisaation ja paskapuheen aikana? Dosentti, vanhempi tutkija Antto Vihma, Ulkopoliittinen instituutti
Alustusten jälkeen paneelikeskustelu, jota vetää toimittaja Johanna Vehkoo.
#totuudenjälkeen
Lisää hankkeista: www.totuudenjälkeinen.fi
Helsingin Sanomain Säätiö on yksityinen, yleishyödyllinen säätiö. Sen tehtävänä on suomalaisen median ja laatujournalismin tulevaisuuden turvaaminen ja sananvapauden tukeminen. Säätiö jakaa apurahoja viestintäalan tutkimukseen ja koulutukseen sekä alan kilpailuihin. Helsingin Sanomain Säätiö ylläpitää Päivälehden museota ja Päivälehden arkistoa.
*Team:
Janne Kauttonen, FT, researcher, NeuroLab, Laurea-ammattikorkeakoulu
Jenni Hannukainen, FM, researcher, NeuroLab, Laurea-ammattikorkeakoulu
Jyrki Suomala, KT (dosentti), Lecturer, NeuroLab, Laurea-ammattikorkeakoulu
Pia Tikka, TaT (dosentti), EU Mobilitas Pluss research professor, University of Tallinn
See full program here
At 12:15 session Pia Tikka, Research Professor, Baltic Film, Media, Arts and Communication School “Neurocinematics”
My 13 minutes will introduce a multidisciplinary research paradigm of neurocinematics. Combining methods of cinema, enactive media, and virtual screen characters with those of cognitive sciences it allows us to unravel new aspects of the neural basis of storytelling, creative imagination, and narrative comprehension. In addition to contributing to academic research on human mind, neurocinematics contributes to a range of more specifically targeted goals, such as the impact of audiovisual media on its audience for artistic, therapeutic, or commercial implementations, to name few of many.
