Talk @ Hong Kong Baptist University

Pia Tikka:

Public lecture

Enactive Virtuality – Conceptualizing the dynamics of narrative cognition

Date:  21 March, 2019

Time:  4:30 – 5:45 pm

Venue: AAB 905, Academic and Adminstration Building, Baptist University Road Campus,  Hong Kong Baptist University, March 21, 2019.

https://talent100.hkbu.edu.hk/

Abstract

I will introduce my concept of enactive virtuality, associated with the idea of narratives as fundamental constructs of human mind. Inspired by the approach to enactive mind by Fransisco Varela and colleagues, the concept of enactive virtuality describes the experiencing mind as a dynamical system of body, brain, and the world. The epistemic triangulation is suggested as a means of translating the concept to practice.

While audiovisual narratives have been shown to elicit very similar physiological responses in different viewers, however, individual life experiences define how the story is interpreted by each. The approach of triadic epistemology, a combination of methods from arts, social sciences, and psychophysiology allows generating integrated knowledge about how different viewers experience particular narratives. The method builds on the fundamentally pragmatist idea that no two domains of knowledge are enough to explain each other, but a third is always required to provide the interpretative angle. Therefore, understanding narrative content needs to be analyzed not only based on subjective reports of the viewers, because that is not anchored to any overall understanding, but they also need to be related to a neurophysiological repertoire of experiences. Similarly, being able to describe the neural activity data collected during the viewing of a film, it is not enough to relate it only to subjective reports of the viewers, but the observations also need to be interpreted to conventions of narrative dramaturgy and filmmaking. A selection of cases are described to clarify the concept of enactive virtuality and the proposed triadic method.

 

 

And a great aftermath of the Talk at the HKBU campus Cafe with Roger Carcia, Chianly Sze, Samantha Goh, Mette Hjort, and Eugene Birman.

2 TALKS on EMOTIVE VR Film by Marie-Laure Cazin

Freud’s last Hypnosis, a neuro-interactive 360 movie for EMOTIVE VR

Presentation of the ongoing project Emotive VR prototype, an innovative form combining VR and EEG headsets. A neuro-interactive omnidirectional movie has been realized, visualized in Virtual Reality (VR) Head-Mounted Display (HMD). During the visualization, the EEG signals are recorded and analyzed in real time. Some visual effects and an interactive music vary according to the emotional state of the viewer.

Marie-Laure Cazin is a Fine Arts teacher in the High school of Arts and Design ESAD-TALM (France) and in Paris 1 Panthéon-Sorbonne. Part of the Enactive Virtuality Research Group, in BFM, University of Talinn, she is currently completing a PhD in Aix-Marseille University (France) on Cinema and Neurosciences. As an artist and a filmmaker, she has developed many experimental cinematic prototypes, using digital tools to create a live interaction between the film and performers or spectators. She collaborated with scientists for art-science projects, working with brains’ datas of emotions in her last interactive projects.

See projects on line (Fr)

Sound designer Matias Harju webpages (Eng

EMOTIVE VR documentation online (Fr)

2 TALKS on EMOTIVE VR by Marie-Laure Cazin (ESAD-TALM, France)

Visiting lecture at Aalto University, Aalto Studios, organised by Virtual Cinema Lab & Enactive Virtuality Lab, January 30, at 14-15

January 30, at 14-15 Place: N-416, BFM, Narva Mnt 27, Nova Building, Tallinn University.

 

Neurocinematics @ the Worlding the Brain Conference, Aarhus University

Enactive Virtuality Lab presented the collaborative research with the Brain and Mind Lab of the Aalto School of Science at the Worlding the Brain Conference in Aarhus University, Nov 27-29.

Image: The son (Juha Hippi) confronting his father (Vesa Wallgren). Short film The Queen (Kuningatar) is directed by Pia Tikka, Production Aalto University in collaboration with Oblomovies Oy 2013.

 


TITLE: Narrative priming of moral judgments in film viewing

Authors: Pia Tikka, Jenni Hannukainen, Tommi Himberg, and Mikko Sams

How does narrative priming influence the moral judgements of the film viewers? In two studies we focus on the evaluation of the rightness of the perceived action of the characters and the acceptability of these actions, in relation to the viewers experience of sympathy and filmic tension.
Providing additional narrative information beforehand for the viewers is an effective method to manipulate how they perceive and make sense of the film narrative. Our experiment data is collected from two different studies, behavioral and psychophysiological. In both experimental settings two groups receive additional background information of either the male or the female character, while the third controls are not primed. All subjects view the same 25 minute long drama film and reply to post questionnaires online.
Based on the collected data in the first experiment using parallel mixed method analysis we showed that the narrative priming itself does not increase the spectrum of the moral judgment statements and the acceptance of the wrong-doings by the characters but more influential factor seems to be the type of the action and its relation to the generally accepted moral norms. Yet, the narrative priming increased the explanatory spectrum of the subjects, which showed to some extent the trend for accepting or trying to understand actions that embody socio-emotionally complex situations. In the second currently on-going psychophysiological study (HR, EDA; EEG) we expect the explanatory spectrum collected via online questionnaires to correlate with the results of the first behavioral study. However, we also expect to show more priming dependent and spatio-temporal film-event dependent differences in arousal between all groups, indicating the influence of priming to the unconscious emotional and cognitive processes related to moral judgements

The Booth @ Worlding the Brain conference in Aarhus Uni

The Booth

Due to on-going experiments more details added about this art/science project only after the experimental data collection has been completed. Our presentation in Aarhus University Worlding the Brain Conference, 27–29 Nov, 2018, showed initial findings.

Team: Pia Tikka, Ilkka Kosunen, Lynda Joy Gerry, Eeva R Tikka, Victor Pardinho, Can Uzer, Angela Kondinska, Michael Becken & Ben Ighoyota Ajenaghughrure, with others.

Finnish Cultural Foundation Huhtamäki Fund; Virtual Cinema Lab Aalto University School of ARTS; BioLab by Digital Technology Insitute, Tallinn University; Tikka & Kosunen: EU Mobilitas Pluss Top Researcher Grant (2017-2022), Estonian Research Council in association with Tallinn University.

Neurocinematic film “The Queen” – collaboration with Aalto Behavioral Lab

Enactive Virtuality Lab is collaborating with the Brain and Mind Lab of the Aalto School of Science, studying how narrative priming affects the viewer’s narrative story construction. Study on-going.

Images: Tea (Maria Järvenhelmi), Henrik  (Vesa Wallgren) and his son Daniel (Juha Hippi). Short film The Queen (Kuningatar) directed by Pia Tikka, screenplay by Eeva R Tikka. Production Oblomovies Oy and Dreammill Productions and in association with Aalto University 2013.
The film has been produced for experimental brain research purposes, which cannot be revealed at this point, as the study is on-going.
In the image Enactive Virtuality Lab’s student team Angela Kondinska and Michael Becken at Aalto University Behavioural Lab measuring physiological signals of volunteers at the end of October.  Collaborators from Aalto School of Science, Brain and Mind Lab are professor Mikko Sams,. Dr.Tommi Himberg, Veli-Matti Saarinen, and project researcher Jenni Hannukainen, Enactive Virtuality Lab, Tallinn University.

Tallinn Summer School in St. Petersburg August 26–31

Ilkka Kosunen

With the support of the Estonia 100 programme, Tallinn University, Tallinn Summer School, and ITMO University  offered a summer school course “Experimental Interaction Design: Physiological Computing Technologies for Performative Arts.”

The main goal of the one-week extensive hands-on course in interaction design was to empower people to shape their digital environments thus providing a new level of digital literacy. This edition focused on Neurotheatre, a specific type of interactive theatre, where audience and/or actors can communicate via brain and neural computer interfaces using multimodal sensors and actuators.

The course introduced core design and interaction design topics in a provocative stance, inviting participants to reflect upon ongoing shifts, connections, and re-framings in just about every area of interaction design, and inciting a rebellion against passivity. This was complemented with the development of skills in systematic evaluation of usability and user experience of interaction designs. The expectation is to see participants take ownership of the interaction design process.

http://summerschool.tlu.ee/russia/

 

Neurocinematic publication – Narrative comprehension beyond language

Pia Tikka, Janne Kauttonen & Yevhen Hlushchuk (2018): “Narrative comprehension beyond language: Common brain networks activated by a movie and its script”

A young girl Nora stares shocked at her mother Anu. Anu stands expressionless by the kitchen table and scrapes the left-over spaghetti from Nora’s plate into a plastic bag. She places the plate into the bag and starts putting there other dining dishes, takes a firm hold of the bag and smashes it against the table. Nora is horrified: “Mother! What are you doing?. Anu continues smashing the bag without paying attention to her daughter. Nora begs her to stop. Anu collapses crying against the table top. Nora approaches, puts her arms around the crying mother and starts slowly caressing her hair.

The dramatic scene describes a daughter witnessing a nervous breakdown of her mother. Its narrative content remains the same should one read it in a textual form or viewed it as a movie. It is relatively well known how narratives are processed in the distinct human sensory cortices depending on the sensory input through which the narrative is perceived (reading, listening, viewing; [15]). However, far less is known of how the human brain processes meaningful narrative content independent of the media of presentation. To tackle this classical dichotomy issue between form and content in neuroimaging terms, we employed functional magnetic resonance imaging (fMRI) to provide new insights into brain networks relating to a particular narrative content while overlooking its form.

In the image Nora (actress Rosa Salomaa); director Saara Cantell, cinematography Marita Hällfors (F.S.C), producer Outi Rousu, Pystymetsä Oy, 2010.

Abstract

Narratives surround us in our everyday life in different forms. In the sensory brain areas, the processing of narratives is dependent on the media of presentation, be that in audiovisual or written form. However, little is known of the brain areas that process complex narrative content mediated by various forms. To isolate these regions, we looked for the functional networks reacting in a similar manner to the same narrative content despite different media of presentation. We collected 3-T fMRI whole brain data from 31 healthy human adults during two separate runs when they were either viewing a movie or reading its screenplay text. The independent component analysis (ICA) was used to separate 40 components. By correlating the components’ time-courses between the two different media conditions, we could isolate 5 functional networks that particularly related to the same narrative content. These TOP-5 components with the highest correlation covered fronto-temporal, parietal, and occipital areas with no major involvement of primary visual or auditory cortices. Interestingly, the top-ranked network with highest modality-invariance also correlated negatively with the dialogue predictor, thus pinpointing that narrative comprehension entails processes that are not language-reliant. In summary, our novel experiment design provided new insight into narrative comprehension networks across modalities.

Conference presentation @Virtual Reality as a Transformative Technology to Develop Empathy June 20

A video conference presentation at the 1st “Virtual Reality as a Transformative Technology to Develop Empathy” Conference, organised by the “empathic Reactive MediaLab Coalition” (eRMLab Coalition) and EmpaticaXR research group.

 

Lynda Joy Gerry: Envisioning Future Technology for Compassion Cultivation

Experiments in cognitive neuroscience have recently dissociated neural pathways underpinning the experience of empathy from that of compassion. Namely, whereas the experience of empathy involves neural activations regarding an affective pain resonance with the distress or suffering of the target, compassion is correlated to reward centers and positive affect regions of the brain. The import of this finding is that empathy may in some instances lead to a withdrawal reflex if there is excessive sharing of distress with the target, whereas compassion appears to involve a care for the welfare of another and an approach motivation. Within the last five years, virtual environments (VEs) have been increasingly researched and developed towards the goal of enhancing users’ social intelligence, self-compassion, positive attitudes towards out-groups, and empathy. However, most of these VEs do not help the user to become more aware of his or her own emotional and psychological states in response to another person or persons, arguably a crucial step in the recovery from empathic distress or over-arousal. Biofeedback training has also been used for improving social cognition skills, but only a few projects have incorporated biofeedback into empathy-enhancing virtual environments. Recovery from empathic distress as a skill trained through biofeedback VEs could enhance interpersonal connectedness, quality of life, and social cohesion.

Article “Screendance as enactment…” DATJournal Design Art and Technology

In this article Deren’s film At Land is analyzed as an expression of a human body-brain system situated and enactive within the world, with references to neuroscience, neurocinematic studies, and screendance.

TIKKA, Pia. Screendance as enactment in Maya Deren’s At Land. DATJournal Design Art and Technology, [S.l.], v. 3, n. 1, p. 9-28, june 2018. ISSN 2526-1789. First published in The Oxford Handbook of Screendance Studies Edited by Douglas Rosenberg,Aug 2016. Available  online here

Hybrid Arts and Science Collaboration Day dedicated to Leonardo 50

Hybrid Arts and Science Collaboration Day, Celebrating Leonardo’s 50th Anniversary  

The full day event brought together the local and international long term supporters and authors who have contributed to the 50 years of Leonardo, the leading international peer-reviewed journal on the use of contemporary science and technology in the arts and music and, increasingly, the application and influence of the arts and humanities on science and technology.

Upper image: The executive director of Leonardo-journal Roger F. Malina at the map that visualizes the broad international activity of the Leonardo community Below: Vice-dean of Aalto Arts Rasmus Vuori, Leonardo board member artist-curator Nina Czegledy, and Leonardo’s executive director  Roger F. Malina with Leonardo author Pia Tikka.

The journal Leonardo was founded in 1968 in Paris by kinetic artist and astronautical pioneer Frank Malina. Malina saw the need for a journal that would serve as an international channel of communication among artists, with emphasis on the writings of artists who use science and developing technologies in their work. After the death of Frank Malina in 1981 and under the leadership of his son, Roger F. Malina, Leonardo moved to San Francisco, California, as the flagship journal of the newly founded nonprofit organization Leonardo/The International Society for the Arts, Sciences and Technology (Leonardo/ISAST). Leonardo/ISAST has grown along with its community and today is the leading organization for artists, scientists and others interested in the application of contemporary science and technology to the arts and music.