Call for NEUROCINEMATIC papers — Baltic Screen Media Review, Special Issue 2023

Call for NEUROCINEMATIC papers – Baltic Screen Media Review, Special Issue 2023

Cinematic minds in making – Investigation of subjective and intersubjective experiences of storytelling

Guest editors Pia Tikka and Elen Lotman with Maarten Coëgnarts

Contact and submission to NeuroCineBFM@tlu.ee

 

 

One of the key foundations of everyday activities in society is intersubjectively shared communication between people. Stories, films, and other audiovisual narratives promote shared understanding of possible situations in other people’s lives. Narratives expose complex social situations with their ethical, political, and cultural contexts (Hjort and Nannicelli, 2022). They also play a role for the human kind as a means to learn from protagonists and their positive examples and successes, but also their mistakes, false motivations, and blinded desires that may lead to dramatic situations, sometimes even disasters. An exhaustive range of contextual situatedness as the constitutive essence in narratives does not only serve entertainment and education, but also scientific studies of human mind and behavior.

Since the beginning of this millennium, narratives mediated by films have allowed researchers to simulate complex socio-emotional events in behavioral and neuroimaging  laboratories, accumulating new insights to the human behavior, emotion, and memory, to name a few of many topics. The proponents of the so-called naturalistic neuroscience and, in particular its subfield neurocinematics (Hasson et al. 2008) have shown how experiencing naturally unfolding events evokes synchronized activations in the large-scale brain networks across different test participants (see, Jaaskelainen et al. 2021, for review). The  tightly framed contextual settings in cinematic narratives have opened a fresh window for researchers interested in understanding linkages between individual subjective experiences and intersubjectively shared experiences.

So far, neurocinematic studies have nearly exclusively focused on mapping  the correlations between narrative events and observed physiological behaviors of uninitiated test participants.. The knowledge accumulated so far does not tell much of the affective or cognitive functions of the experts of audiovisual storytelling, with few exceptions (e.g. deBorst et al 2016; Andreu-Sánchez et al. 2021).  Along with this Special Issue we want to extend the scope of studies to the embodied cognitive processes  of storytellers themselves. As an example, consider the term “experiential heuristics” proposed by cinematographer ELen Lotman (2021) to describe the practice-based knowledge accumulation by cinematographers, or the embodied dynamics of the filmmaker in the process of simulating the experiences of the fictional protagonists and/or that of  imagined viewers described as “enactive authorship”(Tikka 2022). Another question that merits further understanding is how the embodied decision-making processes of filmmakers further lead to the creation of dynamic embodied structures in the cinematic form. Appealing to the shared embodiment of both the filmmaker and the film viewer, these pre-conceptual patterns of bodily experience or “image schemas” have been argued to play a significant expressive role in the representation and communication of meaning in cinema (Coëgnarts 2019).

We call for papers that focus on the creative experiential processes of the filmmakers,  storytelling experts and their audience. We encourage the proposed papers to discuss how temporally unfolding of contextual situatedness depicted in narratives manifests in reported subjective experiences, the observed  body-brain behaviors,  and time-locked content descriptions.

TOPICS

We invite boldly multidisciplinary papers to contribute with theoretical, conceptual and practical approaches to the experiential nature of filmmaking and viewing. They may draw, for instance, from social and cognitive sciences, psychophysiology, neurosciences, ecological psychology, affective computing, cognitive semantics, aesthetics, or empirical phenomenology. 

Accordingly, we encourage papers that discuss the relations of data from these approaches, all concerning film experience of the film professionals and/or film viewers.  The focus of a submission may also focus on a specific expertise, for example that of the writer, editor, cinematographer, scenographer or sound designer. The papers may describe, for example 1) subjective experiences, 2) intersubjectively shared experiences; 3) context or content annotation, 4) semantic description; 5) first-person phenomenal description, and/or 6) physiological observation (e.g. neuroimaging, eye-tracking, psychophysiological measures).

Contributions addressing topics such as (but not limited to) the following are particularly welcome:

  • Social cognition and embodied intersubjectivity
  • Interdisciplinary challenges for methods; annotations linking cinematic features to physiological data
  • First-and second-person methodologies; empirical phenomenological observations
  • Embodied enactive mind, embodied simulation, and theory of theory mind
  • Embodied metaphors in film; embodied film style; bodily basis of shared film language; semantics;
  • Cinematic empathy; emotions; simulation  of character experiences
  • Audience engagement; immersion; cognitive identification;
  • Temporality of experiences; context-dependent memory coding; story reconstruction; narrative comprehension;
  • Experiential heuristics; multisensory and tacit knowledge;
  • Storytelling strategies, aesthetics, fIlm and media literacy; genre conventions

GUIDELINES

We will accept long research articles (4000 – 8000 words w/o ref) and short articles and commentaries (2000 – 2500 words w/o ref). Submitted papers need to follow Submission guidelines

All submissions should be sent via email attachment to Guest editors at NeuroCineBFM@tlu.ee

BSMR embraces visual storytelling, we thus invite authors to use photos and other illustrations as part of their contributions. See Journal info https://sciendo.com/journal/BSMR


Key dates


01.04.2023 – Submit abstracts of 200–300 words.
10.04.2023 –Acceptance of abstracts
30.06.2023 –Submit full manuscripts for blind peer review
20.09.2023 –Resubmit revisions
31.12.2023 –Special Issue published online


This issue of BSMR will be published both online and in print in December 2023. 

All submissions should be sent via email attachment to Guest editors at NeuroCineBFM@tlu.ee

Guest editors 

Maarten Coëgnarts https://www.filmeu.eu/alliance/people/maarten-coegnarts

Elen Lotman  https://www.filmeu.eu/alliance/people/elen-lotman

Pia Tikka https://www.etis.ee/CV/Pia_Tikka/eng

 

References

Andreu-Sánchez, C., Martín-Pascual, M.A., Gruart, A. and Delgado-García, J.M. (2021). The effect of media professionalization on cognitive neurodynamics during audiovisual cuts. Frontiers in Systems Neuroscience, 15: 598383. doi: https://doi.org/10.3389/fnsys.2021.598383

de Borst, A. W., Valente, G., Jääskeläinen, I. P., & Tikka, P. (2016). Brain-based decoding of mentally imagined film clips and sounds reveals experience-based information patterns in film professionals. NeuroImage, 129, 428–438. https://doi.org/10.1016/j.neuroimage.2016.01.043

Hasson, U., Landesman, O., Knappmeyer, B. Vallines, I., Rubin, N., & Heeger, D. (2008). Neurocinematics: The Neuroscience of Film. Projections, 2, 1–26. https://doi.org/10.3167/proj.2008.020102.

Hjort, M and Nannicelli, T. (Ed.).(2022) The Wiley Blackwell Companion to Motion Pictures and Public Value. Wiley-Blackwell Press.

Jääskeläinen, I. P., Sams, M., Glerean, E., & Ahveninen, J. (2021). Movies and narratives as naturalistic stimuli in neuroimaging. NeuroImage, 224, 117445. https://doi.org/10.1016/j.neuroimage.2020.117445

Lotman, E. (2021) Experiential heuristics of fiction film cinematography. PhD Diss. Tallinn University.

Tikka, Pia (2022). Enactive Authorship Second-Order Simulation of the Viewer Experience— A Neurocinematic Approach. Projections: the Journal for Movies & Mind, 16 (1), 47−66. DOI: 10.3167/proj.2022.160104.

 

CUDAN Pizza-pitching 22.02.

The CUDAN Show & Tell seminars with brief (10 minute) presentations!

Abstract

The field of social robotics, be these physical or virtual, is advancing fast, however, the challenge continues to be the socially relevant context awareness of these artificial characters so that human interaction with them would become meaningful in varying situations of everyday life.

The research aims at developing novel ways to adopt the films and their scripts  as a training material for social robotics. This would constitute a major breakthrough for modeling systemically adaptive dynamics of artificial characters with increased human relevance for a variety of societal XR applications, e.g. in medical or educational fields.

The proposed research focus converges the multidisciplinary expertise at TLU on audiovisual narratives, semiotics, VR/XR, cultural data analytics, machine learning, psychophysiology, and social robotics. It strengthens TLU’s international collaborations within these fields.

https://docs.google.com/document/d/1MYL3nLMpQ_vWJYh9bH04PbgAucyCpydOuuHnxggcsC0/edit?usp=sharing

Enactive Virtuality Lab presents April 16

Welcome. Please join us!

Enactive Virtuality Lab presents the on-going work in online seminar.

Date: Friday April 16, 2021 
Time: 09:15 -13:00 Helsinki

Zoom Meeting

Meeting ID: 947 9465 7125
Passcode: 019362

VIDEO recording (t.b.a.)

Program

To be confirmed

9:15

Pia Tikka (Enactive Virtuality Team leader, MOBTT90)
Introduction

9:20 KEYNOTE

Ighoyota Ben Ajenaghughrure (t.b.c.) 30 min + Q&A  (Vistiting lecturer of Software Engineering @ DT)

Symbiosis in information society  

10:00

Debora Conceição Firmino De Souza (MA Thesis @DTI)
Exploring the affective states of an interviewee during a Q&A session


10:15

Abdallah Sham (doctoral student @DTI)
Facial Expression Recognition in a Human-human dyadic setting


10:30

Ats Kurvet (computer graphics specialist)
Playing  with MetaHuman for “real” in narrative VR environment


10:45-11:00 coffee break (15 min)


11:00

Robert McNamara (doctor in law (US); doctoral student @BFM)
Cinematic Narratives in Gaming Engines to Explore Compassion towards Refugees

11:15

Sampsa Huttunen (doctoral student @Helsinki University)(t.b.a)
Psychophysiological Reactions to Human Face under Different Lighting Conditions: A Pilot Study


11:30

Mehmet Burak Yılmaz (doctoral student @BFM)
Affective functions of camera movements in cinematographic storytelling


11:45 -12:30 
Discussion

 

Open for public!

Join Zoom Meeting
https://zoom.us/j/94794657125?pwd=VHhmL0g2UWFTeHFhWkU3enNjZ3NGUT09

Meeting ID: 947 9465 7125
Passcode: 019362
One tap mobile
+3728801188,,94794657125# Estonia
+3726601699,,94794657125# Estonia

Dial by your location
+372 880 1188 Estonia
+372 660 1699 Estonia
+1 301 715 8592 US (Washington DC)
+1 312 626 6799 US (Chicago)
+1 346 248 7799 US (Houston)
+1 646 558 8656 US (New York)
+1 669 900 9128 US (San Jose)
+1 253 215 8782 US (Tacoma)
Meeting ID: 947 9465 7125
Find your local number: https://zoom.us/u/adMFxkh2vL

Join by Skype for Business
https://zoom.us/skype/94794657125

Biofiction -panel: Neurotechnology and how it will shape our future bodies April 8, 2021

BIO·FICTION

Neurotechnology and how it will shape our future bodies

The BIO·FICTION Science Art Film Festival is one of a kind: It’s a creative and boundary-crossing event with a program filled to the brim with content exploring cutting-edge emerging sciences – in the present, but also in possible futures. https://bio-fiction.com/

BIO·FICTION Panel

8 Apr 2021 19:00 — 21:00

 

BIO·FICTION Panelist (in image)
with Riitta Hari, Pia Tikka, Markus Schmidt and Kasperi Mäki-Reinikka (moderator)
Thursday, April 8, 19–21h (Finnish time, UTC+2)
via Zoom | in English

Neurotechnology and how it will shape our future bodies is the underlying question of the BIO·FICTION Science Art Film Festival. For the kickoff of the Helsinki edition, where we will show a selection of nine films, all of which have been awarded or screened at the festival, we invite you to join the BIO·FICTION online panel. During it, Kasperi Mäki-Reinikka will discuss together with festival director Markus Schmidt, neuroscientist Riitta Hari and artist Pia Tikka neurotechnology and its current and potential impact on society.

Join the panel via Zoom: https://us02web.zoom.us/j/83861833745

– –

Riitta Hari MD PhD is Professor Emerita of Systems Neuroscience and Human Brain Imaging at Aalto University, Finland. She has been developing magnetoencephalography (MEG) for tracking millisecond-scale activation sequences in the human brain, providing fundamental insights into human sensory, motor, cognitive, and social functions in both healthy and diseased individuals. Hari is Academician of Science in Finland since 2010 and member of the National Academy of Sciences USA since 2004. She currently attempts to bridge art and neuroscience without privileging either.

Kasperi Mäki-Reinikka is a Helsinki-based media artist, art educator and researcher working with technological notions of sense. As part of interdisciplinary Brains on Art collective his practice is informed by collaboration with scientists and researchers and the friction between art and science. Mäki-Reinikka is a board member of the Bioart Society, a foil fencer and a teacher of Art and Artificial Intelligence in Aalto University. Mäki-Reinikka is writing an artistic dissertation on interdisciplinary art and its possibilities to discuss changes in human-machine relation. Since August 2020, Mäki-Reinikka has been teaching art in Kallio Upper Secondary School of Performing Arts.

Dr Markus Schmidt founded Biofaction, a technology assessment, science communication and art-science company in Vienna, Austria. With a background in electronic engineering, biology and risk assessment, he carried out environmental risk assessment and public perception studies in various fields, such as GM-crops, nanotechnology, converging technologies, and synthetic biology. He has published over 35 peer-reviewed papers and three edited books about the future of life. In 2010, he helped to chart the field of xenobiology. Schmidt was part of the FUTUREBODY project.

Dr Pia Tikka is a filmmaker and EU Mobilitas Research Professor at the Baltic Film, Media, and Arts School, Tallinn University. She is a founder of NeuroCine research group that studies the neural basis of storytelling. She has published widely on the topics of enactive media, narrative complex systems, and neurocinematics. A Fellow of the Society for Cognitive Studies of the Moving Image and a member of European Film Academy, her filmography includes international productions as well as fiction films, interactive films and VR films she has directed. Currently, she leads Enactive Virtuality Lab at Tallinn University.

– –

BIO·FICTION is part of the ERA-NET project FUTUREBODY and funded by the Austrian Science Fund (FWF: I 3752-B27). The BIO·FICTION programme at Bioart Society is funded via the Biofriction European collaboration project co-funded by the Creative Europe Programme of the European.

Enactive Virtuality Lab presents Dec 1, 2020 

Welcome. Please join us!

Enactive Virtuality Lab presents the on-going work in online seminar.

Date: Tuesday Dec 1, 2020 
Time: 09:30 -12:00 Helsinki
Zoom Meeting

VIDEO recording (not edited) Passcode: &%10sVN$

Program

9:30

Pia Tikka (Enactive Virtuality Team leader, MOBTT90)
Introduction

9:45

Mehmet Burak Yılmaz (doctoral student @BFM)
Emotional impacts of camera movements
Exploring the emotional effects of different camera movement techniques (dolly, Steadicam, handheld) and the direction of the movement. Conducting psychophysiological experiments where the viewers watch cinematographic scenes.

10:00

Robert McNamara (doctor in law; doctoral student @BFM)
The creative potential of cinematic game narratives for evoking empathy for asylum seekers.
Exploring machine learning in the asylum seeker narratives in the “Refugee Status Determinations”. Measuring types of empathy response in depicting child separation by immigration enforcement officers in game engine-based cinematic narratives.

10:15

Debora Conceição Firmino De Souza (MA Thesis @DTI)
Humanizing interactions at the Border Control 
Drawing upon topics of HCI and game development, the study investigates the emotional states elicited by interactions with anthropomorphic Virtual Agents at the Border Control.

10:30

Ats Kurvet (computer graphics specialist)
Creating digital humans on a budget
The challenges and options when creating the visual component for avatars/digital humans.

10:45  

Valentin Siltsenko (research assistant)
Real-time text to speech synthesis 
Synthesizing natural sounding human speech with ability to set the emotion of the speaker.

11:00  

Abdallah Sham (doctoral student @DTI)
Machine learning in dyadic human – artificial agent interaction
Exploring the implementation of machine learning to training virtual human behaviour.

 11:15

Ermo Säks (doctoral student @BFM)
Storytelling in Cinematic Virtual Reality: The role of cinematographic techniques in evoking immersion in virtual environments
Using practical research methods this doctoral project seeks the cinematic techniques feasible to increase the perceived immersion in Cinematic Virtual Reality (CVR) where the user’s main agency is to look around in a narrative story-based CVR drama experience that features a beginning, middle, and end.

11:30-12:00

Discussion
 

Open for public!

A talk at the Forum on Arts & Social Robotics in Hong Kong

Invited talk by Pia Tikka at the Hong Kong Forum Nov 2, 2019.

Hong Kong Baptist University will be partnering with the Consul General of France and the Alliance Francaise to mount an exhibition and a forum, October 31 – November 2, 2019.

The exhibition will focus on the work of the French artist Yves Gellie, specifically his photographs and films related to social robotics and artificial empathy. (t.b.c.)

Hong Kong offers possibilities to play with Sophia from Hanson Robotics.

 

Prague Quadrennial’s 36Q˚ exhibits The State of Darkness

The State of Darkness is exhibited at BLUE HOUR of the Prague Quadrennial’s 36Q˚ June 8-16, 2019.

Enactive Virtuality Lab is presented by associated team members Tanja Bastamow (Virtual Cinema Lab, Aalto ARTS) and Victor Pardinho (Sense of Space Oy). Biosensor adaptation for  the event by Ilkka Kosunen.

Prague Quadrennial’s 36Q˚ (pronounced “threesixty”) presents the artistic and technical side of performance design concerned with creation of active, sensorial and predominantly nontangible ironments. Just like a performer, these emotionally charged environments follow a certain dramatic structure, change and evolve in time and invite our visitors to immerse themselves in a new experience.

WORKSHOPS, MASTERCLASSES
Curated by Markéta Fantová and Jan K. Rolník
8 – 16 June
Small Sports Hall

Our global society seems to be obsessed with fast paced progress of technology and elevates rational intellectual and scientific pursuits above arts that are intuitive and visceral in their nature. And yet creative minds based in the arts are proving that the boundless imagination paired with new technological advancements often result in original and highly inspiring mind-expanding projects. Even though performance design doesn’t need to use modern technology and is often the most inspiring when it uses simple human interaction, we need to explore and experiment with wide range of possibilities new technologies have to offer. PQ Artistic Director Marketa Fantova established 36Q˚ with those thoughts in mind and with a focus on the young, emerging generation of creatives.

Blue Hour

An experimental, interactive environment that fills the entire space of the Industrial Palace Sports Arena will welcome visitors on 8 June and remain open until the end of PQ 2019. The project, based on intensive team work that brings together experienced artists with emerging designers to collaboratively create, will be led by renowned French visual new media artist Romain Tardy.  The curatorial team seeks to experiment with the shifting boundaries between the “non-material” or “virtual” and the “real” world, to explore the capacity of performance design to enlist technology in cultural production.

See more here

TALK by professor Iiro P Jääskeläinen Brain and Mind Lab Aalto Uni

 

Invited lecture and a collaboration meeting with professor Iiro P Jääskeläinen and Enactive Virtuality Lab May 21-22, 2019.

Image: Pia Tikka, Iiro P Jääskeläinen, Jelena Rosic, and Ilkka Kosunen at MEDIT meeting space.

May 21 at 3-4 pm Dr Iiro P. Jääskeläinen, Associate Professor of the Brain and Mind Laboratory, Department of Neuroscience and Biomedical Engineering, Aalto University School of Science, Finland,
gave an open neurocinematic talk on “Using movies as real-life like stimuli during neuroimaging to study the neural basis of social cognition” (room M-134).

Abstract:
Movies and narratives are increasingly used as stimuli in neuroimaging studies. This in many ways helps bridge the gaps between neuroscience, psychology, and even social sciences by allowing stimulation of, and thus also measurement of neural activity underlying, phenomena that have been less amenable to study with more traditional neuroimaging stimulus-task designs. Observation of signature patterns underlying discrete emotions across largely shared brain structures have suggested that both basic and dimensional emotion theories are partly correct. Robust differences in brain activity when viewing genetic vs. adopted sisters going through a moral dilemma in a movie clip have shown that knowledge of shared genes shapes perception of social interactions, thus demonstrating how neuroimaging can offer important measures for social sciences that complement the traditional behavioral ones. Furter, more idiosyncratic brain activity has been observed in high-functioning autistic than neurotypical subjects specifically in putative social brain regions when watching a drama movie. Development of data analysis algorithms holds keys to rapid advances in this relatively new area of research. Modeling the stimulus and recording brain activity is significantly complemented by behavioral measures on how the subjects experienced the movie stimulus.

Image: Jelena Rosic and Ilkka Kosunen engaged in  discussing correlations between ‘pheno’-dynamics and ‘neuro’-dynamics for our micro-phenomenological Memento study, a follow-up for Kauttonen et al 2018.