Follow-up event of the biannual Baltic NeuroCine kick-off conference 2022 (next scheduled to 2024).
Organizers Elen Lotman & Pia Tikka, Baltic Film, Media and Arts School, Tallinn Unversity
Neurocine Seminar is part of FilmEU_RIT Pilot activities.
Photo by Heikki Leis.
Open call for PARTICIPATION
Tallinn invites students and film professionals to join the Baltic NeuroCine seminar & workshop days in May 5-7, 2023. Registration opens in mid April.
The event aims to enhance interdisciplinary research endeavors at European film schools on cinematic storytelling, film viewing, and related first-person experiences.
LINK TO Neurocine seminar PROGRAM May 5-7 HERE
Since the beginning of this millennium, narratives mediated by films have allowed researchers to simulate complex socio-emotional events in behavioral and neuroimaging laboratories, accumulating new insights to the human behavior, emotion, and memory, to name a few of many topics. The proponents of the so-called naturalistic neuroscience and, in particular its subfield neurocinematics (Hasson et al. 2008) have shown how experiencing naturally unfolding events evokes synchronized activations in the large-scale brain networks across different test participants (see, Jaaskelainen et al. 2021, for review). The tightly framed contextual settings in cinematic narratives have opened a fresh window for researchers interested in understanding linkages between individual subjective experiences and intersubjectively shared experiences.
So far, neurocinematic studies have nearly exclusively focused on mapping the correlations between narrative events and observed physiological behaviors of uninitiated test participants.. The knowledge accumulated so far does not tell much of the affective or cognitive functions of the experts of audiovisual storytelling, with few exceptions (e.g. de Borst et al 2016; Andreu-Sánchez et al. 2021). Along with this Special Issue we want to extend the scope of studies to the embodied cognitive processes of storytellers themselves. As an example, consider the term “experiential heuristics” proposed by cinematographer Elen Lotman (2021) to describe the practice-based knowledge accumulation by cinematographers, or the embodied dynamics of the filmmaker in the process of simulating the experiences of the fictional protagonists and/or that of imagined viewers described as “enactive authorship”(Tikka 2022).
- Andreu-Sánchez, C., Martín-Pascual, M.A., Gruart, A. and Delgado-García, J.M. (2021). The effect of media professionalization on cognitive neurodynamics during audiovisual cuts. Frontiers in Systems Neuroscience, 15: 598383. doi: https://doi.org/10.3389/fnsys.2021.598383
- de Borst, A. W., Valente, G., Jääskeläinen, I. P., & Tikka, P. (2016). Brain-based decoding of mentally imagined film clips and sounds reveals experience-based information patterns in film professionals. NeuroImage, 129, 428–438. https://doi.org/10.1016/j.neuroimage.2016.01.043
- Hasson, U., Landesman, O., Knappmeyer, B. Vallines, I., Rubin, N., & Heeger, D. (2008). Neurocinematics: The Neuroscience of Film. Projections, 2, 1–26. https://doi.org/10.3167/proj.2008.020102.
- Jääskeläinen, I. P., Sams, M., Glerean, E., & Ahveninen, J. (2021). Movies and narratives as naturalistic stimuli in neuroimaging. NeuroImage, 224, 117445. https://doi.org/10.1016/j.neuroimage.2020.117445
- Lotman, E. (2021) Experiential heuristics of fiction film cinematography. PhD Diss. Tallinn University.
- Tikka, Pia (2022). Enactive Authorship Second-Order Simulation of the Viewer Experience— A Neurocinematic Approach. Projections: the Journal for Movies & Mind, 16 (1), 47−66. DOI: 10.3167/proj.2022.160104.
MISSION FOR NEUROCINE SEmINAR
We take as the starting point the assumption that filmmakers and viewers alike build their experience and sense-making on the same embodied foundation, as their occasional references to “instincts”, or “gut feelings” may suggest. These experiences and actions could be likewise addressed using neurocinematic methods. The question our NeuroCine seminar puts forward is, in what ways could neuroimaging observations of viewer experience be extended to accumulating novel insights on the professional practices?
We ENCOURAGE PhD studentS TO JOIN*
During the NeuroCine seminar & workshop days, we will discuss a range of research questions that could help to understand more of the creative processes of professional filmmaking practitioners, e.g., screenwriters, editors, sound-designers, cinematographers, etc.
The talks will focus on the film experience of the film professionals and/or film viewers. The theoretical, conceptual and practical approaches to the experiential nature of filmmaking and viewing may draw, for instance, from social and cognitive sciences, psychophysiology, neurosciences, ecological psychology, affective computing, cognitive semantics, aesthetics, or empirical phenomenology.
The focus of the talk may also be on a specific expertise, for example that of the writer, editor, cinematographer, scenographer or sound designer. The papers may describe, for example 1) subjective experiences, 2) intersubjectively shared experiences; 3) context or content annotation, 4) semantic description; 5) first-person phenomenal description, and/or 6) physiological observation (e.g. neuroimaging, eye-tracking, psychophysiological measures).
Additional possibility is that you may offer your topic to be published as an article in the Baltic Screen Media Review, Special Issue, Guest editors Pia Tikka and Elen Lotman with Maarten Coëgnarts, entitled: “Cinematic minds in making – Investigation of subjective and intersubjective experiences of storytelling” – All journal articles go through a separate submission and a rigorous peer-review process. Contact: NeuroCineBFM@tlu.ee
*Possibility to gain PhD study credits: By request, upon participation PhD students receive 1 ECT for participation and reading assignment and 3 EAP for participation, presentation, reading and writing assignment. International PhD students are advised to negotiate eligibility of these study points with their own local study coordinators.
Elen Lotman https://www.filmeu.eu/alliance/people/elen-lotman
Pia Tikka https://www.etis.ee/CV/Pia_Tikka/eng
SEE Neurocine seminar PROGRAM May 5-7 HERE
PRESENTERS MAY 5
WORKSHOP MAY 5
Workshop: Creating cognitive overload: image density in the script
In this workshop the participants will use a script-writing method developed by Maude Sills-Néron that allows a new avenue for communicating emotional content of the scene in the creative writing process. By deliberately producing an overload of information on their own scene, for instance, from a script they are currently working with, the participants will learn new ways to balance the emotional content they are creating. The applied two-columns script format illuminates some effects that this iteration may have on the production level, such as convey the intention behind a scene or allowing a clearer spatialization of the mise-en-scène. The workshop guides the participants to identify, manage, but also creatively apply the overload of information in their scripts beyond regular scriptwriting programs.
(Alphabetical; to be completed)
CREATIVITY TALK May 5
The Multimodal Writer: on storytelling and writing visuals
Dr. Josie Barnard SFHEA; Associate Professor in Creative and Digital Practice, De Montfort University, UK
This lecture views film storytelling through the prism of the theory of creativity that Barnard provides in her book, The Multimodal Writing: Creative Writing Across Genres and Media (Bloomsbury, 2019). Writing for film is complex and dynamic, with multiple external factors involved. To develop a creative flexibility that enables negotiation of such external multimodality, ‘a writer must acknowledge and learn how to deploy an internal multimodality – that is, all the mental work (conscious and unconscious) a writer does before operationalising any medium. The channels of communication that work together are, in this context, primarily internal channels – intellectual enquiry, moods, sensations, for example – which are brought to work together by a co-ordinating unconscious capacity which [Barnard conceptualises] as the “inner auteur”’ (ibid, p. 6). The lecture draws on a programme of academic research starting 2012 and Barnard’s extensive practitioner experience as award-winning author of novels, creative non-fiction books and scripts for publishers including Virago/LittleBrown, BBC Radio and Film 4.
Barnard, Josie (2019) The Multimodal Writer: Creative Writing Across Genres and Media, London: Bloomsbury/Red Globe.
Talk is facilitated by the official BFM PhD program.
MetaHuman with Wav2lip: New Virtual Representations and their Ethical Implications
The proposed paper will address ethical concerns surrounding representation of vulnerable groups as well as the methodological challenges inherent in using virtual characters in human studies which involve representing such groups. Such concerns focus on consequences arising from the technological affordances of new systems for creating graphical and audio representations which are capable of portraying human-like beings with near identical resemblance. Enacting such virtual representations of humans inevitably gives rise to important ethical questions: (1) Who has the right to tell certain stories? (2) Is it ethical to change the medium of a narrative and the identity of a narrator? (3) Do such changes, or technological mediations, affect whether a vulnerable group will be fairly and accurately portrayed? (4) And what are the implications, either way? While the backdrop of the paper involves discussing the potential of virtual representation as a tool for moral and social change, the ethical implications inherent in the use of new technologies such as Unreal Engine’s MetaHuman to create human-like virtual characters in conjunction with software such as Wav2lip, which allows for accurate lip syncing of audio by virtual characters, call for needed theoretical scrutiny from a methodological perspective.
Bio: Robert McNamara has a background in American criminal law as well as degrees related to audiovisual ethnography and eastern classics. He is from New York State, but has lived in Tallinn for the last five years. Currently, he specifically focuses on researching related to ethical, socio-political, anthropological, and legal aspects in the context of employing human-like VR avatars and related XR technology. He is co-authoring journal papers in collaboration with other experts on the team related to the social and legal issues surrounding the use of artificial intelligence, machine learning, and virtual reality avatars for governmental immigration regimes. During 2020-2023 Robert G. McNamara has worked with the MOBTT90 team. Currently he focuses on his PhD project as a doctoral student.
PRESENTERS MAY 6
How does cinematography influence the perception of film narrative?
Does cinematography influences on its own, the narrative perception of the audience? Does cinematography influence the course of the narrative despite the director’s orientation? In order to answer these questions a practical essay was performed to try to reach a better understanding at what extent can cinematography influence audience perception. The essay tried to isolate as much as possible the role of the cinematographer versus the one from the director and from other departments. Viewing sessions were made with focus groups who have answered a survey with a few questions about their experience by watching the film. The research essay demanded a methodology that was based on producing a film inspired on a poem. The film was shot three times, with the same director, same script, same actors, same location, same crew, the only element that was changed in the production was the cinematographer. Each film was shot by a different cinematographer. The total length of the three films is 9 minutes. Trailer: https://vimeo.com/815886196
Bio: Tony Costa s both a professional cinematographer and a teacher at Lusofona University where he is the head of the cinematography department. has PhD in Communication Studies by Lusofona University 2022, and Master’s degree in film studies since 2014. He is an award winner cinematographer. He got the Golden Prize in Tbilisi Georgia in 2014 with his work on «Lapis Azul». Since 2004 he has been a professor in The Lusofona University in Lisbon by maintaining his professional assignments as cinematographer. Costaa is one of the founders of the Portuguese Academy of cinema and of the Portuguese Society of Cinematographers where he keeps a seat in the board of directors.
Jaime lopes DIez
Diegetic and non-diegetic surprises, and their effect on liking, long-term recall and comprehension in narrative television commercials.
Surprise is one of the most used emotions in audiovisual stories. In this talk, we will show the results of an experiment in which we tried to categorize different types of surprise, based on artificial intelligence, and we tested the preference and intensity of the experience of surprise using as independent variables very short narratives, narrative television commercials.
Bio: Dr. Jaime Lopez-Diez is an Assistant Professor of Audiovisual Communication at the Complutense University of Madrid (Spain). His main research interests are the study of film narratives from a cognitive and neuroscientific point of view. His Doctoral thesis addresses the effectiveness of different types of surprises in audiovisual storytelling from these two points of view. He is also int erested in cross-cultural film studies.
Bio: Katrin Heimann is trained in philosophy (M.A) and cognitive neuroscience (M.Sc and PhD) and currently PostDoc both at the Interacting Minds Center at Aarhus University, Denmark, as well as the Max Planck Institute of Empirical Aesthetics in Frankfurt, Germany. Her methodological focus lies on methods to work with subjective experience, in particular micro-phenomenology, which she uses to explore playfulness, aesthetic experiences (with a focus on audiovisual productions and film) and their their relation to sustainable development of individual and society. Heimann is passionate about considering and developing science as an investigative intervention. She is part of the Experimenting, Experiencing, Reflecting Team of the IMC and work package holder in the Horizon 2020 grant ARTIS (Art and Research on Transformation of Individuals and society), set to explore and co-create transformative art experiences from 2020-2024.
Ecological Approach to Cinematographic Lighting of the Human Face – A Pilot Study
Bio: Sampsa Huttunen is a Finnish documentarist and cinematographer based in Helsinki. He holds a Master’s degree in Social Sciences and a BA in Film and Media Design. Apart from his cinematography and documentary film work, Huttunen is a senior lecturer at the Metropolia University of Applied Sciences and a doctoral researcher at the University of Helsinki, where he is preparing a doctoral thesis about the emotional effects of film.
Perceptual professionalization – eye tracking pilot experiment
As a passionate proponent of including tacit knowledge from filmmakers to neurocinematic studies, Elen has experienced first hand the struggles of bringing filmmakers into the context of academic studies. The two worlds often seem to be too far apart. Challenges include both the terminology and mutual understanding, but also technical and human resources for conducting large studies. During the last couple of years, Elen has started to test a model of small pilot studies, that are less scientifically rigorous, but offer a possibility to test original hunches that come from both filmmaking and film teaching experience. In her talk, she will discuss her experiences of conducting small pilot experiments, elaborate on the challenges and opportunities that come with this model and present one of the studies in more detail. The pilot experiment that will be addressed in more detail is the one where she studied perceptual professionalization in an eye tracking experiment.
Bio:Elen Lotman is a cinematographer who has shot numerous feature films, shorts and documentaries. For various film projects she has travelled to Japan, Tibet, China, Thailand, India, Nicaragua, Costa Rica, Lapland and the Russian Arctic. Her films have won awards and have been shown in competition programs of A-class film festivals like Camerimage, IDFA, Tampere Film Festival, Black Nights Film Festival, Moscow International Film Festival etc. Most recently, her full-length documentary ‘Dear Mother’ was nominated for Best Cinematography at the ReelHeart Filmfestival in Toronto and won the Best Feature Documentary in 2020. Full-length fiction ‘Good Bye Soviet Union’ was nominated for Camerimage Golder Frog in the Directors’ Debuts Competition and won the Black Nights Film Festival audience award in 2021. Her feature film screenplay ‘Container’ made it to the quarterfinalist at the American Film Academy Nicholl screenwriting competition 2021 and her virtual exhibitions filmed for Tallinn Art Hall were chosen among world’s 10 best virtual exhibitions of 2020 by both New York Times and Wallpaper Magazine.
Elen Lotman is the Associate Professor of Film Arts at Baltic Film, Media and Arts School (BFM) of Tallinn University and curates the artistic research branch in BFM PhD studies. Elen was elected co-President of the European Federation of Cinematographers (IMAGO) after serving as IMAGO Board Member and also Diversity and Inclusion Committee co-chair. She defended her PhD thesis titled “Experiential Heuristics in Fiction Film Cinematography” with laudatur in 2021.
PRESENTERS MAY 7
Shaping films from the inside-out: Embodied mental schemas in filmmaking- and viewing
This paper discusses the role of embodied mental schemas in filmmaking- and viewing by highlighting three premises: (1) perception is inferential and relies on prior embodied cognitive schemas, (2) in film form these embodied schemas, as resulting from creative thinking processes, become highly indicative of the films’ intended meanings, and (3) film viewers actively engage in a process of mind-modelling to comprehend these fleshed-out meanings. Because viewers share the same embodied knowledge by virtue of which artful purposes are fleshed out in cinematic form, I will argue that viewers are able to create a mental model of the filmmakers’ embodied mind on the basis of the schemas at work in the film’s functional form. Following recent scholarship on authorship, I will define this mind not in singular terms (e.g., the author-director view), but in terms of the distributed creativity of multiple film practitioners.
BIO: Maarten Coëgnarts is Assistant Professor in Film Studies at the University of Antwerp and fellow of SCSMI. His research on embodied cognition, metaphor and cinema has been widely published in various international peer-reviewed journals including Art & Perception, Cinéma & Cie, Metaphor and Symbol, New Review of Film and Television Studies, Palgrave Communications and Projections. He is co-editor of the book Embodied Cognition and Cinema (Leuven University Press, 2015) and author of the book Film as Embodied Art: Bodily Meaning in the Cinema of Stanley Kubrick (Academic Studies Press, 2019). ). He has presented at several international conferences and has recently been appointed research fellow at the Department of Art History and Image Studies at the University of the Free State. He is currently involved in the FilmEU_RIT pilot project “Artistic Research and Cognitive Film Studies” which aims at furthering transdisciplinary collaboration between artistic researchers and cognitive film scholars.
Mircea Valeriu Deaca
POV shots in light of Cognitive Grammar
The paper applies Ronald Langacker’s Cognitive Grammar notions to cinematic editing constructions. Notions like head, complement, autonomous and dependent structures and the concept of elaboration site (e-site) are applied to editing constructions. The e-site represents a schematic region in the situation evoked by a simulation, and plays the role of a simulator and a goal concept. Simulations can contain schematic components that only partially represent entities and event frames, not just detailed representations of them. Starting from grammatical constructions, we will detail the couplings between control models in sequences of POV shots. The visual expression is a composite structure that “inherits’ its profile from a component structure. Different construals and constructional schemas are instantiated by the juxtapositions of shots. The e-site plays the role a goal-concept. Goal-derived categories constitute the mappings between roles in event sequences, and instantiations in the environment. As structures of these mappings emerge, the conceptual system becomes organized around the action–environment interface (Barsalou 2003: 522). Since conceptual representations are dynamically contextualized to support diverse courses of goal pursuit, situated conceptualizations support conceptual inferences via pattern completion. When one part of a situated conceptualization is perceived, the remainder of the conceptualization becomes active, constituting inferences about the current situation. In short, the e-site can be defined as a summary conceptualization that inhabits a schematic region of a simulation, and prompts the film viewer to construct concepts that will categorize other components down the chain of a sequence of shots. In this paper, I describe different semantic construals evoked by a POV sequence of shots. The claim is that sequences of shots are conceptual systems bound by circular loops at several levels. Several configurations of the system coexist and the viewer has the executive control necessary for implementing alternative meaning construals.
Bio: Mircea Valeriu Deaca teaches film analysis and theory, film history and contemporary Romanian film at the University of Bucharest, Faculty of History, Department of Ancient History, Archaeology and History of Art. He has a Ph.D thesis on the carnival tradition and Federico Fellini’s films at the Paris 3 – Sorbonne nouvelle. A key focus of his research is the cognitive approach to film studies. He had in 2018 a Fulbright Grant for research at the Centre for Literature and Mind (University of California at Santa Barbara). Among his books are: The Carnival and Federico Fellini’s Films (2009), The Anatomy of Film (2013), Investigations in Cognitive Film Analysis (2015), and The Kitchen Scenes in the New Romanian Cinema (2017). He recently published a monographical study on the The Control Cycle in Film. Circular Coupling from Brain to Culture (2020). He is hosting the international conference Cinema: Cognition and Arts since 2016, and was editor for the 2018 issue of Ekphrasis dedicated on Cinema and cognition.
IIro P. Jääskeläinen
Online assessment of emotional and cognitive states of movie viewers with neuroimaging: a tool for testing whether the minds of the viewership synchronize with the mind of the director?
Bio: Iiro P. Jääskeläinen is a Full Professor of Systems Neuroscience at Aalto University School of Science, Espoo, Finland. He obtained his PhD in Psychology at the University of Helsinki in 1995, after which he worked as a post-doc and as a junior faculty member at Harvard Medical School, Boston, MA. He joined Helsinki University of Technology, the predecessor of Aalto University, in 2003. Prof. Jääskeläinen’s interests currently encompass the use of movies and narratives as media-based stimuli for elicitation of social cognition and emotions in neuroimaging studies. The publications of Prof. Jääskeläinen can be found in this link: https://scholar.google.fi/citations?hl=en&user=F-i6heMAAAAJ ”
Mehmet Burak Yilmaz
NeuroCine conference is organised as a biannual follow-up seminar for the 2022 biannual international NeuroCine conference at the Baltic Film, Media and Arts School (BFM).
Baltic Film, Media and Arts School (BFM) is a competence centre for communication and audiovisual knowledge and expertise with wide-based international education curricula embedded in interdisciplinary community of Tallinn University . Our Bachelor’s, Master’s and PhD level study program both in Estonian and English offers tools and skills for working on various positions in film production, TV, new media, communication, choreography, art and music.
The NeuroCine research group at Baltic Film, Media, and Arts School is led by the research professor Dr. Pia Tikka and the cinematographer, Dr. Elen Lotman. Its objective is to coordinate ongoing and future research that applies neuroimaging and psychophysiological methods into studying cinematic arts and narrative sense-making under one umbrella. So far most activities that relate to human brain when viewing films (neurocinematics) have taken place within the cognitive and neuroscientific paradigms, while filmmakers and humanist film scholars in general have had minor roles. The NeuroCine team at BFM aims to fix this imbalance.
For the second time NeuroCine at BFM will bring this exciting study field into a practical film school. It gathers together an interdisciplinary group of professionals, scholars, teachers and students from film and media studies, cognitive sciences, social neurosciences, psychophysiology, and phenomenology.
Feel free to address any content related questions to
Header photo by Heikki Leis.