Andrew Gordon leads the Narrative Group at the University of Southern California Institute for Creative Technologies where his research is devoted to getting computers to be able to read and generate stories and to use the knowledge in stories in order to become more intelligent. He is also a professor in the USC Department of Computer Science. His project involves reimagining a 70-year-old social science experiment for the digital age.
In 1944, Fritz Heider and Marianne Simmel created a simple animated film depicting the motion of two triangles and a circle as they moved in and around a box that alternated between being opened or closed. Heider and Simmel asked people to describe what they saw. Now a classic work in the field of social psychology, the subjects responded with creative narratives that ascribed human-like goals, plans, beliefs, and emotions to the moving objects. Popular themes included romantic relationships and prison breaks. See also an article by professors of psychology Fritz Heider and Marianne Simmel: “An Experimental Study of Apparent Behavior,” American Journal of Psychology 57.2 (April 1944): 243-59.
Pia Tikka and Martin Jaroszewicz gave a joint talk at the CNA Sound and Storytelling Conference.
Image: Dr. Martin Jaroszewicz disucssing his ideas of enactive VR soundscapes at the Chapman University conference, Orange, CA, March 22.
“Enactive Virtuality – a framework for dynamically adaptive soundscapes”.
The novel notion of enactive virtuality is discussed, drawing from the theories of enactive mind , and the concept of enactive cinema . The key attribute of ‘enactive’ refers here to the setting in which the human agent is in continuous feed-back-looped interaction with the surrounding world. Enactive virtuality in turn refers to the story emerging in the agent’s mind in this dynamical setting in order to make sense of the world. This story is based on the agent’s current situation and previous experiences , and in relation to others through neurally built-in imitation of their actions . Thus, the concept of ‘enactive virtuality’ extends beyond the common techno-spatial buzzword of ‘virtual reality’. While virtual reality technologies allow platforms where the perception of sound events can be modulated algorithmically, we describe the human agent’s experience of a dynamically adaptive soundscape as an expression of enactive virtuality.Theories and techniques of sound transformations in the spectral domain [5,6,7] for this setting are discussed.
 Varela F, Thompson E, Rosch E. 1991. Embodied Mind. Cambridge, MA: MIT Press.
 Tikka P. 2008. Enactive Cinema: Simulatorium Eisensteinense. PhD diss. Helsinki: Univ. Art and Design Publ.
 Heyes CM, Frith CD. The cultural evolution of mind reading. Science. 2014 Jun 20;344(6190):1243091. doi:10.1126/science.1243091.
 Gallese V, Eagle MN, Migone P. 2007. Intentional attunement: mirror neurons (…) J Am Psychoanal Assoc. 55(1):131-76.
 Jaroszewicz, M. 2015 Compositional Strategies in Spectral Spatialization. PhD thesis,University of California Riverside,
 Jaroszewicz, M. 2017 ”Interfacing Gestural Data from Instrumentalists,” ART – Music Review, vol. 32.
 Kim-Boyle. 2008. “Spectral spatialization – an overview,” RILM Abstracts of Music Literature. http://hdl.handle.net/2027/spo.bbp2372.2008.086
(Seminar in Finnish language)
Keväällä 2017 Helsingin Sanomain Säätiö järjesti suunnatun haun teemana Totuudenjälkeinen aika. Säätiön hallitus myönsi yli puoli miljoonaa euroa kymmenelle hankkeelle. Mitä hankkeille kuuluu nyt? Mitä ilmiöstä on saatu selville? Miten tästä eteenpäin?
Totuudenjälkeinen aika – nyt -seminaariin maanantaina 19.3. klo 16–18 Päivälehden museoon (Ludviginkatu 2–4).
Politiikka ja distruptio. Toimitusjohtaja Taru Tujunen, Ellun Kanat
Poliittiset kuplat ja media. Dosentti, yliopisto-opettaja Arttu Saarinen, Turun yliopisto
Miten media selviää polarisaation ja paskapuheen aikana? Dosentti, vanhempi tutkija Antto Vihma, Ulkopoliittinen instituutti
Alustusten jälkeen paneelikeskustelu, jota vetää toimittaja Johanna Vehkoo.
Lisää hankkeista: www.totuudenjälkeinen.fi
Helsingin Sanomain Säätiö on yksityinen, yleishyödyllinen säätiö. Sen tehtävänä on suomalaisen median ja laatujournalismin tulevaisuuden turvaaminen ja sananvapauden tukeminen. Säätiö jakaa apurahoja viestintäalan tutkimukseen ja koulutukseen sekä alan kilpailuihin. Helsingin Sanomain Säätiö ylläpitää Päivälehden museota ja Päivälehden arkistoa.
Janne Kauttonen, FT, researcher, NeuroLab, Laurea-ammattikorkeakoulu
Jenni Hannukainen, FM, researcher, NeuroLab, Laurea-ammattikorkeakoulu
Jyrki Suomala, KT (dosentti), Lecturer, NeuroLab, Laurea-ammattikorkeakoulu
Pia Tikka, TaT (dosentti), EU Mobilitas Pluss research professor, University of Tallinn
See full program here
At 12:15 session Pia Tikka, Research Professor, Baltic Film, Media, Arts and Communication School “Neurocinematics”
My 13 minutes will introduce a multidisciplinary research paradigm of neurocinematics. Combining methods of cinema, enactive media, and virtual screen characters with those of cognitive sciences it allows us to unravel new aspects of the neural basis of storytelling, creative imagination, and narrative comprehension. In addition to contributing to academic research on human mind, neurocinematics contributes to a range of more specifically targeted goals, such as the impact of audiovisual media on its audience for artistic, therapeutic, or commercial implementations, to name few of many.
Tallinn University is celebrating Brain Awareness Week (BAW) for the second year in a row! This year we have an open seminar
“Meet the Researchers”
Monday, March 12 from 2PM to 6 PM “Meet the Researchers”, University of Tallinn (Narva mnt. 29), room A-325.
Take part of a seminar where experts from the field take the floor and discuss brain, promote the benefits of brain research and inform the public about what we do as neuroscientist and brain researchers.
The film that all subjects viewed in fMRI and in MEG brain scanning labs at Aalto University was At Land from 1944 by Maya Deren. I suggested this experimental film for our brain imaging studies due to its special characteristics.
“The dancer, choreographer, and filmmaker Maya Deren can be seen as one of the pioneers of screendance. Her experimental films have challenged conventional plot- driven mainstream cinema by emphasizing an ambiguous experience, open for multiple interpretations. For Deren film viewing is a socially determined ritual embodying intersubjectively shared experiences of participants. This makes her films particularly interesting for today’s neurocinematic studies. Deren’s ideas also anticipate the recent enactive mind approach, according to which the body-brain system is in an inseparable manner situated and coupled with the world through interaction. It assumes that both private, such as perception and cognition, and intersubjective aspects of human enactment, such as culture, sciences, or the arts, are based on the embodiment of life experience. Reflecting this discourse, Deren’s film At Land is analyzed as an expression of a human body-brain system situated and enactive within the world, with references to neuroscience, neurocinematic studies, and screendance” (Tikka & Kaipainen 2016).
Read more of the film and why it is of interest to cognitive neuroscience studies here:
Pia Tikka and Mauri Kaipainen (2016). “Screendance as Enactment in Maya Deren’s At Land: Enactive, Embodied, and Neurocinematic Considerations.” In The Oxford Handbook of Screendance Studies Edited by Douglas Rosenberg. Print Publication Date: Aug 2016 Subject: Music, Dance Online Publication Date: Jun 2016 DOI: 10.1093/oxfordhb/9780199981601.013.15
One of the challenges of naturalistic neurosciences using movie-viewing experiments is how to interpret ob- served brain activations in relation to the multiplicity of time-locked stimulus features. As previous studies have shown less inter-subject synchronization across viewers of random video footage than story-driven films, new methods need to be developed for analysis of less story-driven contents. To optimize the linkage between our fMRI data collected during viewing of a deliberately non-narrative silent film ‘At Land’ by Maya Deren (1944) and its annotated content, we combined the method of elastic-net regularization with the model- driven linear regression and the well-established data-driven independent component analysis (ICA) and inter-subject correlation (ISC) methods.(…)
“In our study, we aimed to go beyond story-driven narratives. What methods could improve the analysis of time-locked interdependence between the brain data and the content of more ambiguous non- narrative films, or video recordings of non-structured events, such as improvised conversation? This question motivated us to study the link- age between our fMRI data collected during viewing of a non-narrative silent film ‘At Land’ directed by Maya Deren (1944, 14′40′′) and its annotated content. The film ‘At Land’ shows an expressionless young woman wandering in her surroundings without any explicit motivation for her behavior, such as collecting stones in the beach, or jumping down from a rock. In addition, according to the director-actress herself, she deliberately avoided emotional expressions, while the cinemato- graphic aspects, such as camera movements and framing, have been carefully composed (Deren, 2005). How to link the cinematic features of such an ambiguous film with the viewers’ brain activation detected while they are trying to make sense of it? As the film ‘At Land’ does not give any tools for inferring the character’s mental state, goals of her actions, or inner reasons, story-based film structure analysis methods do not necessarily allow adequate interpretation for the resulting linkages. Instead, by annotating all bodily actions and camera-related features specifically pointed out by the director of the film, one might find meaningful linkages between the film content and collected fMRI data”(Kauttonen et al 2016).
Read more here: Kauttonen, J., Hlushchuk, Y., and Tikka, P. (2015). “Optimizing methods for linking cinematic features to fMRI data.” NeuroImage 110C:136–148. doi: 10.1016/j.neuroimage.2015.01.063.
Observation of another person’s actions and feelings activates brain areas that support similar functions in the observer, thereby facilitating inferences about the other’s mental and bodily states. In real life, events eliciting this kind of vicarious brain activations are intermingled with other complex, ever-changing stimuli in the environment. One practical approach to study the neural underpinnings of real-life vicarious perception is to image brain activity during movie viewing. Here the goal was to find out how observed haptic events in a silent movie would affect the spectator’s sensorimotor cortex. (…)
Kaisu Lankinen, Eero Smeds, Pia Tikka, Elina Pihko, Riitta Hari, Miika Koskinen (2016) Haptic contents of a movie dynamically engage the spectator’s sensorimotor cortex. Human Brain Mapping.
Another article that harnessed our data collected when people watched At Land by Deren (1944):
Movie-viewing allows human perception and cognition to be studied in complex, real-life-like situations in a brain-imaging laboratory. Previous studies with functional magnetic resonance imaging (fMRI) and with magneto- and electroencephalography (MEG/EEG) have demonstrated consistent temporal dynamics of brain activity across movie viewers. However, little is known about the similarities and differences of fMRI and MEG/EEG dynamics during such naturalistic situations. (…)
Below Fig. 6. Time courses of the most similar MEG and fMRI signals.
For detailed information, the reader is referred to the original article: Lankinen K, Saari J, Hlushchuk Y, Tikka P, Parkkonen L, Hari R, Koskinen M. (2018). Consistency and similarity of MEG- and fMRI-signal time courses during movie viewing. NeuroImage. https://www.sciencedirect.com/science/article/pii/S1053811918301423
You can find an article about prof Pia Tikka and her studies on enactive co-presence between the viewer and screen character in the Nordica in flight magazine “Time Flies”.
You can also read the article in BFM’s blog: