Tag: affective computing

Activities engaging affective computing, art-science, academic and popular events

Aalto MA evaluation on Robots & Poetics

Examination of Johanna Lehto’s MA thesis “Robots and Poetics – Using narrative elements in human-robot interaction” for the Department of Media, programme New Media Design and Production on the 16th of May 2018.

As a writer and a designer, Johanna Lehto sets out to reflect upon the phenomenon of human-robot interaction through her own artistic work. To illustrate plot structure and narrative units of the interaction between a robot and human, she reflects upon how Aristotle’s dramatic principles. In her work, she applies Aristotelian drama structure to analyse a human-robot encounter as a dramatic event. Johanna made an interactive video installation in which she created a presentation of an AI character, Vega 2.0 (image). The installation was exhibited in Tokyo in Hakoniwa-exhibition on 22.-24.6.2017 and in Musashino Art University Open Campus -festival 10.-11.6.2017.

Neurotech drama VFC by Charles S. Roy in Cannes Next May 8-13

News from our international network.

Since connected by Storytek Content+Tech Acceleator in fall 2017 Pia Tikka has consulted the VFC project directed by Charles S. Roy on screenplay and audience interaction.

Charles S. Roy, Film Producer & Head of Innovation at the production company La Maison de Prod, develops his debut narrative film+interactive project VFC as producer-director. VFC has been selected at the Storytek Content+Tech Accelerator, the Frontières Coproduction Market, the Cannes NEXT Cinema & Transmedia Pitch, the Sheffield Crossover Market, and Cross Video Days in Paris.  In the vein of classic portrayals of female anxiety such as Roman Polanski’s REPULSION, Todd Haynes’ SAFE and Jonathan Glazer’s BIRTH, VFC is a primal and immersive psychological drama about fear of music (cinando.com). Its main innovation is in bringing brain-computer interface storytelling to the big screen by offering an interactive neurotech experience.

On the premises of the Cannes Film Market, as a grant holder for the Estonian innovation and development incubator Storytek Accelerator, Charles presented his work to the audience of the tech-focused NEXT section (8-13 May).

 

Testing behavioral strategies for Enactive Avatar

Testing facial expressions of the viewer driving the behavior of a screen character with the Louise’s Digital Double (under a Creative Commons Attribution Non-Commercial No Derivatives 4.0 license). see Eisko.com

In the image Lynda Joy Gerry, Dr. Ilkka Kosunen and Turcu Gabriel, Erasmus exchange student from the  University of Craiova @Digital Technology Institute, TLU) examining facial expressions of a screen character driven by an AI.

“Sümbiootiline helilooming” – New publication by Ilkka Kosunen

The manifesto for the new field of science, Symbiotic Composing, by Dr. Ilkka Kosunen, was recently published in the Teater. Muusika. Kino.

Symbiotic composing connects the topics of deep learning, physiological computing and computational creativity to facilitate new type of creative process where technology and human aesthetic judgment merge into one.

The article is in Estonian language.

 

TMK_aprill_2018

Meeting with Andrew Gordon at the ICT/USC

Andrew Gordon  leads the Narrative Group at the University of Southern California Institute for Creative Technologies where his research is devoted to getting computers to be able to read and generate stories and to use the knowledge in stories in order to become more intelligent. He is also a professor in the USC Department of Computer Science. His project involves reimagining a 70-year-old social science experiment for the digital age.

In 1944, Fritz Heider and Marianne Simmel created a simple animated film depicting the motion of two triangles and a circle as they moved in and around a box that alternated between being opened or closed. Heider and Simmel asked people to describe what they saw. Now a classic work in the field of social psychology, the subjects responded with creative narratives that ascribed human-like goals, plans, beliefs, and emotions to the moving objects. Popular themes included romantic relationships and prison breaks. See also an article by professors of psychology Fritz Heider and Marianne Simmel: “An Experimental Study of Apparent Behavior,” American Journal of Psychology 57.2 (April 1944): 243-59.

ICT’s Andrew Gordon Brings Seminal 1940’s Social Science Experiment Online

Invited young lecture Ilkka Kosunen at Tohoku Forum for Creativity, Tokyo

Image: Ilkka Kosunen presenting as the Invited Young Lecturer at the Tohoku Universal Acoustical Communication Month 2018 October 20.

Seminar on music, sound, speech and artificial intelligence

TOKYO ELECTRON House of Creativity 3F, Lecture Theater, Katahira Campus, Tohoku University [Access]

 

Doctoral Defense: Ilkka Kosunen “Exploring the dynamics of the biocybernetic loop in physiological computing”

ILKKA KOSUNEN: EXPLORING THE DYNAMICS OF THE BIOCYBERNETIC LOOP IN PHYSIOLOGICAL COMPUTING

Enactive Virtuality team member Ilkka Kosunen defended the doctoral dissertation entitled “Exploring the dynamics of the biocybernetic loop in physiological computing” in the Faculty of Science, University of Helsinki, on 22 March 2018 at 12:00. The public examination  took place at the following address: Exactum, D 122, Kumpula. Professor Fairclough Stephen, Liverpool John Moores University, will serve as the opponent, and Professor Jacucci Giulio as the custos.
The dissertation is published in the series Series of publications A / Department of Computer Science, University of Helsinki. The dissertation is also available in electronic form through the E-thesis service.

Neurocinematics – viewing “At Land” by Maya Deren (1944) in brain scanning lab

 

Pia Tikka

The film that all subjects viewed in fMRI and in MEG brain scanning labs at Aalto University  was At Land from 1944 by Maya Deren. I suggested this experimental film for our brain imaging studies due to its special characteristics.

“The dancer, choreographer, and filmmaker Maya Deren can be seen as one of the pioneers of screendance. Her experimental films have challenged conventional plot- driven mainstream cinema by emphasizing an ambiguous experience, open for multiple interpretations. For Deren film viewing is a socially determined ritual embodying intersubjectively shared experiences of participants. This makes her films particularly interesting for today’s neurocinematic studies. Deren’s ideas also anticipate the recent enactive mind approach, according to which the body-brain system is in an inseparable manner situated and coupled with the world through interaction. It assumes that both private, such as perception and cognition, and intersubjective aspects of human enactment, such as culture, sciences, or the arts, are based on the embodiment of life experience. Reflecting this discourse, Deren’s film At Land is analyzed as an expression of a human body-brain system situated and enactive within the world, with references to neuroscience, neurocinematic studies, and screendance” (Tikka & Kaipainen 2016).

Read more of the film and why it is of interest to cognitive neuroscience studies here:

Pia Tikka and Mauri Kaipainen (2016). “Screendance as Enactment in Maya Deren’s At Land: Enactive, Embodied, and Neurocinematic Considerations.” In The Oxford Handbook of Screendance Studies Edited by Douglas Rosenberg. Print Publication Date: Aug 2016 Subject: Music, Dance Online Publication Date: Jun 2016 DOI: 10.1093/oxfordhb/9780199981601.013.15

2015

One of the challenges of naturalistic neurosciences using movie-viewing experiments is how to interpret ob- served brain activations in relation to the multiplicity of time-locked stimulus features. As previous studies have shown less inter-subject synchronization across viewers of random video footage than story-driven films, new methods need to be developed for analysis of less story-driven contents. To optimize the linkage between our fMRI data collected during viewing of a deliberately non-narrative silent film ‘At Land’ by Maya Deren (1944) and its annotated content, we combined the method of elastic-net regularization with the model- driven linear regression and the well-established data-driven independent component analysis (ICA) and inter-subject correlation (ISC) methods.(…)

“In our study, we aimed to go beyond story-driven narratives. What methods could improve the analysis of time-locked interdependence between the brain data and the content of more ambiguous non- narrative films, or video recordings of non-structured events, such as improvised conversation? This question motivated us to study the link- age between our fMRI data collected during viewing of a non-narrative silent film ‘At Land’ directed by Maya Deren (1944, 14′40′′) and its annotated content. The film ‘At Land’ shows an expressionless young woman wandering in her surroundings without any explicit motivation for her behavior, such as collecting stones in the beach, or jumping down from a rock. In addition, according to the director-actress herself, she deliberately avoided emotional expressions, while the cinemato- graphic aspects, such as camera movements and framing, have been carefully composed (Deren, 2005). How to link the cinematic features of such an ambiguous film with the viewers’ brain activation detected while they are trying to make sense of it? As the film ‘At Land’ does not give any tools for inferring the character’s mental state, goals of her actions, or inner reasons, story-based film structure analysis methods do not necessarily allow adequate interpretation for the resulting linkages. Instead, by annotating all bodily actions and camera-related features specifically pointed out by the director of the film, one might find meaningful linkages between the film content and collected fMRI data”(Kauttonen et al 2016).

Read more here: Kauttonen, J., Hlushchuk, Y., and Tikka, P. (2015). “Optimizing methods for linking cinematic features to fMRI data.” NeuroImage 110C:136–148. doi: 10.1016/j.neuroimage.2015.01.063.

2016

Observation of another person’s actions and feelings activates brain areas that support similar functions in the observer, thereby facilitating inferences about the other’s mental and bodily states. In real life, events eliciting this kind of vicarious brain activations are intermingled with other complex, ever-changing stimuli in the environment. One practical approach to study the neural underpinnings of real-life vicarious perception is to image brain activity during movie viewing. Here the goal was to find out how observed haptic events in a silent movie would affect the spectator’s sensorimotor cortex. (…)

Kaisu Lankinen, Eero Smeds, Pia Tikka, Elina Pihko, Riitta Hari, Miika Koskinen (2016) Haptic contents of a movie dynamically engage the spectator’s sensorimotor cortex. Human Brain Mapping.
DOI: 10.1002/hbm.23295

2018

Another article that harnessed our data collected when people watched At Land by Deren (1944):

Movie-viewing allows human perception and cognition to be studied in complex, real-life-like situations in a brain-imaging laboratory. Previous studies with functional magnetic resonance imaging (fMRI) and with magneto- and electroencephalography (MEG/EEG) have demonstrated consistent temporal dynamics of brain activity across movie viewers. However, little is known about the similarities and differences of fMRI and MEG/EEG dynamics during such naturalistic situations. (…)

Below Fig. 6. Time courses of the most similar MEG and fMRI signals.

For detailed information, the reader is referred to the original article: Lankinen K, Saari J, Hlushchuk Y, Tikka P, Parkkonen L, Hari R, Koskinen M. (2018). Consistency and similarity of MEG- and fMRI-signal time courses during movie viewing. NeuroImage. https://www.sciencedirect.com/science/article/pii/S1053811918301423

 

Workshop on Enactive Mind and Media at Politecnico di Milano, 12-16 Feb 2018

 

At the Politecnico Milano, School of Design, professor Pia Tikka (MEDIT/BFM) and Ilkka Kosunen (MEDIT/DTI – in image on the right) are running their workshop week with a group of 40 students of Communication Design.

The topic is as challenging as thrilling: “Enactive Mind and Media”.

In addition to providing the students at Polimi new insights to Enactive media and biosensor-driven interaction design, the workshop also contributes to the international dissemination strategy of the EU Mobilitas Pluss Top Research project by Dr. Pia Tikka. Ilkka Kosunen’s travel is funded by Erasmus Mobility, TLU. The workshop utilizes (DTI) Digital Technology Institute’s biosensor technology and expertise. During the workshop also contacts with the local hi-tech researchers and industry are created.

A great news is that the bilateral agreement of Erasmus Mobility for Staff Exchange has been established between Tallinn University BFM and Politecnico Milano School of Design starting 2018 January. The agreement was initiated at TLUby Enactive Virtuality Lab.