Contributions

Peer-reviewed manuscripts

2024

1. 

   Artificial intelligence meets body sense: Task-driven neural networks reveal computational principles of the proprioceptive pathway

   van Dyck, L. E., Bremmer, F., and Dobs, K.

   Signal Transduction and Targeted Therapy

   [Open Access]

   Abs DOI

   In a recent study published in Cell, Marin Vargas and Bisi et al. present an innovative approach to unravel the computational principles underlying proprioceptive processing in non-human primates. Their findings showcase the utility of task-driven modeling in advancing neuroscience and offer translational potential by providing seminal insights into the goals and mechanisms by which the brain encodes body position and movements.

2023

1. 

   Modeling biological face recognition with deep convolutional neural networks

   van Dyck, L. E., and Gruber, W. R.

   Journal of Cognitive Neuroscience

   [Open Access]

   Abs DOI

   Deep convolutional neural networks (DCNNs) have become the state-of-the-art computational models of biological object recognition. Their remarkable success has helped vision science break new ground, and recent efforts have started to transfer this achievement to research on biological face recognition. In this regard, face detection can be investigated by comparing face-selective biological neurons and brain areas to artificial neurons and model layers. Similarly, face identification can be examined by comparing in vivo and in silico multidimensional “face spaces.” In this review, we summarize the first studies that use DCNNs to model biological face recognition. On the basis of a broad spectrum of behavioral and computational evidence, we conclude that DCNNs are useful models that closely resemble the general hierarchical organization of face recognition in the ventral visual pathway and the core face network. In two exemplary spotlights, we emphasize the unique scientific contributions of these models. First, studies on face detection in DCNNs indicate that elementary face selectivity emerges automatically through feedforward processing even in the absence of visual experience. Second, studies on face identification in DCNNs suggest that identity-specific experience and generative mechanisms facilitate this particular challenge. Taken together, as this novel modeling approach enables close control of predisposition (i.e., architecture) and experience (i.e., training data), it may be suited to inform long-standing debates on the substrates of biological face recognition.

2022

1. 

   Guiding visual attention in deep convolutional neural networks based on human eye movements

   van Dyck, L. E., Denzler, S. J., and Gruber, W. R.

   Frontiers in Neuroscience

   [Open Access]

   Abs DOI

   Deep Convolutional Neural Networks (DCNNs) were originally inspired by principles of biological vision, have evolved into best current computational models of object recognition, and consequently indicate strong architectural and functional parallelism with the ventral visual pathway throughout comparisons with neuroimaging and neural time series data. As recent advances in deep learning seem to decrease this similarity, computational neuroscience is challenged to reverse-engineer the biological plausibility to obtain useful models. While previous studies have shown that biologically inspired architectures are able to amplify the human-likeness of the models, in this study, we investigate a purely data-driven approach. We use human eye tracking data to directly modify training examples and thereby guide the models’ visual attention during object recognition in natural images either toward or away from the focus of human fixations. We compare and validate different manipulation types (i.e., standard, human-like, and non-human-like attention) through GradCAM saliency maps against human participant eye tracking data. Our results demonstrate that the proposed guided focus manipulation works as intended in the negative direction and non-human-like models focus on significantly dissimilar image parts compared to humans. The observed effects were highly category-specific, enhanced by animacy and face presence, developed only after feedforward processing was completed, and indicated a strong influence on face detection. With this approach, however, no significantly increased human-likeness was found. Possible applications of overt visual attention in DCNNs and further implications for theories of face detection are discussed.

2021

1. 

   Comparing object recognition in humans and deep convolutional neural networks—an eye tracking study

   van Dyck, L. E., Kwitt, R., Denzler, S. J., and Gruber, W. R.

   Frontiers in Neuroscience

   [Open Access]

   Abs DOI

   Deep convolutional neural networks (DCNNs) and the ventral visual pathway share vast architectural and functional similarities in visual challenges such as object recognition. Recent insights have demonstrated that both hierarchical cascades can be compared in terms of both exerted behavior and underlying activation. However, these approaches ignore key differences in spatial priorities of information processing. In this proof-of-concept study, we demonstrate a comparison of human observers (N = 45) and three feedforward DCNNs through eye tracking and saliency maps. The results reveal fundamentally different resolutions in both visualization methods that need to be considered for an insightful comparison. Moreover, we provide evidence that a DCNN with biologically plausible receptive field sizes called vNet reveals higher agreement with human viewing behavior as contrasted with a standard ResNet architecture. We find that image-specific factors such as category, animacy, arousal, and valence have a direct link to the agreement of spatial object recognition priorities in humans and DCNNs, while other measures such as difficulty and general image properties do not. With this approach, we try to open up new perspectives at the intersection of biological and computer vision research.

2020

1. 

   Predicting engagement in electronic surveillance in romantic relationships

   Hertlein, K. M., and van Dyck, L. E.

   Cyberpsychology, Behavior, and Social Networking

   DOI

Conference proceedings

2024

1. Core neural dimensions of functionally selective areas in the human visual cortex

   van Dyck, L. E., Hebart, M. N., and Dobs, K.

   In Cognitive Computational Neuroscience (CCN)
   Boston, MA, USA

   Abs DOI

   Prior research has extensively documented functional selectivity for categories within visual cortical areas, primarily by contrasting neural responses to images from various categories. However, such categorical approaches are less suitable to capture the diversity of neural representations within these areas. Do category-selective areas encode holistic categories, or are they instead tuned to multifaceted features? To address this question, we employed non-negative matrix factorization (NMF) for analyzing human fMRI responses to natural images in face-, body-, and scene-selective areas, which uncovered a consistent set of interpretable neural dimensions across participants. These dimensions not only aligned with the areas’ respective category preferences, but also revealed finer within-category distinctions, indicating selective tuning to diverse visual input features. Mapping these dimensions onto the cortical surface showed both clustered and distributed topographies, which accounted for overlaps between areas. Our results suggest that category-selective areas show multifaceted feature tuning, challenging traditional views and highlighting the complex interplay of neural dimensions in encoding visual information.

2. Neural representational dimensions capture the nested functional organization of the human visual cortex

   van Dyck, L. E., Hebart, M. N., and Dobs, K.

   In SFB Workshop Categorization in Perception and Action: Minds, Models, Mechanisms
   Marburg, Germany
3. Core neural dimensions of functionally selective areas in the human visual cortex

   van Dyck, L. E., Dobs, K., and Hebart, M. N.

   In Workshop on CONCEPTS, ACTIONS, and OBJECTS (CAOs)
   Rovereto, Italy

2022

1. Analyzing and increasing the similarity of humans and deep convolutional neural networks in object recognition

   van Dyck, L. E., Denzler, S. J., and Gruber, W. R.

   In European Conference on Visual Perception (ECVP)
   Nijmegen, The Netherlands
2. Spatial similarities between human eye movements and deep convolutional neural network saliency maps across time

   van Dyck, L. E., Denzler, S. J., Schöllkopf, C. P., and Gruber, W. R.

   In Salzburg Mind-Brain Annual Meeting (SAMBA)
   Salzburg, Austria

Theses

2023

1. 

   Unraveling top-down and bottom-up processes in theory of mind with layer fMRI

   van Dyck, L. E.

   University of Salzburg, Department of Psychology

   [Open Access]

   Abs PDF

   Theory of Mind (ToM), the ability to reason about the mental states of others, is a crucial social skill of humans. Neuroimaging studies have found that the Temporo-Parietal Junction (TPJ) and an entire network of additional brain regions are specifically activated during mental state reasoning. Two common tasks that are used to investigate ToM are the False Belief (FB) task and the Social Animations (SA) task, which are thought to activate the posterior part (pTPJ) and the anterior part (aTPJ) of TPJ, respectively. While previous research has suggested potential explanations for this functional specialization, the exact mechanisms are not yet fully understood. In this study, high-resolution layer fMRI was used to examine neural activity in cortical layers of pTPJ and aTPJ during the FB and SA tasks. Firstly, the results corroborate the importance of the ToM network and especially TPJ in mental state reasoning. Secondly, a significant interaction between task and region was revealed, which underlines the expected functional specialization of TPJ clusters. Thirdly, the layer profiles of the two tasks indicated feedback-like activity, but when separated by region, pTPJ showed feedback-like activity for the FB task, while aTPJ displayed feedforward-like activity for the SA task. This pattern was further confirmed by a hierarchical cluster analysis. Overall, these findings suggest that the functional specialization, which is even reflected at the level of cortical layers, may enable TPJ to switch between detecting social cues externally and contemplating about them internally.

2020

1. 

   Seeing eye-to-eye? A comparison of object recognition performance in humans and deep convolutional neural networks under image manipulation

   van Dyck, L. E.

   University of Salzburg, Department of Psychology

   [Open Access]

   DOI

Science communication

2023

1. 

   Modellierung der biologischen Gesichtswahrnehmung mit Künstlicher Intelligenz

   van Dyck, L. E., and Dobs, K.

   DER AUGENSPIEGEL