Click here to find out about our laboratories
Click here to find out about our sponsors.
Our research deals with visual and haptic information processing, including colors, textures, materials, objects and natural scenes. Our laboratories have measuring instruments and experimental setups for studies of visual and haptic perception as well as sensorimotor coordination. This includes state-of-the-art equipment for measuring eye movements (EyeLink 1000, Tobii Eye Tracker 4c) and other motor movements (Optotrak-3020 System, Zebris Tracking System, Qualisys Motion Capture System), for manipulating visual-proprioceptive information (PHANToM-force feedback device) and for creating virtual environments (HTC Vive). In addition to psychophysical measurements, a major focus of our department is the modeling of perception and behavior. For this purpose we also have state-of-the-art systems for computational modeling and machine learning at our disposal.
Our department is part of national and international collaborations and networks that are, for example, focused on investigating the most important aspects of human perception (Collaborative Research Center Cardinal Mechanisms of Perception) or multisensory integration in a dynamic environment (Research Training Network Dyvito).
Perception of material qualities
A crucial task of the human visual system is to determine the material that an object is made of. Correct identification of material qualities affects basic decisions such as whether food is edible, skin is healthy, or whether an object is pliable. Humans perceive these qualities in a split second. Yet, little is known about how the brain recognizes materials. Using a combination of psychophysics, image analysis, computational modeling, eye -/ hand tracking, and fMRI we investigate questions such as: What information does the brain use to estimate and categorize material qualities? How do expectations about materials affect how we perceive them, look at them or interact with them? Does freely interacting with a material change how it feels or appears?
Schmid, A.C., Barla, P. & Doerschner, K. (2020). Material category determined by specular reflection structure mediates the processing of image features for perceived gloss. bioRxiv. DOI
- Schmid, A.C., Boyaci, H. & Doerschner, K. (2020). Dynamic dot displays reveal material motion network in the human brain. bioRxiv. DOI
- Schmid, A.C. & Doerschner, K. (2019). Representing stuff in the human brain. Current Opinion in Behavioral Sciences, 30, pp.178-185. DOI
- Toscani, M., Yucel, E. & Doerschner, K. (2019). Gloss and speed judgements yield different fine tuning of saccadic sampling in dynamic scenes. i-Perception, 10(6): 2041669519889070. DOI
- Cavdan, M., Doerschner, K. & Drewing, K. (2019). The many dimensions underlying perceived softness: How exploratory procedures are influences by material and the perceptual task. IEEE World Haptic Conference. DOI
Computational neuroscience of visual recognition
From a brief glimpse of a complex scene, we recognize people and objects, their relationships to each other, and the overall gist of the scene – all within a few hundred milliseconds and with no apparent effort. What are the computations underlying this remarkable ability and how are they implemented in the brain? To address these questions, our research bridges recent advances in machine learning with human behavioral and neural data to provide a computationally precise account of how visual recognition works in humans. Specifically, we leverage the recent successes in artificial neural networks of vision to address long-standing questions concerning the functional organization of the human visual cortex, such as:
- Why do we have functional specialization in the human visual cortex?
- Is visual experience necessary to form functional specificity?
- Which aspects of visual recognition are idiosyncrasies of the human brain and which arise in any system optimized for visual recognition?
Dobs, K., Martinez, J., Kell, A. J. E., & Kanwisher, N. (2022). Brain-like functional specialization emerges spontaneously in deep neural networks. Science Advances, 8, eabl8913. [DOI]
Dobs, K., Isik, L., Pantazis, D., & Kanwisher, N. (2019). How face perception unfolds over time. Nature communications, 10, 1258. DOI
Dobs, K., Ma, W. J., & Reddy, L. (2017). Near-optimal integration of facial form and motion. Scientific Reports, 7(1):11002, 1-9. DOI
Perception & Action (PerAct) Lab
The PerAct Lab studies how humans use sensory information to guide their actions, focusing on two themes: spatial coding for action and sensorimotor predictions.
Spatial coding for action. To successfully interact with the environment, the human brain needs to build up a representation of where the action goal is located in space. We are interested in how humans spatially represent targets for actions and how they use this spatial information to plan and control eye and hand movements, e.g., directing gaze to a reach goal or grasping an object. Experiments in virtual reality allow us to expand our research on spatial coding for action from small-scale, static to large-scale, dynamic environments.
Sensorimotor predictions. As processing of sensory information takes time, predictive mechanisms can help us to overcome these delays and to prepare our motor system to react at the right place to the right time, e.g. predicting the upcoming position of our moving arm when catching a ball helps us to quickly adjust our arm movement if the ball's trajectory suddenly changes. We investigate how well humans can establish and use sensorimotor predictions, and how these predictions influence the processing of sensory feedback. We believe that feedback signals are up- and down-weighted depending on the accuracy of sensorimotor predictions.
For our research, we use a variety of state-of-the-art methods in real and virtual environments, ranging from eye, hand and whole-body movement tracking to psychophysics and functional neuroimaging (fMRI).
Fiehler, K., Brenner, E., & Spering, M. (2019). Prediction in goal-directed action. Journal of Vision, 19(9), 10, 1-21. DOI
Gertz, H. & Fiehler, K. (2015). Human posterior parietal cortex encodes the movement goal in a pro-/anti-reach task. Journal of Neurophysiology, 114, 170-183. DOI
Karimpur, H., Morgenstern, Y., & Fiehler, K. (2019) Facilitation of allocentric coding by virtue of object-semantics. Scientific Reports, 9, 6263. DOI
Klinghammer, M., Blohm, G., & Fiehler, K. (2015). Contextual factors determine the use of allocentric information for reaching in a naturalistic scene. Journal of Vision, 15(13), 24. DOI
Voudouris D., & Fiehler K. (2017). Enhancement and suppression of tactile signals during reaching. Journal of Experimental Psychology: Human Perception and Performance, 43(6), 1238-1248. DOI
Visual Perception of the Physical Properties of Objects
Research in my lab focusses on how the brain visually estimates the physical and functional properties of objects in our surroundings. When we look at things, we don't experience the world as a meaningless jumble of lines, colours or motions. Instead, whenever we open our eyes, we immediately gain access to a richly detailed world of meaningful visual sensations. We recognise objects; perceive what things are made of; identify risk and pleasures and can even work out how objects might respond to forces or actions. Based on how things look, we are able to make a remarkable range of subtle judgements about the physical properties of objects, such as whether food is fresh or stale or whether an object is stable or likely to topple over. Without touching an object, we can usually work out what it would feel like were we to reach out and touch it, based on the curves and contours of its shape and the way light plays across its surface. My research program aims to understand how the brain estimates the 3D shape of surfaces, and the material properties of objects such as elasticity, translucency or viscosity. In order to do this, we use a combination of computer graphics, image analysis techniques, neural modelling and psychophysical experiments.
Phillips F* & RW Fleming* (2020). The Veiled Virgin illustrates visual segmentation of shape by cause. Proceedings of the National Academy of Sciences 201917565 (*the authors contributed equally to this work) DOI
Fleming RW & KR Storrs (2019) Learning to See Stuff. Current Opinion in Behavioral Sciences. 30: 100–108. DOI
Van Assen, JJ, Barla P & Fleming, RW (2018). Visual Features in the Perception of Liquids. Current Biology 28(3), 452–458. DOI
Fleming RW (2017). Material Perception. Annual Reviews of Vision Science 3(1). 365-388. DOI
Individual differences in perception
Dr. Ben de Haas
Visual perception feels like an objective window to the world. When we claim to have seen something ‘with our own eyes’, we mean to imply the objective certainty of what we saw, rather than the subjective nature of our impressions. Nevertheless, our perception of the world isn’t always the same. We can look at the same image, but move our eyes to different parts of it; look at the same face and disagree about its trustworthiness; and sometimes even look at the same object and disagree about its colour.
Our research aims to understand more about three related questions:
1) How does perception vary from one person to the next?
2) What are the mechanisms behind these differences?
3) How do they shape who we are and how we interact with others?
To investigate these questions, we use eyetracking, psychophysics and neuroimging in healthy and clinical populations. More information can be found here.
de Haas, B., Iakovidis, A. L., Schwarzkopf, D. S., & Gegenfurtner, K. R. (2019). Individual differences in visual salience vary along semantic dimensions. Proceedings of the National Academy of Sciences, 116(24), 11687-11692. DOI
Moutsiana, C., de Haas, B., Papageorgiou, A., Van Dijk, J. A., Balraj, A., Greenwood, J. A., & Schwarzkopf, D. S. (2016). Cortical idiosyncrasies predict the perception of object size. Nature communications, 7(1), 1-12. DOI
de Haas, B., Kanai, R., Jalkanen, L., & Rees, G. (2012). Grey matter volume in early human visual cortex predicts proneness to the sound-induced flash illusion. Proceedings of the Royal Society B: Biological Sciences, 279(1749), 4955-4961. DOI
Prof. Karl Gegenfurtner, Ph.D. , Dr. Thorsten Hansen
The perception of color is a central component of primate vision. Colour facilitates object perception and recognition, and has an important role in scene segmentation and visual memory. Despite the long history of colour vision studies, much there is still much to be learned about the physiological basis of colour perception. Recent studies are beginning to indicate that colour is processed not in isolation, but together with information about luminance and visual form to achieve a unitary and robust representation of the visual world.
- Hansen, T., & Gegenfurtner, K. R. (2006). Higher level chromatic mechanisms for image segmentation. Journal of Vision, 6(3), 239\u2013259. [PDF]
- Hansen, T., Olkkonen, M., Walter, S. & Gegenfurtner, K. R. (2006). Memory modulates color appearance. Nature Neuroscience, 9(11), 1367–1368.
- Gegenfurtner, K.R. (2003) Cortical mechanisms of colour vision. Nature Reviews Neuroscience, 4, 563–572. [PDF]
- Gegenfurtner, K.R. & Kiper, D.C. (2003) Color vision. Annual Review of Neuroscience, 26, 181–206.[PDF]
Haptic and Multisensory Perception
Perception is an active process during that we purposively gather sensory information. Haptic perception is the prime example for this principle. For example, when we aim to haptically judge an object's softness, we first will have to appropriately explore the object in order to obtain the relevant sensory information. Often a single indentation of the object is not sufficient, but we repeatedly indent the object before we deliver a judgment.
In the HapLab we study how humans control natural explorations in active touch, and how information of different type is integrated into a percept of the stimulus. We found, for example, that humans perceive objects to be softer, when they already believe that these objects are relatively soft (Metzger & Drewing, 2019). We also modeled the integration of serially gathered texture information over the course of the exploration, using an ideal observer model (Lezkan, Metzger & Drewing, 2018). Other results suggest that people fine-tune their exploratory movements in order to optimize perception (Zoeller, Lezkan, Paulun, Fleming, & Drewing, 2019). Further research foci in the HapLab are on different dimensions of softness (Cavdan, Doerschner, & Drewing, 2019), haptic saliency and haptic search, multisensory integration (i.e. size-weights illusion), signals to haptic perception of environmental properties (time, space, softness, texture, shape) and on emotional effects of haptic stimulation (Drewing, Weyel, Celebi, & Kaya, 2018).
Typical visuo-haptic VR setup, 3D-printed and custom-made stimuli
The main competences of the lab are on behavioral and perceptual aspects of haptic and multisensory signal processing and of associated movement control. Presently we use in the first place a visuo-haptic VR/AR-setup including a force-feedback device, force sensors and a stereo display, a high-resolution 3D printer (Stratasys Objet Pro), vibrotactile actuators and various custom-made stimuli.
Cavdan, M., Doerschner, K., & Drewing, K. (2019). The Many Dimensions Underlying Perceived Softness: How Exploratory Procedures are Influenced by Material and the Perceptual Task. IEEE World Haptics Conference, WHC 2019 (pp. 437-442), IEEE. DOI
Drewing, K., Weyel, C., Celebi, H., & Kaya, D. (2018). Systematic Relations between Affective and Sensory Material Dimensions in Touch. IEEE Transactions on Haptics 11(4), 611-622. DOI
Lezkan, A., & Drewing, K. (2018). Processing of haptic texture information over sequential exploration movements. Attention, Perception, & Psychophysics, 80(1), 177-192. DOI
Metzger, A., & Drewing, K. (2019). Memory influences haptic perception of softness. Scientific Reports, 9(1), 1-10. DOI
Zoeller, A. C., Lezkan, A., Paulun, V. C., Fleming, R. W., & Drewing, K. (2019). Integration of prior knowledge during haptic exploration depends on information type. Journal of Vision, 19(4), 1-15. DOI
Perception and eye movements in natural scenes
Prof. Karl Gegenfurtner
We study the principles underlying the selection of fixation targets under natural viewing conditions. We study fixation patterns and saccadic latencies of human subjects viewing under natural images and videos of natural scenes and ask how stimulus features like contrast, color and spatial frequency content interact with top-down mediated expectations.
- Thorpe, S., Gegenfurtner, K.R., Fabre-Thorpe, M. & Bülthoff, H.H. (2001) Detection of animals in natural images using far peripheral vision. European Journal of Neuroscience, 14, 869-876. <Get PDF file>
- Gegenfurtner, K.R. & Rieger, J. (2000) Sensory and cognitive contributions of color to the perception of natural scenes.Current Biology, 10, 805-808. <Get PDF file>
Eye movements and Visual Perception
Humans frequently move their eyes, either to fixate a new location in the visual field (saccadic eye movements), or to keep fixation on a moving object (smooth pursuit eye movements). These eye movements pose two problems. First, an appropriate target location and execution time has to be selected for the eye movements. Hence we study, which visual signals are used to guide these eye movements, i.e. how visual perception influences the control of eye movements. Second, the execution of eye movements changes the visual image on the retina. To maintain a clear and stable perception of the world, the visual system has to cope with the retinal image motion. In this context we study how visual perception is affected by the execution of concurrent eye movements.
Our experimental approach comprises psychophysics measurements under simultaneous tracking of eye movements to investigate the bidirectional relationship between perception and eye movements.
- Schütz, A.C., Braun, D.I., Kerzel, D. & Gegenfurtner, K.R. (2008) Improved visual sensitivity during smooth pursuit eye movements. Nature Neuroscience, 11, 1211-1216. DOI
- Spering, M. & Gegenfurtner, K.R. (2008). Contextual effects on motion perception and smooth pursuit eye movements. Brain Research, 1225, 76-85. DOI
- White, B.J., Stritzke, M. & Gegenfurtner, K.R. (2008) Saccadic facilitation in natural backgrounds. Current Biology, 18, 124-128. DOI
Visually guided motor behavior
We investigate the complex mechanisms involved in interactions of humans with the environment. The versatility of the human visuo-motor system can be seen in the ease with which we perform everyday tasks such as reaching and grasping for objects under varying visual input. For example, we can easily grasp fragile objects like eggs (we might even learn to juggle them), or we might learn to adapt quickly to the distortions introduced by wearing left-right reversing prisms, etc. On the other hand, it is still very difficult to devise technical systems which are capable of only a subset of the capabilities of the human motor system.
One of the questions we have been studying intensively during recent years is whether the visual guidance of motor behavior is achieved by different processes (and neuronal substrates) as our conscious (visual) perception. Studies on neurological patients suggest such a division of labor in the human brain and it was suggested that this dissociation between vision-for-action and vision-for-perception can also be found in healthy humans. Support for this view came from studies which found that grasping is less affected by visual illusions than perception. Our results, to the contrary, suggest that the motor system uses very similar processes and neuronal signals as visual perception. This suggests that the brain is more coherent than currently proposed by a number of theories in visual neuroscience.
- V. H. Franz. Planning versus online control: Dynamic illusion effects in grasping? Spatial Vision, 16(3-4):211 - 223, 2003. [PDF]
- V. H. Franz. Action does not resist visual illusions. Trends in Cognitive Sciences, 5(11):457 - 459, 2001. [PDF]
- V. H. Franz, K. R. Gegenfurtner, H. H. Bülthoff, and M. Fahle. Grasping visual illusions: No evidence for a dissociation between perception and action. Psychological Science, 11(1):20 - 25, 2000. [PDF]