The integration of multisensory information is an essential mechanism in perception and action control. Research in multisensory integration is concerned with how the information from the different sensory modalities, such as the senses of vision, hearing, smell, taste, touch, and proprioception, are integrated to a coherent representation of objects (for an overview, see e.g., Calvert et al., 2004). The combination of information from the different senses is central for action control. For instance, when you grasp for a rubber duck, you can see its size, feel its compliance and hear the sound it produces. Moreover, identical physical properties of an object can be provided by different senses. You can both see and feel the size of the rubber duck. Even when you grasp for the rubber duck with a tool (e.g., with tongs), the information from the proximal hand, from the effective part of the distal tool and from the eyes are integrated in a manner to act successfully (for limitations of this integration see Sutter et al., 2013). Sutter, C., Drewing, K., & Müsseler, J. (2014). Multisensory integration in action control. Frontiers in psychology, 5, 544.
Perception during active touch essentially depends on the executed exploratory movements. Humans use different movement schemes to perceive different haptic properties, the so-called exploratory procedures (EPs). The stereotypically used EPs are normally superior to other EPs in perceiving the associated property and it has been speculated that the EPs are a means of maximising pickup of the relevant sensory information. However, EPs are not always executed identically as they vary in a number of ways. For instance, the peak force and the number of fingers used during exploration are not fixed. This chapter reviews existing findings on the exploratory movement strategies that humans use in softness perception and gives an overview on how different manners of exploration affect the performance in softness tasks. It is shown that observers adapt their movement strategies depending on variations of the stimulus value and the exact conditions of the exploratory task, and that different movement parameters, e.g. the peak exploratory forces, considerably affect performance. Overall, results suggest that humans adjust their exploratory strategies to achieve the highest levels of performance in softness discrimination. Drewing, K. (2014). Exploratory movement strategies in softness perception. Multisensory Softness, 109-125.
The contribution of vision to visuo-haptic softness judgments has been observed to be non-optimal and it has been speculated that the visual weights are “sticky”, i.e., do not account for the senses’ reliabilities. The present study tested the hypothesis of sticky weights by varying the quality of the visual information. Participants discriminated between the softness of two objects with deformable surfaces using only visual, only haptic, or bisensory information. Visually, we displayed the finger’s positions and stimulus deformations in a noisy or precise quality or in a precise quality enhanced by visual force information from color changes of the finger nail. We assessed the reliabilities of the judgments using the method of constant stimuli. In bisensory conditions, discrepancies between the two senses’ information were used to measure each sense’s weight. The reliability of visual judgments was lower with noisy as compared to precise position information, visual force information did not affect reliability. The reliability of bisensory judgments was suboptimal and visual weights were higher than optimal. Not as expected, the visual weights shifted with the visual reliability. The results confirm that visuo-haptic integration of softness information is suboptimal and biased towards vision, but with weights that are “lazy” rather than sticky. Drewing, K., & Kruse, O. (2014, June). Weights in visuo-haptic softness perception are not sticky. In International Conference on Human Haptic Sensing and Touch Enabled Computer Applications (pp. 68-76). Springer, Berlin, Heidelberg.
The sense of touch is characterized by its sequential nature. In texture perception, enhanced spatio-temporal extension of exploration leads to better discrimination performance due to combination of repetitive information. We have previously shown that the gains from additional exploration are smaller than the Maximum Likelihood Estimation (MLE) model of an ideal observer would assume. Here we test if this suboptimal integration can be explained by unequal weighting of information. Participants stroke 2 to 5 times across a virtual grating and judged the ridge period in a 2IFC task. We presented slightly discrepant period information in one of the strokes in the standard grating. Results show linearly decreasing weights of this information with spatio-temporal distance (number of intervening strokes) to the comparison grating. For each exploration extension (number of strokes) the stroke with the highest number of intervening strokes to the comparison was completely disregarded. The results are consistent with the notion that memory limitations are responsible for the unequal weights. This study raises the question if models of optimal integration should include memory decay as an additional source of variance and thus not expect equal weights. Lezkan, A., & Drewing, K. (2014, June). Unequal but fair? Weights in the serial integration of haptic texture information. In International Conference on Human Haptic Sensing and Touch Enabled Computer Applications (pp. 386-392). Springer, Berlin, Heidelberg.
