Sensory cue integration
The brain has a multitude of sensory cues about the world to make perceptual inferences and to guide motor behavior. Vision itself contains a number of qualitatively different cues that are coded by at least partially distinct neural mechanisms (stereoscopic disparities, motion, shading, texture, contour, shadows, etc.). Other sensory modalities provide independent sources of information as well. Touch can provide information about an object's shape, orientation and position in space, kinesthetic cues provide information about limb and body movements, audition provides information about an object's position, motion and material makeup.
We are studying how the brain integrates different cues, both within the visual modality and between senses. Most recently we have begun working on how the brain integrates non-visual signals (e.g. audition, kinesthesis) with visual signals to perceive the world. One of the striking phenomena in this realm is the ability of one sense changing one's perception of another sense, as in the famous McGurk effect in which vision of someone's lip movements can dramatically change what you think you hear. We have shown that felt hand movements can similarly dramatically change your visual percept of a pattern's motion. We use both theoretical and psychophysical studies to unravel the computations that underlay these sensory interactions.
- Dieter, K. C., Hu, B., Knill, D. C., Blake, R. and Tadin, D. (In press) Kinesthesis can make an invisible hand visible, Psychological Science. PDF
- Landy, M. S., Banks, M. and Knill, D. C. (2011) Ideal-Observer Models of Cue Integration, in (Trommershauser, J. Kording, K., and Landy, M. S. eds.) Sensory Cue Integration, Osford Univ. Press, Oxford, England.
-
Seydell, A., Knill, D. C. and Trommershauser, J. (2011) Priors and Learning in Cue Integration, in (Trommershauser, J. Kording, K., and Landy, M. S. eds.) Sensory Cue Integration, Osford Univ. Press, Oxford, England.
- Michel, M., Brouwer, A. Jacobs, R. and Knill, D. C. (2011) Optimality Principles Apply to a Broad Range of Information Integration Problems in Perception and Action, in (Trommershauser, J. Kording, K., and Landy, M. S. eds.) Sensory Cue Integration, Osford Univ. Press, Oxford, England.
- Moreno-Bote, R., Knill, D. C. and Pouget, A. (2011) Bayesian sampling in visual perception, Proceedings of the National Academy of Sciences, 108(30):12491-6. PDF
- Bejjanki, V. R., Clayards, M., Knill, D. C. and Aslin, R. N. (2011) Cue Integration in Categorical Tasks: Insights from Audio-Visual Speech Perception, PLoS one, 6(5): e19812. doi:10.1371/journal.pone.0019812 PDF
-
Hu, B. and Knill, D. C. (2010) Kinesthetic information disambiguates visual motion signals, Current Biology, 20 (20), 20, R436-R437.
PDF Supplement
-
Seydell, A., Knill, D. C. and Trommershauser, J. (2010) Adapting internal statistical models for interpreting visual cues to depth, Journal of Vision. 10 (4), Article 1.
PDF
-
Greenwald, H. and Knill, D. C. (2009) Cue Integration Outside Central Fixation:
A Study of Grasping in Depth, Journal of Vision, Vol. 9, No. 2, 1 - 16.
PDF
-
Greenwald, H. and Knill, D. C. (2008) A Comparison of Visuomotor Cue Integration Strategies for Object Placement and Prehension, Visual Neuroscience, Aug., 1 - 10.
PDF
-
Knill, D. C. (2007) Robust cue integration: a Bayesian model and evidence from psychophysical studies with stereoscopic and figure cues to slant, Journal of Vision, 7 (7), 1 – 24.
PDF
-
Knill, D. C. (2007) Learning Bayesian priors for depth perception, Journal of Vision, 7 (8), 1 – 20.
PDF
-
Knill, D. C. (2007) Bayesian models of sensory cue integration, in (Doya, K., Ishii, S., Pouget, A. and Rao, R., eds.) Bayesian Brain: Probabilistic Approaches to Neural Coding, MIT Press, Cambridge, MA.
PDF
-
Greenwald, H., Knill, D. C. and Saunders, J. (2005) Integrating depth cues for visuomotor control: A matter of time, Vision Research, 45 (15), 1975 - 1989.
PDF
-
Atkins, J. E., Jacobs, R. A. and Knill, D. C. (2003) Experience-Dependent Visual Cue Recalibration Based on Discrepancies Between Visual and Haptic Percepts, Vision Research., 43 (25): 2603-2613.
PDF
-
Knill, D. C. and Saunders, J. (2003) Do humans optimally integrate stereo and texture information for judgments of surface slant? Vision Research, 43 (24), 2539-58.
PDF
-
Knill, D. C. (2003) Mixture models and the probabilistic structure of depth cues, Vision Research, 43 (7), 831-854.
PDF
-
Saunders, J. and Knill, D. C. (2001) Perception of 3D surface orientation from skew symmetry, Vision Research, 41 (24), 3163 - 3185.
PDF