Phoster

Research and Development

Mixed-initiative Dialogue Systems, Description, Mental Imagery, Mental Simulation and Imagination

Description

Rhetorical Status of the Descriptive by Philippe Hamon and Patricia Baudoin, Picture Theory: Essays on Verbal and Visual Representation by William J. T. Mitchell, Pictures into Words: Theoretical and Descriptive Approaches to Ekphrasis edited by Valerie Robillard and Els Jongeneel, Descriptive Imagery by Michael Riffaterre, From Image Sequences towards Conceptual Descriptions by Hans-Hellmut Nagel, Pictorial Models and Narrative Ekphrasis by Tamar Yacobi, Writing for Art: The Aesthetics of Ekphrasis by Stephen Cheeke, Ekphrasis and Representation by James A. W. Heffernan, The Poetics of Ekphrasis by John Hollander, Painting, Poetry, Parallelism: Ekphrasis, Stylistics and Cognitive Poetics by Peter Verdonk, Cognitive Poetics and Imagery by Line Brandt and Per Aage Brandt, Fidelity without Mimesis: Mental Imagery from Visual Description by Anežka Kuzmičová, From Text to Images through Meanings by Jayakumar S. Kandhasamy and Ming Xie, Imagery in Sentence Comprehension: An fMRI Study by Marcel A. Just, Sharlene D. Newman, Timothy A. Keller, Alice McEleney and Patricia A. Carpenter and Incremental Natural Language Description of Dynamic Imagery by Gerd Herzog, Chen-Ko Sung, Elisabeth André, Wilfried Enkelmann, Hans-Hellmut Nagel, Thomas Rist, Wolfgang Wahlster and Georg Zimmermann.

Attention

Selective Attention to the Parts of an Object by Shaun P. Vecera, Marlene Behrmann and John McGoldrick, Neural Activity during Attention Shifts between Object Features by Yasuhiro Nagahama, Norihiro Sadato, Hiroshi Yamauchi, Yukinori Katsumi, Takuya Hayashi, Hidenao Fukuyama, Jun Kimura, Hiroshi Shibasaki and Yoshiharu Yonekura, Objects and Attention: The State of the Art by Brian J. Scholl, Multiple Object Recognition with Visual Attention by Jimmy Ba, Volodymyr Mnih and Koray Kavukcuoglu, Contextual Cueing of Visual Attention by Marvin M. Chun, Attention, Perception and Memory: An Integrated Introduction by Elizabeth A. Styles, Attention Improves Perceptual Quality by Britt Anderson and Michael Druker, Attention during Natural Vision Warps Semantic Representation across the Human Brain by Tolga Çukur, Shinji Nishimoto, Alexander G. Huth and Jack L. Gallant, Attentional Modulation of Object Representations in Working Memory by Lepsien Jöran and Anna C. Nobre, Computational Modelling of Visual Attention by Laurent Itti and Christof Koch, Modeling the Influence of Task on Attention by Vidhya Navalpakkam and Laurent Itti, A Goal Oriented Attention Guidance Model by Vidhya Navalpakkam and Laurent Itti, Distributing versus Focusing Attention in Visual Short-term Memory by Tal Makovski and Yuhong V. Jiang, Orienting Attention to Locations in Perceptual versus Mental Representations by Nobre Anna C., Jennifer T. Coull, Pierre Maquet, Christopher D. Frith, Rik Vandenberghe and M. M. Mesulam, Orienting Attention to Locations in Internal Representations by Ivan C. Griffin and Anna C. Nobre, Directing Spatial Attention in Mental Representations: Interactions between Attentional Orienting and Working-memory Load by Lepsien Jöran, Ivan C. Griffin, Joseph T. Devlin and Anna C. Nobre, Visual Working Memory as Visual Attention Sustained Internally over Time by Marvin M. Chun, Attention and Information Transfer from Visual Sensory to Visual Working Memory by Jane Jacob, Shon MonDragon and Bruno Breitmeyer, Shared Representations for Working Memory and Mental Imagery in Early Visual Cortex by Anke M. Albers, Peter Kok, Ivan Toni, H. Chris Dijkerman and Floris P. de Lange, Interactions between Attention and Working Memory by Edward Awh, Edward K. Vogel and Sei-Hwan Oh, Crossmodal Space and Crossmodal Attention by Spence, Charles and Jon Driver, Multisensory Integration and Crossmodal Attention Effects in the Human Brain by John J. McDonald, Wolfgang A. Teder-Sälejärvi and Lawrence M. Ward, Eye Movement Predictions on Natural Videos by Martin Böhme, Michael Dorr, Christopher Krause, Thomas Martinetz and Erhardt Barth, Efficient Visual Coding and the Predictability of Eye Movements on Natural Movies by Eleonora Vig, Michael Dorr and Erhardt Barth, A Learned Saliency Predictor for Dynamic Natural Scenes by Eleonora Vig, Michael Dorr, Thomas Martinetz and Erhardt Barth, Dynamic Scene Classification using Spatial and Temporal Cues by Arun B. Vasudevan, Srikanth Muralidharan, Shiva P. Chintapalli and Shanmuganathan Raman, Spatiotemporal Saliency Detection and its Applications in Static and Dynamic Scenes by Wonjun Kim, Chanho Jung and Changick Kim, A Model of Saliency-based Visual Attention for Rapid Scene Analysis by Laurent Itti, Christof Koch and Ernst Niebur, Spatiotemporal Saliency in Dynamic Scenes by Vijay Mahadevan and Nuno Vasconcelos, Visual Causes versus Correlates of Attentional Selection in Dynamic Scenes by Ran Carmi and Laurent Itti, Attention for Fine-grained Categorization by Pierre Sermanet, Andrea Frome and Esteban Real, Attention and Scene Understanding by Vidhya Navalpakkam, Michael A. Arbib and Laurent Itti, Memory for Elements of a Complex Scene: Binding and the Influence of Attention by Stephanie L. Martin, The Role of Memory in Guiding Attention during Natural Vision by Ran Carmi and Laurent Itti, Integrating Episodic and Semantic Information in Memory for Natural Scenes by Pernille Hemmer and Mark Steyvers, Grounded Semantic Composition for Visual Scenes by Peter Gorniak and Deb K. Roy, Guidance of Visual Attention by Semantic Information in Real-world Scenes by Chia-Chien Wu, Farahnaz A. Wick and Marc Pomplun, Interaction of Language and Visual Attention: Evidence from Production and Comprehension by Ramesh K. Mishra, On the Mental Representations Originating during the Interaction between Language and Vision by Ramesh K. Mishra and Fernando Marmolejo-Ramos and Mechanisms and Representations of Language-mediated Visual Attention by Falk Huettig, Ramesh K. Mishra and Christian N. L. Olivers.

Perception

Language and Perception by George A. Miller and Philip N. Johnson-Laird, On the Interdependence of Language and Perception by David L. Waltz, From Observations on Language to Theories of Visual Perception by Johan Wagemans, The Senses Considered as Perceptual Systems by James J. Gibson, Perceptions of Perceptual Symbols by Lawrence W. Barsalou, Perceptual Organization Influences Visual Working Memory by Geoffrey F. Woodman, Shaun P. Vecera and Steven J. Luck, Perceptual Processing Affects Conceptual Processing by Saskia van Dantzig, Diane Pecher, René Zeelenberg and Lawrence W. Barsalou and Perceptual Simulation in Developing Language Comprehension by Jan A. A. Engelen, Samantha Bouwmeester, Anique B. H. de Bruin and Rolf A. Zwaan.

Mental Simulation

The Relationship of Language to the Formation of Concepts by Eric H. Lenneberg, Language and Simulation in Conceptual Processing by Lawrence W. Barsalou, Ava Santos, W. Kyle Simmons and Christine D. Wilson, Interactive Dimensions in the Construction of Mental Representations for Text by David N. Rapp and Holly A. Taylor, The Brain’s Concepts: The Role of the Sensory-motor System in Conceptual Knowledge by Vittorio Gallese and George Lakoff, Simulation Theory and Mental Concepts by Alvin I. Goldman, Situated Simulation in the Human Conceptual System by Lawrence W. Barsalou, Grounding Conceptual Knowledge in Modality-specific Systems by Lawrence W. Barsalou, W. Kyle Simmons, Aron K. Barbey and Christine D. Wilson, Automatic Activation of Episodic Information in a Semantic Memory Task by Gail McKoon and Roger Ratcliff, Perceptual Knowledge Retrieval Activates Sensory Brain Regions by Robert F. Goldberg, Charles A. Perfetti and Walter Schneider, Verifying Different-modality Properties for Concepts Produces Switching Costs by Diane Pecher, René Zeelenberg and Lawrence W. Barsalou, A Meta-analytic Review of Multisensory Imagery Identifies the Neural Correlates of Modality-specific and Modality-general Imagery by Chris McNorgan, Multisensory Integration and Crossmodal Attention Effects in the Human Brain by John J. McDonald, Wolfgang A. Teder-Sälejärvi and Lawrence M. Ward, Crossmodal Mental Imagery by Charles Spence and Ophelia Deroy, Imagery, Context Availability, Contextual Constraint, and Abstractness by Katja Wiemer-Hastings, Jan Krug and Xu Xu, Kinematic Mental Simulations in Abduction and Deduction by Sangeet S. Khemlani, Robert Mackiewicz, Monica Bucciarelli and Philip N. Johnson-Laird, Predicting Object Dynamics in Scenes by David F. Fouhey and C. Lawrence Zitnick, Predictive Feedback and Conscious Visual Experience by Matthew F. Panichello, Olivia S. Cheung and Moshe Bar, The Functional Impact of Mental Imagery on Conscious Perception by Joel Pearson, Colin W. G. Clifford and Frank Tong, Mental Imagery Changes Multisensory Perception by Christopher C. Berger and H. Henrik Ehrsson and Theories Relating Mental Imagery to Perception by Ronald A. Finke.

Imagination

Handbook of Imagination and Mental Simulation by William M. P. Klein, Julie A. Suhr and Keith D. Markman, The Role of Imagination in Perception by Michael Pendlebury, Imagination as Simulation: Aesthetics Meets Cognitive Science by Gregory Currie, Imagining: A Phenomenological Study by Edward S. Casey, Imagery and Mental Processes by Allan Paivio, Mental Imagery and Visual Working Memory by Rebecca Keogh and Joel Pearson, Mental Images and the Brain by Stephen M. Kosslyn, Dynamics of Brain Processing during Mental Imagery by Livia Tomova, Neural Foundations of Imagery by Stephen M. Kosslyn, Giorgio Ganis and William L. Thompson, The Neural Correlates of Visual Working Memory Encoding: A Time-resolved fMRI Study by J. Jay Todd, Suk Won Han, Stephenie Harrison and René Marois, Neural Systems Activated during Visual Mental Imagery by William L. Thompson and Stephen M. Kosslyn, When is Early Visual Cortex Activated during Visual Mental Imagery? by Stephen M. Kosslyn and William L. Thompson, Shared Representations for Working Memory and Mental Imagery in Early Visual Cortex by Anke M. Albers, Peter Kok, Ivan Toni, H. Chris Dijkerman and Floris P. de Lange, Functional Anatomy of High-resolution Visual Mental Imagery by Emmanuel Mellet, Nathalie Tzourio-Mazoyer, Stephan Bricogne, Bernard Mazoyer, Stephen M. Kosslyn and Michel Denis, Crossmodal Mental Imagery by Charles Spence and Ophelia Deroy, Multimodal Images in the Brain by Stephen M. Kosslyn, Giorgio Ganis and William L. Thompson, Using Imagination to Understand the Neural Basis of Episodic Memory by Demis Hassabis, Dharshan Kumaran and Eleanor A. Maguire, Cognitive Control and Episodic Memory by Anthony D. Wagner, Voluntary versus Automatic Control Over the Mind’s Eye’s Movement by John Jonides, Voluntary and Automatic Attentional Control of Visual Working Memory by Brandon K. Schmidt, Edward K. Vogel, Geoffrey F. Woodman and Steven J. Luck, Reading the Mind’s Eye: Decoding Category Information during Mental Imagery by Leila Reddy, Naotsugu Tsuchiya and Thomas Serre, The Representation of Object Concepts in the Brain by Alex Martin, A Continuous Semantic Space Describes the Representation of Thousands of Object and Action Categories across the Human Brain by Alexander G. Huth, Shinji Nishimoto, An T. Vu and Jack L. Gallant, Attention during Natural Vision Warps Semantic Representation across the Human Brain by Tolga Çukur, Shinji Nishimoto, Alexander G. Huth and Jack L. Gallant and Evaluating the Mind’s Eye: The Metacognition of Visual Imagery by Joel Pearson, Rosanne L. Rademaker and Frank Tong.

Objects

Language Comprehenders Mentally Represent the Shapes of Objects by Rolf A. Zwaan, Robert A. Stanfield and Richard H. Yaxley, The Representation of Object Concepts in the Brain by Alex Martin, A Continuous Semantic Space Describes the Representation of Thousands of Object and Action Categories across the Human Brain by Alexander G. Huth, Shinji Nishimoto, An T. Vu and Jack L. Gallant, Attention during Natural Vision Warps Semantic Representation across the Human Brain by Tolga Çukur, Shinji Nishimoto, Alexander G. Huth and Jack L. Gallant, Mental Models of the Structure of Visual Objects by Lynn A. Cooper, Mental Representation of Three-dimensional Objects in Visual Problem Solving and Recognition by Lynn A. Cooper, Storage of Features, Conjunctions, and Objects in Visual Working Memory by Steven J. Luck, Geoffrey F. Woodman and Edward K. Vogel, Exploration of Complex Visual Feature Spaces for Object Perception by Daniel D. Leeds, John A. Pyles and Michael J. Tarr, Models of Object Recognition by Maximilian Riesenhuber and Tomaso Poggio, Parts of Recognition by Donald D. Hoffman and Whitman A. Richards, Object Activation from Features in the Semantic System by Michael A. Kraut, Sarah Kremen, Jessica B. Segal, Vincent Calhoun, Lauren R. Moo and John Hart Jr., Creating Concepts from Converging Features in Human Cortex by Marc N. Coutanche and Sharon L. Thompson-Schill, The Emergence of Semantic Meaning in the Ventral Temporal Pathway by Thomas A. Carlson, Ryan A. Simmons, Nikolaus Kriegeskorte and L. Robert Slevc, Hierarchical Models of Object Recognition in Cortex by Maximilian Riesenhuber and Tomaso Poggio, Neural Mechanisms of Object Recognition by Maximilian Riesenhuber and Tomaso Poggio, Computational Models of Object Recognition in Cortex: A Review by Maximilian Riesenhuber and Tomaso Poggio, A Distributed Computational Cognitive Model for Object Recognition by Yong Jin Liu, Qiu Fang Fu, Ye Liu and Xiaolan Fu, Objects in Context by Andrew Rabinovich, Andrea Vedaldi, Carolina Galleguillos, Eric Wiewiora and Serge Belongie and Visual Objects in Context by Moshe Bar.

Attributes

Describing Objects by Their Attributes by Ali Farhadi, Ian Endres, Derek Hoiem and David Forsyth, Interactive Discovery of Task-specific Nameable Attributes by Devi Parikh and Kristen Grauman, Learning Models for Object Recognition from Natural Language Descriptions by Josiah Wang, Katja Markert and Mark Everingham, A Joint Model of Language and Perception for Grounded Attribute Learning by Cynthia Matuszek, Nicholas FitzGerald, Luke Zettlemoyer, Liefeng Bo and Dieter Fox, Attribute-centric Recognition for Cross-category Generalization by Ali Farhadi, Ian Endres and Derek Hoiem, Learning Visual Attributes by Vittorio Ferrari and Andrew Zisserman, Joint Learning of Visual Attributes, Object Classes and Visual Saliency by Gang Wang and David Forsyth, Visual Determinants of Preferred Adjective Order by Eva Belke, Storage of Features, Conjunctions, and Objects in Visual Working Memory by Steven J. Luck, Geoffrey F. Woodman and Edward K. Vogel, Exploration of Complex Visual Feature Spaces for Object Perception by Daniel D. Leeds, John A. Pyles and Michael J. Tarr, The Development of Features in Object Concepts by Philippe G. Schyns, Robert L. Goldstone and Jean-Pierre Thibaut, Property Generation Reflects Word Association and Situated Simulation by Ava Santos, Sergio E. Chaigneau, W. Kyle Simmons and Lawrence W. Barsalou, Perceptual Simulation in Conceptual Tasks by Lawrence W. Barsalou, Karen O. Solomon and Ling-Ling Wu, Perceptual Simulation in Conceptual Combination: Evidence from Property Generation by Ling-ling Wu and Lawrence W. Barsalou and Perceptual Simulation in Property Verification by Karen O. Solomon and Lawrence W. Barsalou.

Nouns

The Major Functions of the Noun Phrase by Avery Andrews, Definite Noun Phrases and the Semantics of Discourse by Manfred Pinkal, Models for the Semantic Classification of Noun Phrases by Dan Moldovan, Adriana Badulescu, Marta Tatu, Daniel Antohe and Roxana Girju, Syntactic Processes in the Production of Noun Phrases by Herbert Schriefers and A Basis for Generating Expectancies for Verbs from Nouns by Ken McRae, Mary Hare, Jeffrey L. Elman and Todd Ferretti.

Adjectives

The Description of Adjectives for Natural Language Processing: Theoretical and Applied Perspectives by Pierrette Bouillon and Evelyne Viegas, Adjectival Modification in Text Meaning Representation by Victor Raskin and Sergei Nirenburg, Lexical Semantics of Adjectives by Victor Raskin and Sergei Nirenburg, Clustering Adjectives for Class Acquisition by Gemma B. Torrent and Laura Alonso I Alemany, Adjectives as Indicators of Subjectivity in Documents by Robert Rittman, Nina Wacholder, Paul Kantor, Kwong Bor Ng, Tomek Strzalkowski and Ying Sun, The Semantics of Natural Language Descriptions of Continuous Quantities by Shenghui Wang, Modelling the Semantics of Adjectives in the Ontology-lexicon Interface by John P. McCrae, Christina Unger, Francesca Quattri and Philipp Cimiano, Adjectives: A Uniform Semantic Approach by Nabil Abdullah and Richard A. Frost, A Theory of Vague Adjectives Grounded in Relevant Observables by Brandon Bennett, Finetuning NLG through Experiments with Human Subjects: The Case of Vague Descriptions by Kees van Deemter, Vagueness: Facts, Models, and Programs by Kees van Deemter, Adjective Based Inference by Marilisa Amoia and Claire Gardent, A First Order Semantic Approach to Adjectival Inference by Marilisa Amoia, A Test Suite for Inference Involving Adjectives by Marilisa Amoia and Claire Gardent, Two Set-theoretic Approaches to the Semantics of Adjective-noun Combinations by Nabil Abdullah and Richard Frost, Two Theories about Adjectives by Hans Kamp and Beyond Nouns: Exploiting Prepositions and Comparative Adjectives for Learning Visual Classifiers by Abhinav Gupta and Larry S. Davis.

Events

Language and Event Perception: Steps toward a Synthesis by Robert R. Verbrugge, Event Perception: A Mind-brain Perspective by Jeffrey M. Zacks, Nicole K. Speer, Khena M. Swallow, Todd S. Braver and Jeremy R. Reynolds, Visual Event Perception by Gunnar Johansson, Visual Movement and the Neural Correlates of Event Perception by Jeffrey M. Zacks, Khena M. Swallow, J. M. Vettel and M. P. McAvoy, A Cortical Network for the Encoding of Object Change by Nicholas C. Hindy, Sarah H. Solomon, Gerry T. M. Altmann and Sharon L. Thompson-Schill, Kinematic Mental Simulations in Abduction and Deduction by Sangeet S. Khemlani, Robert Mackiewicz, Monica Bucciarelli and Philip N. Johnson-Laird, Predicting Object Dynamics in Scenes by David F. Fouhey and C. Lawrence Zitnick, Events, Processes, and States by Alexander P. D. Mourelatos, Event Structure in Perception and Conception by Jeffrey M. Zacks and Barbara Tversky, Event Cognition by Gabriel A. Radvansky and Jeffrey M. Zacks, Event Memory: A Theory of Memory for Laboratory, Autobiographical, and Fictional Events by David C. Rubin and Sharda Umanath, Perceiving, Remembering, Communicating Structure in Events by Jeffrey M. Zacks, Barbara Tversky and Gowri Iyer, Understanding Scene Descriptions as Event Simulations by David L. Waltz, Natural Language Description of Time-varying Scenes by Bernd Neumann, Time in Language: Event Duration in Language Comprehension by Marta Coll-Florit and Silvia P. Gennari, The Role of Verb Tense and Verb Aspect in the Foregrounding of Information during Reading by Manuel Carreiras, Núria Carriedo, María Angeles Alonso and Angel Fernández, A Theory of Aspectuality: The Interaction between Temporal and Atemporal Structure by Henk J. Verkuyl, Grammatical Aspect and Mental Simulation by Benjamin K. Bergen and Kathryn Wheeler, Mental Models and Temporal Reasoning by Walter Schaeken, Philip N. Johnson-Laird and Gery d’Ydewalle, Time in Language, Situation Models, and Mental Simulations by Rolf A. Zwaan, Events, Discourse Representations, and Temporal References by Hans Kamp, Action, Action Description, and Narrative by Teun A. van Dijk, Description and Event in Narrative by Peter Klaus, Segmentation in Reading and Film Comprehension by Jeffrey M. Zacks, Nicole K. Speer and Jeremy R. Reynolds, Hierarchical Language-based Representation of Events in Video Streams by Ram Nevatia, Tao Zhao and Somboon Hongeng and Changes in Situation Models Modulate Processes of Event Perception in Audiovisual Narratives by Markus Huff, Tino G. K. Meitz and Frank Papenmeier.

Verbs

The Study of Verbs in Cognitive Science by Roberto G. de Almeida and Christina Manouilidou, Building Verb Meanings by Malka Rappaport Hovav and Beth Levin, Integrating Verbs, Situation Schemas, and Thematic Role Concepts by Todd R. Ferretti, Ken McRae and Andrea Hatherell, Inferring the Meaning of Verbs from Context by Peter Wiemer-Hastings, Arthur Graesser and Katja Wiemer-Hastings, Abstract Verbs by Power Richard, Verbs and Prototype Theory: State of the Art and Possibilities by Dušan Stamenković, Thematic Roles as Verb-specific Concepts by Ken McRae, Todd R. Ferretti and Liane Amyote, The Role of Thematic Structures in Interpretation and Parsing by Michael K. Tanenhaus, Greg Carlson and John C. Trueswell, Continuous Mapping from Sound to Meaning in Spoken-language Comprehension: Immediate Effects of Verb-based Thematic Constraints by Delphine Dahan and Michael K. Tanenhaus, Thematic Role Assignment in Context by Gerry T. M. Altmann, Simulation and Knowledge of Action edited by Jérôme Dokic and Joëlle Proust, Perception, Action, and Word Meanings in the Human Brain: The Case from Action Verbs by Marina Bedny and Alfonso Caramazza, Functional Anatomy of Execution, Mental Simulation, Observation, and Verb Generation of Actions: A Meta‐analysis by Julie Grezes and Jean Decety, Verb Interpretation for Basic Action Types: Annotation, Ontology Induction and Creation of Prototypical Scenes by Francesca Frontini, Irene De Felice, Fahad Khan, Irene Russo, Monica Monachini, Gloria Gagliardi and Alessandro Panunzi, English Verbs of Motion and Prototype Theory by Dušan Stamenković and Miloš Tasić, Visual Semantics and Ontology of Eventive Verbs by Minhua Ma and Paul Mc Kevitt, Spatial Representations Activated during Real-time Comprehension of Verbs by Daniel C. Richardson, Michael J. Spivey, Lawrence W. Barsalou and Ken McRae, Grammatical Aspect and Mental Simulation by Benjamin K. Bergen and Kathryn Wheeler, A Theory of Aspectuality: The Interaction between Temporal and Atemporal Structure by Henk J. Verkuyl, Tense and Aspect edited by Philip J. Tedeschi and Annie Zaenen, Tense Variation in Narrative by Deborah Schiffrin, The Role of Verb Tense and Verb Aspect in the Foregrounding of Information during Reading by Manuel Carreiras, Núria Carriedo, María A. Alonso and Angel Fernández, Events, Discourse Representations, and Temporal References by Hans Kamp, The Language of Time edited by Mani Inderjeet, Language and Time: A Cognitive Linguistics Approach by Vyvyan Evans, Incremental Interpretation at Verbs: Restricting the Domain of Subsequent Reference by Gerry T. M. Altmann and Yuki Kamide and Verbs Semantics and Lexical Selection by Zhibiao Wu and Martha Palmer.

Adverbs

Verbs and Adverbs: Multidimensional Motion Interpolation by Charles Rose, Michael F. Cohen and Bobby Bodenheimer, A Semantic Theory of Adverbs. by Richmond H. Thomason and Robert C. Stalnaker, The Grammar of Adverbials by Renate Bartsch, Adverb Classification by Olivier Bonami, Daniele Godard and Brigitte Kampers-Manhe and Adverbs and Adverbials by Claudia Maienborn and Martin Schäfer.

Spatial Descriptions and Visuospatial Reasoning

Visuospatial Reasoning by Barbara Tversky, Spatial Reasoning by Ruth M. J. Byrne and Philip N. Johnson-Laird, Discourse Comprehension by Arthur C. Graesser, Keith K. Millis and Rolf A. Zwaan, Descriptions and Discourse Models by Philip N. Johnson-Laird and Alan Garnham, Discourse Models as Interfaces between Language and the Spatial World by Simon C. Garrod and Anthony J. Sanford, Space and the Vision–language Interface: A Model-theoretic Approach by Francesco-Alessio Ursini, Spatial Language and Spatial Cognition by Barbara Landau, Interpreting Words in Spatial Descriptions by Daniel G. Morrow and Herbert H. Clark, Composition of Spatial Relations by Stefan Högg and Irmgard Schwarzer, Spatial Language, Visual Attention, and Perceptual Simulation by Kenny R. Coventry, Dermot Lynott, Angelo Cangelosi, Lynn Monrouxe, Dan Joyce and Daniel C. Richardson, Visuospatial Working Memory and Mental Representation of Spatial Descriptions by Rossana De Beni, Francesca Pazzaglia, Valerie Gyselinck and Chiara Meneghetti, Spatial Perspective in Descriptions by Barbara Tversky, Switching Points of View in Spatial Mental Models by Nancy Franklin, Barbara Tversky and Vicky Coon, Frames of Reference in Spatial Memories Acquired from Language by Weimin Mou, Kan Zhang and Timothy P. McNamara, Multiple Systems of Spatial Memory: Evidence from Described Scenes by Marios N. Avraamides and Jonathan W. Kelly, Spatial and Visual Working Memory: A Mental Workspace by Robert H. Logie, Visuo-spatial Working Memory by Robert H. Logie, Working Memory in Developing and Applying Mental Models from Spatial Descriptions by Tad T. Brunyé and Holly A. Taylor, The Representation of Space in Mental Models Derived from Text by William Langston, Douglas C. Kramer and Arthur M. Glenberg, Mental Models of Spatial Relations and Transformations from Language by Barbara Tversky, Joseph Kim and Andrew Cohen, Qualitative Spatial Reasoning in Interpreting Text and Narrative by Ernest Davis, Working Memory in Developing and Applying Mental Models from Spatial Descriptions by Tad T. Brunyé and Holly A. Taylor, Dealing with Indeterminacy in Spatial Descriptions by Jean-Baptiste Van der Henst, Coralie Chevallier, Walter Schaeken, Hugo Mercier and Ira Noveck, The Representation of Space in Mental Models Derived from Text by William Langston, Douglas C. Kramer and Arthur M. Glenberg, Mental Models of Spatial Relations and Transformations from Language by Barbara Tversky, Joseph Kim and Andrew Cohen, Judging Distances: Mental Spaces, Distance, and Viewpoint in Literary Discourse by Barbara Dancygier and Lieven Vandelanotte, Qualitative Spatial Reasoning in Interpreting Text and Narrative by Ernest Davis, Temporal and Spatial Distance in Situation Models by Mike Rinck and Gordon H. Bower, The Visuospatial Dimension of Writing by Thierry Olive and Jean-Michel Passerault, Verbal, Visual, and Spatial Working Memory in Written Language Production by Ronald T. Kellogg, Thierry Olive and Annie Piolat, The Role of Visuospatial Sketchpad in the Written Production of Descriptive and Argumentative Texts by Jean-Michel Passerault and Jérôme Dinet, The Contribution of Different Components of Working Memory to Knowledge Transformation during Writing by David Galbraith, Sheila Ford, Gillian Walker and Jessica Ford, The Interface between Language and Visuo-spatial Representations by Manuel de Vega, Marguerite Cocude, Michel Denis, Maria José Rodrigo and Hubert D. Zimmer, Differential Roles for Visuospatial and Verbal Working Memory in Situation Model Construction by Naomi P. Friedman and Akira Miyake, Spatial and Linguistic Aspects of Visual Imagery in Sentence Comprehension by Benjamin K. Bergen, Shane Lindsay, Teenie Matlock and Srini Narayanan, Verbal, Visual, and Spatial Working Memory Demands during Text Composition by Thierry Olive, Ronald T. Kellogg and Annie Piolat, Assessing the Symbolic Distance Effect in Mental Images Constructed from Verbal Descriptions: A Study of Individual Differences in the Mental Comparison of Distances by Michel Denis, Neural Basis of Mental Scanning of a Topographic Representation Built from a Text by Emmanuel Mellet, Stephan Bricogne, Fabrice Crivello, Bernard Mazoyer, Michel Denis and Nathalie Tzourio-Mazoyer, A Computational Theory of Visuo-spatial Mental Imagery by Jan F. Sima and Inferencing from Spatial Information by Merideth Gattis.

Adpositional Phrases

Learning Prepositions by Katharina J. Rohlfing, How Do Prepositional Phrases Fit into Compositional Syntax and Semantics? by Dieter Wunderlich, The Semantics of Locative Prepositional Phrases in English by Seungho Nam, Semantics and Pragmatics of Locative Expressions by Annette Herskovits, The Language of Space: The Acquisition and Interpretation of Spatial Adpositions by Francesco-Alessio Ursini, Neuroanatomical Correlates of Locative Prepositions by Daniel Tranel and David Kemmerer and Neural Correlates of Locative Prepositions by Matthijs L. Noordzij, Sebastiaan F. W. Neggers, Nick F. Ramsey and Albert Postma.

Scenes

Creating and Reasoning over Scene Descriptions in a Physically Realistic Simulation by Nick Marton, John Licato and Selmer Bringsjord, Simulation as an Engine of Physical Scene Understanding by Peter W. Battaglia, Jessica B. Hamrick and Joshua B. Tenenbaum, Representing 3D Space in Working Memory: Spatial Images from Vision, Hearing, Touch, and Language by Jack M. Loomis, Roberta L. Klatzky and Nicholas A. Giudice, Attention and Scene Understanding by Vidhya Navalpakkam, Michael A. Arbib and Laurent Itti, Understanding Dynamic Scenes by Antonio Chella, Marcello Frixione and Salvatore Gaglio, Towards Scene Understanding: Object Detection, Segmentation, and Contextual Reasoning by Roozbeh Mottaghi, Towards Total Scene Understanding: Classification, Annotation and Segmentation in an Automatic Framework by Li-Jia Li, Richard Socher and Li Fei-Fei, Memory for Elements of a Complex Scene: Binding and the Influence of Attention by Stephanie L. Martin, The Role of Memory in Guiding Attention during Natural Vision by Ran Carmi and Laurent Itti, Integrating Episodic and Semantic Information in Memory for Natural Scenes by Pernille Hemmer and Mark Steyvers, Grounded Semantic Composition for Visual Scenes by Peter Gorniak and Deb K. Roy, Guidance of Visual Attention by Semantic Information in Real-world Scenes by Chia-Chien Wu, Farahnaz A. Wick and Marc Pomplun, Cortical Representation of Animate and Inanimate Objects in Complex Natural Scenes by Thomas Naselaris, Dustin E. Stansbury and Jack L. Gallant, A Statistical Modeling Framework for Investigating Visual Scene Processing in the Human Brain by Dustin E. Stansbury and Jack L. Gallant, The Computational Perception of Scene Dynamics by Richard Mann, Allan Jepson and Jeffrey M. Siskind, The Dynamic Representation of Scenes by Ronald A. Rensink, Decoding Individual Natural Scene Representations during Perception and Imagery by Matthew R. Johnson and Marcia K. Johnson, Film, Narrative, and Cognitive Neuroscience by Jeffrey M. Zacks and Joseph P. Magliano, The Attentional Theory of Cinematic Continuity by Tim J. Smith, Reconstructing Visual Experiences from Brain Activity Evoked by Natural Movies by Shinji Nishimoto, An T. Vu, Thomas Naselaris, Yuval Benjamini, Bin Yu and Jack L. Gallant, Cognitive Interpretation of Everyday Activities: Toward Perceptual Narrative Based Visuo-spatial Scene Interpretation by Mehul Bhatt, Jakob Suchan and Carl Schultz, Computable Scenes and Structures in Films by Hari Sundaram and Shih-Fu Chang, Cohesion in Film: Tracking Film Elements by Chiao I. Tseng, Continuous Visual Cues Trigger Automatic Spatial Target Updating in Dynamic Scenes by Hauke S. Meyerhoff, Markus Huff, Frank Papenmeier, Georg Jahn and Stephan Schwan, Tracking by Location and Features: Object Correspondence across Spatiotemporal Discontinuities during Multiple Object Tracking by Frank Papenmeier, Hauke S. Meyerhoff, Georg Jahn and Markus Huff, Spatial Representation in Cognitive Science and Film by Daniel T. Levin and Caryn Wang, Multimodal Film Analysis: How Films Mean by John Bateman and Karl-Heinrich Schmidt, Video Scene Interpretation using Perceptual Prominence and Mise-en-scene Features by Gaurav Harit and Santanu Chaudhury and Mise-en-scène: Film Style and Interpretation by John Gibbs.

Referring Expressions

Reference to Objects in Text Production by Christiane VonStutterheim, Roland Mangold-Allwinn, Stefan Barattelli, Ute Kohlmann and H-G Kölbing, Generating Referring Expressions Using Perceptual Groups by Kotaro Funakoshi, Satoru Watanabe, Naoko Kuriyama and Takenobu Tokunaga, Attribute Preference and Priming in Reference Production: Experimental Evidence and Computational Modeling by Albert Gatt, Emiel Krahmer and Martijn Goudbeek, Generating Referring Expressions Involving Relations by Robert Dale and Nicholas Haddock, Lexical Choice and Conceptual Perspective in the Generation of Plural Referring Expressions by Albert Gatt and Kees Van Deemter, Reference to Sets: From Empirical Evidence to Generation by Albert Gatt, On Referring to Sets of Objects Naturally by Helmut Horacek, On Identifying Sets by Matthew Stone, Conceptual Coherence in the Generation of Referring Expressions by Albert Gatt and Kees van Deemter, Efficient Context-sensitive Generation of Referring Expressions by Emiel Krahmer and Mariët Theune, Efficient Generation of Descriptions in Context by Emiel Krahmer and Mariët Theune, Utility‐based Generation of Referring Expressions by Markus Guhe, Models and Empirical Data for the Production of Referring Expressions by Albert Gatt, Emiel Krahmer, Kees van Deemter and Roger P. G. van Gompel, Evaluation in Natural Language Generation: Lessons from Referring Expression Generation by Jette Viethen and Robert Dale, Generating Referential Descriptions under Conditions of Uncertainty by Helmut Horacek, Generation of Quantified Referring Expressions: Evidence from Experimental Data by Dale Barr, Kees Van Deemter and Raquel Fernández, Overspecification Facilitates Object Identification by Anja Arts, Alfons Maes, Leo Noordman and Carel Jansen, Multimodal Discourse in Mediated Spaces by Paul McIlvenny and Chaim Noy, Generating Multimodal References by Ielka Van der Sluis and Emiel Krahmer, Mechanisms and Representations of Language-mediated Visual Attention by Falk Huettig, Ramesh K. Mishra and Christian N. L. Olivers, Attention Driven Reference Resolution in Multimodal Contexts, Integrating Visual and Linguistic Salience for Reference Resolution by John D. Kelleher, Information Fusion for Visual Reference Resolution in Dynamic Situated Dialogue by Geert-Jan M. Kruijff, John D. Kelleher and Nick Hawes, Neural Basis of Mental Scanning of a Topographic Representation Built from a Text by Emmanuel Mellet, Stephan Bricogne, Fabrice Crivello, Bernard Mazoyer, Michel Denis and Nathalie Tzourio-Mazoyer, Selecting One among Many Referents in Spatial Situation Models by Gordon H. Bower and Mike Rinck, Searching for Objects Driven by Context by Bogdan Alexe, Nicolas Heess, Yee W. Teh and Vittorio Ferrari, Interactive Object Detection by Angela Yao, Juergen Gall, Christian Leistner and Luc Van Gool, Locating Objects from Memory or from Sight by David J. Bryant and Barbara Tversky, A Model of Object Location Memory by David Peebles and Corinna Jones, Searching for Targets within the Spatial Layout of Visual Short-term Memory by Bo-Cheng Kuo, Anling Rao, Jöran Lepsien and Anna C. Nobre, Resolution of Multimodal Object References Using Conceptual Short Term Memory by Pfeiffer Thies, Ian Voss and Marc E. Latoschik, Language Understanding by Reference Resolution in Episodic Memory by Kevin M. Livingston, Combining Visual Recognition and Computational Linguistics: Linguistic Knowledge for Visual Recognition and Natural Language Descriptions of Visual Content by Marcus Rohrbach, Resolving Object References in Multimodal Dialogues for Immersive Virtual Environments by Thies Pfeiffer and Marc E. Latoschik, Visual Search and Recognition of Objects, Scenes and People by Josef Sivic and Towards Integrated Microplanning of Language and Iconic Gesture for Multimodal Output by Stefan Kopp, Paul Tepper and Justine Cassell.

Language

The Language of Design by Andy Dong, Aspects of Language Use in Design Conversation by Friedrich Glock, The Cognitive Construct of Design Conversation by Khairul A. M. Khaidzir and Bryan Lawson, Analysing Conversation: Studying Design as Social Action by Ben Matthews and Trine Heinemann, A Discourse Model for Collaborative Design by Michael P. Case and Stephen C. Y. Lu, Studying and Supporting Design Communication by Anja M. Maier and Maaike Kleinsmann, An Organizational View of Design Communication in Design Collaboration by Mao-Lin Chiu, Understanding Patterns of Interaction between Designers during Design Process by Gokula V. A. Vasantha and Amaresh Chakrabarti, Analyzing Patterns of Writing and Sketching in the Product Design Process by Rebecca A. Smith, Sources of Inspiration: A Language of Design by Claudia Eckert and Martin Stacey, Exploring the Relationship between Lexical Behavior and Concept Formation in Design Conversations by Andy Dong, Kevin Davies and David McInnes, Using Language as Related Stimuli for Concept Generation by Ivey Chiu and L. H. Shu, Concept Formation as Knowledge Accumulation: A Computational Linguistics Study by Andy Dong, A Survey on Computational Linguistics in Design Documents by Jian X. Wang, Yuan Yong Feng and Ke Qi, Extracting Design Information from Natural Language Specifications by Ian G. Harris, Natural Language Problem Definition for Computer-aided Mechanical Design by Hyunmin Cheong, Wei Li, Li Shu, Alex Tessier, Erin Bradner and Francesco Iorio, Describing the Design Space of Mechanical Computational Design Synthesis Problems by Klaas Gadeyne, Gregory Pinte and Kristof Berx, Verbal Constraint Management for Shape Conceptualization by J. C. Verlinden, Imre Horváth and György Kuczogi, Computer Comprehension of Mechanical Device Descriptions by Michael G. Dyer, Jack Hodges and Margot Flowers, Verbal Interface for Vague Discrete Shape Modeler by György Kuczogi, Imre Horváth and Zoltán Rusák, Multimodal Discourse in Mediated Spaces by Paul McIlvenny and Chaim Noy, Cognitive Principles in Multimodal Interpretation by Joyce Y. Chai and Zahar Prasov, Context-based Multimodal Input Understanding in Conversational Systems by Joyce Y. Chai, Shimei Pan, Michelle X. Zhou and Keith Houck and A Comparative Analysis of Multimodal Communication during Design Sketching in Co-located and Distributed Environments by Ozgur Eris, Nikolas Martelaro and Petra Badke-Schaub.

Sketch

Speech and Sketching: An Empirical Study of Multimodal Interaction by Aaron Adler and Randal Davis, A Comparative Analysis of Multimodal Communication during Design Sketching in Co-located and Distributed Environments by Ozgur Eris, Nikolas Martelaro and Petra Badke-Schaub, Designing the User Interface for Multimodal Speech and Pen-based Gesture Applications: State-of-the-art Systems and Future Research Directions by Sharon Oviatt, Phil Cohen, Lizhong Wu, John Vergo, Lisbeth Duncan, Bernhard Suhm, Josh Bers, Thomas Holzman, Terry Winograd, James Landay, Jim Larson and David Ferro, Multimodal Redundancy across Handwriting and Speech during Computer Mediated Human-human Interactions by Edward C. Kaiser, Paulo Barthelmess, Candice Erdmann and Phil Cohen, Using Speech to Identify Gesture Pen Strokes in Collaborative, Multimodal Device Descriptions by James Herold and Thomas F. Stahovich, Combining Speech and Sketch to Interpret Unconstrained Descriptions of Mechanical Devices by David Tyler Bischel, Thomas F. Stahovich, Eric Jeffrey Peterson, Randall Davis and Aaron Adler and Using Speech to Identify Gesture Pen Strokes in Collaborative, Multimodal Device Descriptions by James Herold and Thomas F. Stahovich.

Gesture

Research on Collaborative Interaction Based on Gesture and Sketch in Conceptual Design by Cuixia Ma, Hongan Wang, Guozhong Dai and Youdi Chen, Hand Gesture Based Intuitive CAD Interface by Jinmiao Huang and Rahul Rai, Gestures and Phases: The Dynamics of Speech-hand Communication by Paul Treffner, Mira Peter and Mark Kleidon, Towards the Integration of Shape-related Information in 3-D Gestures and Speech by Sowa Timo, Towards a Natural User Interface for Comprehensive Support of Conceptual Shape Design by György Kuczogi, Zoltán Rusák and Imre Horváth, Making Digital Shapes by Hand by Steven Schkolne, Using Hand Motions in Conceptual Shape Design: Theories, Methods and Tools by Edit Varga, A Cognitive Model for the Representation and Processing of Shape-related Gestures by Sowa Timo and Stefan Kopp, Understanding Coverbal Iconic Gestures in Shape Descriptions by Sowa Timo, Coverbal Iconic Gestures for Object Descriptions in Virtual Environments: An Empirical Study by Sowa Timo and Ipke Wachsmuth, Understanding Coverbal Dimensional Gestures in a Virtual Design Environment by Sowa Timo and Ipke Wachsmuth, A Model for the Representation and Processing of Shape in Coverbal Iconic Gestures by Sowa Timo and Ipke Wachsmuth, A Computational Model for the Representation and Processing of Shape in Coverbal Iconic Gestures by Sowa Timo and Ipke Wachsmuth, Gestures in Spatial Descriptions by Marianne Gullberg, Gestures, Mental Imagery and Spatial Reasoning by U. Athavankar, Gestures Maintain Spatial Imagery by Richard Wesp, Jennifer Hesse, Donna Keutmann and Karen Wheaton, Gesture in Spatial Cognition: Expressing, Communicating, and Thinking about Spatial Information by Martha W. Alibali, The Role of Gestures in Mental Animation by Mary Hegarty, Sarah Mayer, Sarah Kriz and Madeleine Keehner, Visible Embodiment: Gestures as Simulated Action by Autumn B. Hostetter and Martha W. Alibali, Gesture and Imagination: On the Constitution and Uses of Phantasms by Nemirovsky Ricardo, Molly L. Kelton and Bohdan Rhodehamel, Illuminating Mental Representations through Speech and Gesture by Martha W. Alibali, Miriam Bassok, Karen O. Solomon, Sharon E. Syc and Susan Goldin-Meadow and Towards Integrated Microplanning of Language and Iconic Gesture for Multimodal Output by Stefan Kopp, Paul Tepper and Justine Cassell.

Pragmatics

A Representation of Context for Computer Supported Collaborative Design by Ming Xi Tang and John Frazer, Towards Context-sensitive Modeling Tools by Raluca Dumitrescu, Joris S. M. Vergeest, Tjamme Wiegers and Chensheng Wang, Design Work in Contexts – Contexts in Design Work by Friedrich Glock, Interpretation in Design: Modelling how the Situation Changes during Design Activity by Nick Kelly and John S. Gero, Interaction, External Representation and Sense Making by David Kirsh, A Framework for the Study of External Representations in Collaborative Design Settings by Erica de Vries and Cédric Masclet, Using Artefacts to Mediate Understanding in Design Conversations by Rachael Luck, Using Objects to Coordinate Design Activity in Interaction by Rachael Luck, Thinking Together with Material Representations: Joint Epistemic Actions in Creative Problem Solving by Johanne S. Bjørndahl, Riccardo Fusaroli, Svend Østergaard and Kristian Tylén, The Role of Objects in Design Co-operation: Communication through Physical or Virtual Objects by Claudia Eckert and Jean-François Boujut, Conversations around Design Sketches: Use of Communication Channels for Sharing Mental Models during Concept Generation by Ariff N. S. N. Ahmad, Petra Badke-Schaub and Ozgur Eris, The Role of Mental Models in Collaborative Sketching by Wei Xiang, Lingyun Sun, Shi Chen, Zhiyuan Yang and Zheng Liu, A Situation Model to Support Awareness in Collaborative Design by Farouk Belkadi, Eric Bonjour, Mauricio Camargo, Nadège Troussier and Benoit Eynard, Shared Design Thinking Process Model for Supporting Collaborative Design by Junpeng Du, Shikai Jing and Jihong Liu, Shared Understanding as a Condition for Team Design by Rianne C. Valkenburg, Team Mental Models in Design by Petra Badke-Schaub, Kristina Lauche and André Neumann, The Role of Design Team Interaction Structure on Individual and Shared Mental Models by Matthew Wood, Pinzhi Chen, Katherine Fu, Jonathan Cagan and Kenneth Kotovsky, Investigating Design Cognition in the Construction and Enactment of Team Mental Models by Andy Dong, Maaike S. Kleinsmann and Fleur Deken, Dynamic Aspects of Design Cognition: Elements for a Cognitive Model of Design by Willemien Visser, Cognitive Processes in Iterative Design Behavior by Robin S. Adams and Cynthia J. Atman, Study of Mental Iteration in Different Design Situations by Yan Jin and Pawat Chusilp, Impact of Mental Iteration on Concept Generation by Pawat Chusilp and Yan Jin, Cognitive Modeling of Iteration in Conceptual Design by Pawat Chuslip and Yan Jin, Cognitive Effects of Physical Models in Engineering Idea Generation by Cherickal Viswanathan and Vimal Kumar, Effects of Representations in Engineering Idea Generation Process by Cherickal Viswanathan and Vimal Kumar, Mechanisms and Representations of Language-mediated Visual Attention by Falk Huettig, Ramesh K. Mishra and Christian N. L. Olivers, Interaction of Language and Visual Attention: Evidence from Production and Comprehension by Ramesh K. Mishra, On the Mental Representations Originating during the Interaction between Language and Vision by Ramesh K. Mishra, Neural Bases of Image and Language Interactions by Bernard Mazoyer, Nathalie Tzourio‐Mazoyer, Angélique Mazard, Michel Denis and Emmanuel Mellet, Dynamically Structuring, Updating and Interrelating Representations of Visual and Linguistic Discourse Context by John Kelleher, Fintan Costello and Josef van Genabith, Computational Models for Integrating Linguistic and Visual Information: A Survey by Rohini K. Srihari, Developing Multiagent Systems for Design Activity Analysis by Denis Choulier, Alain-Jérôme Fougères and Egon Ostrosi, Context-aware Agents in Cooperative Design Environment by Hong Liu, Context-driven Interaction in Immersive Virtual Environments by Scott Frees and Context-based Multimodal Input Understanding in Conversational Systems by Joyce Chai, Shimei Pan, Michelle X Zhou and Keith Houck.

Computer-aided Design

Towards a Natural Language Interface for CAD by Tariq Samad and Stephen W. Director, Natural Voice-enabled CAD: Modeling via Natural Discourse by Sukui Xue, Xinyu Kou and Sooi T. Tan, About the Introduction of a Dialogue-based Interaction within CAD Systems by Niccolò Becattini, Yuri Borgianni, Gaetano Cascini and Federico Rotini, Intuitive 3D Computer-aided Design (CAD) System with Multimodal Interfaces by Vishwas G. Nanjundaswamy, Amit Kulkarni, Zhuo Chen, Prakhar Jaiswal, Anoop Verma and Rahul Rai, Knowledge-guided Inference for Voice-enabled CAD by Xinyu Kou, Sukui Xue and Sooi T. Tan, Knowledge Based Engineering: Between AI and CAD: Review of a Language Based Technology to Support Engineering Design by Gianfranco La Rocca and Cognitive, Collaborative, Conceptual and Creative — Four Characteristics of the Next Generation of Knowledge-based CAD Systems: A Study in Biologically Inspired Design by Ashok K. Goel, Swaroop Vattam, Bryan Wiltgen and Michael Helms.