Research

Research lines

Multimodal Pragmatics: Intonational and gestural pragmatic meaning in human languages

This research project investigates the range of pragmatic meanings conveyed by prosody and gesture across languages. Our initial results show that (a) intonation and gestural patterns in some languages encode similar meanings to those encoded by other components of grammar (e.g., morphosyntax, pragmatic markers); and (b) they can interact in complex ways. Given the current lack of a unified treatment of intonational meaning, we work towards a full integration of intonational/gestural meaning into dynamic and multidimensional models of semantics.

Developing standards for Gesture: The MultiModal MultiDimensional (M3D) labeling system

The M3D labeling system represents a joint effort between three gesture labs (Language and Cognition Research Group, UOC; Speech Communication Group, MIT; Prosodic Studies Group, UPF) to design a comprehensive guide for the annotation of multimodal speech corpora, particularly focusing on co-speech gestures, their relationship with prosody, and their semantic and pragmatic properties. The first version of M3D can be found here: https://osf.io/ankdx/

Cognitive and linguistic benefits of prosody and gesture. Their bootstrapping role in pragmatic and narrative development

This project includes a collection of studies aiming at testing the pragmatic and discourse bootstrapping effects of prosodic and gestural patterns in first language acquisition, specifically in complex language skills involving pragmatics and narrative performance. At present, our results indicate that multimodal features do not only enhance information recall and narrative comprehension but also act as predictors and causal mechanisms for narrative and pragmatic performance.

Temporal patterns of gesture-speech integration in adults and children

We know that an appropriate use of prosodic and gestural strategies is essential in the communication process. This research project investigates how prosodic and gestural patterns are coordinated from a temporal and semantic point of view in natural discourse, both in adult and children’s discourse. The findings of the project reveal a strong relevance of prosodic structure in the gesture-speech alignment patterns, both in adults and children’s discourse.

Multimodal and multisensory training of L2 pronunciation
	This project involves a set of laboratory and classroom training studies assessing how multimodal and multisensory techniques can promote second language phonological learning in children and adults. The results show that rhythmic and melodic embodied trainings, as well as gesture-based and touch-based articulatory trainings, are of help in the learning of foreign language pronunciation.

Prosodic and gestural communication as bootstrapping mechanisms of language development. Testing a new treatment paradigm for ASD and LI developmental disorder

The innovative nature of this project resides in testing whether training with prosody and gesture has the potential to serve as a tool for intervention in non-impaired populations and populations with Autism Spectrum Disorder (ASD) and Language Developmental Disorder (DLD). Currently a randomized control trial intervention study is being designed to test the value of prosodic and gestural training in promoting language and pragmatic development. A set of behavioral measures will be recorded before and after treatment using both offline and online methods. The outcome of the study will represent a proof-of-concept of a clinically tested intervention tool intended for promoting the development of pragmatic abilities.

Outreach projects / Dissemination projects ORALITAT: Online workshops, resources, and tips for learning to speak in public (RecerCaixa Project 2018-2021)

ORALITAT has the goal of making current scientific knowledge on oral communication skills available to teachers and the general public. In the online workshops, a group of experts go through a series of innovative public speaking techniques based on the latest scientific developments. The site also includes a set of practical tasks and useful resources for developing the user’s own communication skills, or those of people around them.

Before their first words (RecerCaixa Project 2014)

During the first year of their lives, infants are still not producing their first words. However, they can communicate with their caregivers using the so-called language precursors. The aim of this project was to assess the extent to which these patterns predict the infant’s later linguistic development. Our results indicate that the use of gestural and prosodic patterns are essential in language development.

Current funded projects

• Multimodal Communication (MCOM): The integration of prosody and gesture in human communication and in language learning
  (Proyectos de generación de conocimiento 2021; Ministerio de Ciencia e Innovación / Spanish Ministry of Science and Innovation; ref.: PID2021-123823NB-I00); 01/09/2022 - 01/09/2025
  Investigadora principal: Pilar Prieto.
  Membres: Celia Gorba, Ingrid Vilà-Giménez, Júlia Florit-Pons, Sara Coego, Mariia Pronina, Patrick Rohrer, Ïo Valls, Xiaotong Xi, Yuan Zhang.
  Amount awarded: 85.305 €

• MultIS: Co-speech gestures and prosody as multimodal markers of information structure
  (German Research Foundation (DFG Priority Programme 2329) as part of the project Visual Communication. Theoretical, Empirical, and Applied Perspectives (ViCom); 01/09/22 – 31/08/25
  Investigadors principals: Frank Kügler & Pilar Prieto.
  Membres: Paula G. Sánchez-Ramón & Alina Gregori.
  Amount awarded: 386.436€

• GEHM: GEstures and Head Movements in language
  (Awarded by Independent Research Fund Denmark (IRFD), July 1st, 2019  - February 28th, 2023. GrEP participation in the European Network)
  Principal Investigator: Patrizia Paggio, University of Copenhaguen
  Membres: Florence Baills, Patrick Rohrer, Ingrid Vilà-Giménez, Júlia Florit-Pons, and Pilar Prieto.

• Grup de Recerca Consolidat (GRC) Grup d’Estudis de Prosòdia i Gestualitat (GrEPG).
  (Pla de Recerca de Catalunya: 2021 SGR 00922, 01/03/2023 - 01/03/25).
  Investigadora principal: Pilar Prieto.
  Membres: Lourdes Aguilar, Domenico Baglioni, Sara Coego, Júlia Florit-Pons, Celia Gorba, Jiaqi Huang, Weiqi Li, Mariia Pronina, Patrick Louis Rohrer, Paula Sánchez-Ramón, Ïo Valls-Ratés, Ingrid Vilà-Giménez, Xuanxi Wu, Xiaotong Xi, Ting Yao, Yuan Zhang.
  Amount awarded: 60.000 €

• StoryTIC: Una eina digital per potenciar la narrativa oral infantil 
  (Agència de Gestió d’Ajuts Universitaris i de Recerca AGAUR – Generalitat de Catalunya; ref.: 2021 LLAV 00022) 2022-2023.
  Principal Investigator: Dr. Pilar Prieto
  Membres: Júlia Florit-Pons, Alfonso Igualada, Florence Baills, & Ïo Valls-Ratés.
  Amount awarded: 20.000€

Previous funded projects

• Millorem la competència comunicativa oral a la Universitat.
  (Ajut PLACLIK Universitat Pompeu Fabra, REF: 52 PlaCLIK 2019-20202)
  Investigadora principal: Pilar Prieto
  Members: Joan Carles Mora, Ingrid Mora, Emma Rodero, Ïo Valls
  Amount awarded: 3650,00 euros

• MULTIMODAL LANGUAGE LEARNING (MLL): Prosodic and Gestural Integration in Pragmatic and Phonological Development.
  (Ministerio de Ciencia, Innovación y Universidades / Spanish Ministry of Science, Innovation and Universities; ref.: PGC2018-097007-B-I00; 01/01/2019 - 31/12/2021).
  Investigadora principal: Pilar Prieto.
  Membres: Florence Baills, Joan Borràs-Comes, Iris Hübscher, Peng Li, Mariia Pronina, Patrick Rohrer, Ïo Valls, Ingrid Vilà-Giménez, Laura Vincze, Yuan Zhang, Xiaotong Xi, Júlia Florit-Pons.
  Amount awarded: 44.770 €

• Grup de Recerca Consolidat (GRC) Grup d’Estudis de Prosòdia (GrEP).
  (Pla de Recerca de Catalunya: 2017 SGR _ 971, 01/04/18 - 01/04/21).
  Investigadora principal: Pilar Prieto.
  Membres: Lourdes Aguilar, Carme de-la-Mota, Laura Vincze, Iris Hübscher, Olga Kushch, Judith Llanes, Santiago González-Fuente, Florence Baills, Evi Kiagia, Ingrid Vilà-Giménez, Cristina Sánchez-Conde, Weiqi Li, Peng Li, Mariia Pronina, Patrick Rohrer, Ïo Valls, Yuan Zhang, Xiaotong Xi, Júlia Florit-Pons.
  Amount awarded: 26.029,25 €

• Habilitats orals per a les professions del futur: un programa d'intervenció en el currículum de secundària i batxillerat a Catalunya.
  (RECERCAIXA 2017, project awarded by Obra Social ‘La Caixa’, February 25, 2018 – March 30, 2021).
  Investigadors principals: Pilar Prieto i Joan Carles Mora.
  Membres: Josep Maria Castellà, Emma Rodero, Montserrat Vilà, Ingrid Mora Plaza, Ïo Valls.
  Amount awarded: 79.905,15 €

• Intonational and gestural meaning in language.
  (MINECO, Ministerio de Economía y Competitividad / Spanish Ministry of Economy and Competitiveness; ref.: FFI2015-66533-P; 01/01/2016 - 31/12/2018).
  Investigadora principal: Pilar Prieto.
  Membres: Lluïsa Astruc, Joan Borràs-Comes, Teresa Cabré, Eva Estebas, Santiago González-Fuente, Alfonso Igualada, Evi Kiagia, Olga Kushch, Iris Hübscher, Maria Ohannesian.
  Amount awarded: 60.000 €

• Grup de Recerca Consolidat (GRC) Grup d’Estudis de Prosòdia (GrEP).
  (Pla de Recerca de Catalunya: 2014 SGR _ 925, 01/01/14 - 31/12/16).
  Investigadora principal: Pilar Prieto.
  Membres: Lourdes Aguilar, Joan Borràs-Comes, Carme de-la-Mota, Emma Rodero, Yurena María Gutiérrez, Santiago González-Fuente, Núria Esteve-Gibert, Alfonso Igualada, Olga Kushch, Iris Hübscher, Rafèu Sichel.
  Amount awarded: 14.000 €

• Juguem a comunicar millor! La millora de la competència prosòdica com a via d'integració educativa i d'inclusió social de l'alumnat amb necessitats educatives especials derivades de la discapacitat.
  (RECERCAIXA, 2013, project awarded by Obra Social 'La Caixa', February 28, 2014 - February 28, 2016).
  Investigadora principal: Lourdes Aguilar
  Amount awarded: 77.642,60 €

• Gestures, prosody and linguistic structure.
  (Ministerio de Economía y Competitividad [Ref.: BFU2012-31995], 01/01/2013- 31/12/2015).
  Investigadora principal: Pilar Prieto.
  Membres: Lluïsa Astruc, Meghan Armstrong, Joan Borràs-Comes, Teresa Cabré, Eva Estebas, Núria Esteve-Gibert, Santiago González-Fuente, María Ohannesian, Maria del Mar Vanrell.
  Amount awarded: 60.000 €

• Prosodic strategies to improve the attention and recall of a listener exposed to an auditory stimulus
  (European Union [PIOF-GA-2012-328636], 2013 – 2016).
  Investigadora principal: Emma Rodero.
  Amount awarded: 313.080 €

• Els precursors del llenguatge. Una guia TIC per a pares i educadors.
  (RECERCAIXA 2012, project awarded by Obra Social ‘La Caixa’, February 28, 2013 – February 28, 2015).
  Investigadora principal: Pilar Prieto.
  Membres: Laura Bosch, Núria Esteve-Gibert, Alfonso Igualada, Ferran Pons, Jorgina Solé.
  Amount awarded: 74.917,61 €

• Grup de Recerca Emergent (GRE) Grup d’Estudis de Prosòdia (GrEP)
  (Pla de Recerca de Catalunya: 2009 SGR-701, 29/09/09 - 30/​04/14)
  Investigadora principal: Pilar Prieto.
  Membres: Lourdes Aguilar, Meghan Armstrong, Lluïsa Astruc, Joan Borràs-Comes, Verònica Crespo-Sendra, Núria Esteve-Gibert, Carme de-la-Mota, Eva Estebas, Rafèu Sichel-Bazin, Maria del Mar Vanrell.
  Amount awarded: 33.600 €

• Modelización de los fenómenos prosódicos del español y catalán a partir del corpus Glissando.
  (Ministerio de Educación y Ciencia FFI 2011-29559-C02-02, January 2011 - December 2013)
  Investigadora principal: Lourdes Aguilar
  Membres subprojecte UAB: Mercedes Cabrera, Carme de-la-Mota, Eva Estebas, Francisco Vizcaíno.

• The role of tonal scaling and tonal alignment in distinguishing intonational categories in Catalan and Spanish.
  (Ministerio de Ciencia e Innovación [Ref.: FFI2009-07648/FILO], January 1st, 2010 - December 31st, 2012)
  Investigadora principal: Pilar Prieto.
  Membres: Lluïsa Astruc, Meghan Armstrong, Joan Borràs-Comes, Teresa Cabré, Verònica Crespo-Sendra, Eva Estebas, Núria Esteve, María Ohannesian, Maria del Mar Vanrell.
  Amount awarded: 88.330 €