català | english | español | occitan

Research

Research lines

Multimodal Pragmatics: Intonational and gestural pragmatic meaning in human languages
This research project investigates the range of pragmatic meanings conveyed by prosody and gesture across languages. Our initial results show that (a) intonation and gestural patterns in some languages encode similar meanings to those encoded by other components of grammar (e.g., morphosyntax, pragmatic markers); and (b) they can interact in complex ways. Given the current lack of a unified treatment of intonational meaning, we work towards a full integration of intonational/gestural meaning into dynamic and multidimensional models of semantics.

Developing standards for Gesture: The MultiModal MultiDimensional (M3D) labeling system

The M3D labeling system represents a joint effort between three gesture labs (Language and
Cognition Research Group, UOC; Speech Communication Group, MIT; Prosodic Studies
Group, UPF) to design a comprehensive guide for the annotation of multimodal speech
corpora, particularly focusing on co-speech gestures, their relationship with prosody, and
their semantic and pragmatic properties. The first version of M3D can be found here:
https://osf.io/ankdx/

Cognitive and linguistic benefits of prosody and gesture. Their bootstrapping role in pragmatic and narrative development

This project includes a collection of studies aiming at testing the pragmatic and
discourse bootstrapping effects of prosodic and gestural patterns in first language
acquisition, specifically in complex language skills involving pragmatics and
narrative performance. At present, our results indicate that multimodal features
do not only enhance information recall and narrative comprehension but also act
as predictors and causal mechanisms for narrative and pragmatic performance.

Temporal patterns of gesture-speech integration in adults and children

We know that an appropriate use of prosodic and gestural strategies is essential in
the communication process. This research project investigates how prosodic and
gestural patterns are coordinated from a temporal and semantic point of view in
natural discourse, both in adult and children’s discourse. The findings of the
project reveal a strong relevance of prosodic structure in the gesture-speech
alignment patterns, both in adults and children’s discourse.

 

Multimodal and multisensory training of L2 pronunciation

This project involves a set of laboratory and classroom training studies assessing how multimodal and multisensory techniques can promote second language phonological learning in children and adults. The results show that rhythmic and melodic embodied trainings, as well as gesture-based and touch-based
articulatory trainings, are of help in the learning of foreign language
pronunciation.

Prosodic and gestural communication as bootstrapping mechanisms of language development. Testing a new treatment paradigm for ASD and LI developmental disorder

The innovative nature of this project resides in testing whether training with prosody and gesture has the potential to serve as a tool for intervention in non-impaired populations and populations with Autism Spectrum Disorder (ASD) and Language Developmental Disorder (DLD). Currently a randomized control trial intervention study is being designed to test the value of prosodic and gestural training in promoting language and pragmatic development. A set of behavioral measures will be recorded before and after treatment using both offline and online methods. The outcome of the study will represent a proof-of-concept of a clinically tested intervention tool intended for promoting the development of pragmatic
abilities.

Outreach projects / Dissemination projects

ORALITAT: Online workshops, resources, and tips for learning to speak in public (RecerCaixa Project
2018-2021)

ORALITAT has the goal of making current scientific knowledge on oral
communication skills available to teachers and the general public. In the online workshops, a group of experts go through a series of innovative public speaking techniques based on the latest scientific developments. The site also includes a set of practical tasks and useful resources for developing the user’s own communication skills, or those of people around them.

Before their first words (RecerCaixa Project 2014)

During the first year of their lives, infants are still not producing their first words. However, they can communicate with their caregivers using the so-called language precursors. The aim of this project was to assess the extent to which these patterns predict the infant’s later linguistic development. Our results indicate that the use of gestural and prosodic patterns are essential in language development.

 

Current funded projects

Untitled Document

  • GEHM: GEstures and Head Movements in language.
    (Awarded by Independent Research Fund Denmark (IRFD), July 1st, 2019  - February 28th, 2023. GrEP participation in the European Network)
    Principal Investigator: Patrizia Paggio, University of Copenhaguen
    Membres: Florence Baills, Patrick Rohrer, Ingrid Vilà-Giménez, Júlia Florit-Pons, and Pilar Prieto
  • Millorem la competència comunicativa oral a la Universitat.
    (Ajut PLACLIK Universitat Pompeu Fabra, REF: 52 PlaCLIK 2019-2020 2)
    Investigadora principal: Pilar Prieto
    Members: Joan Carles Mora, Ingrid Mora, Emma Rodero, Ïo Valls
    Amount awarded: 3650,00 euros
  • MULTIMODAL LANGUAGE LEARNING (MLL): Prosodic and Gestural Integration in Pragmatic and Phonological Development.
    (Ministerio de Ciencia, Innovación y Universidades / Spanish Ministry of Science, Innovation and Universities; ref.: PGC2018-097007-B-I00; 01/01/2019 - 31/12/2021).
    Investigadora principal: Pilar Prieto.
    Membres: Florence Baills, Joan Borràs-Comes, Iris Hübscher, Peng Li, Mariia Pronina, Patrick Rohrer, Ïo Valls, Ingrid Vilà-Giménez, Laura Vincze, Yuan Zhang, Xiaotong Xi, Júlia Florit-Pons.
    Amount awarded: 44.770 €
  • Grup de Recerca Consolidat (GRC) Grup d’Estudis de Prosòdia (GrEP).
    (Pla de Recerca de Catalunya: 2017 SGR _ 971, 01/04/18 - 01/04/21).
    Investigadora principal: Pilar Prieto.
    Membres: Lourdes Aguilar, Carme de-la-Mota, Laura Vincze, Iris Hübscher, Olga Kushch, Judith Llanes, Santiago González-Fuente, Florence Baills, Evi Kiagia, Ingrid Vilà-Giménez, Cristina Sánchez-Conde, Weiqi Li, Peng Li, Mariia Pronina, Patrick Rohrer, Ïo Valls, Yuan Zhang, Xiaotong Xi, Júlia Florit-Pons.
    Amount awarded: 26.029,25 €
  • Habilitats orals per a les professions del futur: un programa d'intervenció en el currículum de secundària i batxillerat a Catalunya.
    (RECERCAIXA 2017, project awarded by Obra Social ‘La Caixa’, February 25, 2018 – March 30, 2021).
    Investigadors principals: Pilar Prieto i Joan Carles Mora.
    Membres: Josep Maria Castellà, Emma Rodero, Montserrat Vilà, Ingrid Mora Plaza, Ïo Valls.
    Amount awarded: 79.905,15 €

 

Previous funded projects

Untitled Document

  • Intonational and gestural meaning in language.
    (MINECO, Ministerio de Economía y Competitividad / Spanish Ministry of Economy and Competitiveness; ref.: FFI2015-66533-P; 01/01/2016 - 31/12/2018).
    Investigadora principal: Pilar Prieto.
    Membres: Lluïsa Astruc, Joan Borràs-Comes, Teresa Cabré, Eva Estebas, Santiago González-Fuente, Alfonso Igualada, Evi Kiagia, Olga Kushch, Iris Hübscher, Maria Ohannesian.
    Amount awarded: 60.000 €
  • Grup de Recerca Consolidat (GRC) Grup d’Estudis de Prosòdia (GrEP).
    (Pla de Recerca de Catalunya: 2014 SGR _ 925, 01/01/14 - 31/12/16).
    Investigadora principal: Pilar Prieto.
    Membres: Lourdes Aguilar, Joan Borràs-Comes, Carme de-la-Mota, Emma Rodero, Yurena María Gutiérrez, Santiago González-Fuente, Núria Esteve Gibert, Alfonso Igualada, Olga Kushch, Iris Hübscher, Rafèu Sichel.
    Amount awarded: 14.000 €
  • Juguem a comunicar millor! La millora de la competència prosòdica com a via d'integració educativa i d'inclusió social de l'alumnat amb necessitats educatives especials derivades de la discapacitat.
    (RECERCAIXA, 2013, project awarded by Obra Social 'La Caixa', February 28, 2014 - February 28, 2016).
    Investigadora principal: Lourdes Aguilar
    Amount awarded: 77.642,60 €
  • Gestures, prosody and linguistic structure.
    (Ministerio de Economía y Competitividad [Ref.: BFU2012-31995], 01/01/2013- 31/12/2015).
    Investigadora principal: Pilar Prieto.
    Membres: Lluïsa Astruc, Meghan Armstrong, Joan Borràs-Comes, Teresa Cabré, Eva Estebas, Núria Esteve, Santiago González-Fuente, María Ohannesian, Maria del Mar Vanrell.
    Amount awarded: 60.000 €
  • Prosodic strategies to improve the attention and recall of a listener exposed to an auditory stimulus
    (European Union [PIOF-GA-2012-328636], 2013 – 2016).
    Investigadora principal: Emma Rodero.
    Amount awarded: 313.080 €
  • Els precursors del llenguatge. Una guia TIC per a pares i educadors.
    (RECERCAIXA 2012, project awarded by Obra Social ‘La Caixa’, February 28, 2013 – February 28, 2015).
    Investigadora principal: Pilar Prieto.
    Membres: Laura Bosch, Núria Esteve, Alfonso Igualada, Ferran Pons, Jorgina Solé.
    Amount awarded: 74.917,61 €
  • Grup de Recerca Emergent (GRE) Grup d’Estudis de Prosòdia (GrEP)
    (Pla de Recerca de Catalunya: 2009 SGR-701, 29/09/09 - 30/​04/14)
    Investigadora principal: Pilar Prieto.
    Membres: Lourdes Aguilar, Meghan Armstrong, Lluïsa Astruc, Joan Borràs-Comes, Verònica Crespo-Sendra, Núria Esteve Gibert, Carme de-la-Mota, Eva Estebas, Rafèu Sichel-Bazin, Maria del Mar Vanrell.
    Amount awarded: 33.600 €
  • Modelización de los fenómenos prosódicos del español y catalán a partir del corpus Glissando.
    (Ministerio de Educación y Ciencia FFI 2011-29559-C02-02, January 2011 - December 2013)
    Investigadora principal: Lourdes Aguilar
    Membres subprojecte UAB: Mercedes Cabrera, Carme de-la-Mota, Eva Estebas, Francisco Vizcaíno.
  • The role of tonal scaling and tonal alignment in distinguishing intonational categories in Catalan and Spanish.
    (Ministerio de Ciencia e Innovación [Ref.: FFI2009-07648/FILO], January 1st, 2010 - December 31st, 2012)
    Investigadora principal: Pilar Prieto.
    Membres: Lluïsa Astruc, Meghan Armstrong, Joan Borràs-Comes, Teresa Cabré, Verònica Crespo-Sendra, Eva Estebas, Núria Esteve, María Ohannesian, Maria del Mar Vanrell.
    Amount awarded: 88.330 €

Design and implementation: Optimitza.cat / Joan Borràs-Comes