Multimodal discursive practices in narrative discourse production in a non-native language

Aim. The study identifies the characteristics of multimodal discursive practices in Russian as a foreign language mediated by the level of language competence (B1/B2 vs. C1/C2)

Methodology. The primary method of analysis was a polymodal experiment, which explores verbal and kinetic (manual) behavior of Russian language learners in their recalls of narrative texts. Additionally, the study exploits parametric analysis of local discourse structure and functional gestures.

Results. The study identifies characteristics of multimodal discursive practices in a non-native language, including: the manifestation of local discourse structure relations in speech, the distribution of functional gesture types, as well as the patterns of speech and gesture co-occurrence. The findings reveal that the primary differences in the use of discourse relations in speech pertain to the distribution of Relations of Agreement, Hesitation, and Joint (typical of the B1/B2 group) as opposed to Metadiscursive Relations and Elaboration (typical of the C1/C2 group). Furthermore, the analysis of gesture use accompanying these relations indicates that in order to manage the increased cognitive load during discourse production in a foreign language, speakers frequently employ representational and pragmatic gestures as illustrating discourse-related properties, which attests to their compensatory function in discourse construal.

Research implications. The research contributes to further development of multimodal frameworks exploring narrative discourse construal in a foreign language. Its results are applicable in teaching methodology and in the area of teaching Russian as a foreign language.

1. Khukhuni, G. T. & Valuitseva, I. I. (2023). The preservation of the ethnic and linguistic identity: unity and contradiction. In: Bulletin of the Moscow Region State University. Series: Linguistics, 3, 85–93. DOI: 10.18384/2310-712X-2023-3-85-93 (in Russ.).

2. Wetzel, M., Zufferey, S. & Gygax, P. (2020). Second language acquisition and the mastery of discourse connectives: Assessing the factors that hinder L2-learners from mastering French connectives. In: Languages, 5 (3), article 35. DOI: 3390/languages5030035.

3. Kibrik, A. A. & Podlesskaya, V. I. (2009). Dream Stories: A Corpus-Based Study of Oral Russian Discourse. Moscow: Yazyki slavyanskikh kultur publ. (in Russ.).

4. Litvinenko, A. L. Describing discourse structure in terms of Rhetorical Structure Theory: Application to Russian material. In: Proceedings of the International Seminar “Dialogue 2001” on Computational Linguistics and its Applications (Aksakovo, June 11, 2001 – June 13, 2001). Vol. 1. Moscow: Nauka publ., pp. 159–168 (in Russ.).

5. Yi, E. & Koenig, J. P. (2020). Grammar modulates discourse expectations: evidence from casual relations in English and Korean. In: Language and Cognition, 13 (1), 99–127. DOI: 10.1017/langcog.2020.29.

6. Kiose, M. I., Rzheshevskaya, A. A. & Petrov, A. A. (2025). Local structure of descriptive discourse in the speech of adult learners of Russian as a foreign language. In: Computational Linguistics and Intellectual Technologies: Papers from the Annual International Conference “Dialogue” (Moscow, April 23, 2025). Iss. 23. Moscow: Institute of Russian Language publ., pp. 158–168. DOI: 10.28995/2075-7182-2025-23-158-168 (in Russ.).

7. McNeill D. (1992). Hand and Mind: What Gestures Reveal about Thought. Chicago: Chicago University Press.

8. Goldin-Meadow, S. (2000). Hearing gesture: How our hands help us think. Cambridge, USA: Harvard University Press.

9. Janjić, P., Arikan, G., Gudde, H. B., Murphy, S. J. J., Sivaram, L. & Coventry, K. R (2025). The role of pointing gestures and eye gaze in second language vocabulary learning. In: Discourse Processes, 62 (8–9). 673–691. DOI: 10.1080/0163853X.2024.2343625.

10. Macedonia, M., Müller, K. & Friederici, A. D. (2011). The impact of iconic gestures on foreign language word learning and its neural substrate. In: Human Brain Mapping, 32 (6), 982–998. DOI: 10.1002/hbm.21084 (in Russ.).

11. García-Gámez, A. B. & Macizo, P. (2023). Gestures as Scaffolding to Learn Vocabulary in a Foreign Language. In: Brain Sciences, 13 (21), article 1712. DOI: 10.3390/brainsci13121712.

12. Iriskhanova, О. K. (2023). Simultaneous interpreter in multimodal dimension: the role of gestures in moments of non-interpretation. In: Slovo.ru: Baltic accent, 14 (3), 40–51. DOI: 10.5922/2225-5346-2023-3-3 (in Russ.).

13. Cienki, A. (2024). Self-focused versus dialogic features of gesturing during simultaneous interpreting. In: Russian Journal of Linguistics, 28 (2), 227–242. DOI: 10.22363/2687-0088-34572.

14. Gallo, F. & Abutalebi, J. (2024). The unique role of bilingualism among cognitive reserveenhancing factors. In: Bilingualism: Language and Cognition, 27 (2), 287–294. DOI: 10.1017/S1366728923000317.

15. Iriskhanova, О., Kiose, M., Leonteva, A. & Agafonova, O. (2023). Vague reference in expository discourse: multimodal regularities of speech and gesture. In: Computational Linguistics and Intellectual Technologies: Papers from the Annual International Conference “Dialogue” (Moscow, June 14–16, 2023). Iss. 22. Moscow: Russian State University for the Humanities, pp. 172–180. DOI: 10.28995/2075-7182-2023-22-172-180 (in Russ.).

16. Potekhin, V. O. (2024). Speech and gesture alignment in immersive communication: Human-Human vs. Human-Computer interaction. In: Key Issues of Contemporary Linguistics, 2, 40–51. DOI: 10.18384/2949-5075-2024-2-40-51.

17. Schmid, V. (2003). Narratology. Moscow: Yazyki slavyanskikh kultur publ. (in Russ.).

18. Labov, W. & Waletzky, J. (1967). Narrative analysis: oral versions of personal experience. In: Helm, J. (ed.). Essays on the verbal and visual arts. Washington: University of Washington Press, pp. 12–44.

19. Korotaev, N. A. (2022). Annotating co-construction in a multichannel corpus. In: Cognitive Studies of Language, 4 (51), 393–397 (in Russ.).

20. Ma, Li, Martohardjono, G. & McClure, W. (2022). The functional roles of lexical devices in second language learners’ encoding of temporality: A study of Mandarin Chinese-speaking ESL learners. In: International Review of Applied Linguistics in Language Teaching, 60 (3), 563–597. DOI: 10.1515/iral-2018-0298 (in Russ.).