Journal articles on the topic 'Interpersonal meaning, multimodal discourse analysis, social semiotics'

ABSTRACTThis article attempts to investigate and explore the interpersonal meaning of YOU C1000 on Indonesian television advertisements. This study was conducted qualitatively using case study to check how different semiotic and modes such as music, sound, speech, color, action, and image work together to build the interpersonal meaning. This study discusses the interpersonal meaning in speech and music, interpersonal meaning in movement and interpersonal meaning in image and color. The study aimed to give some contributions to social semiotics studies, television, or video advertisement. YOU C1000 advertisement is successful to attract audiences’ attention. The election of Miss Universe advertisement star, Bali as the shooting location, English as the language function and wedding ceremony as the activity are the significant factors to introduce the product to the market. In addition, its tagline is a successor factor as well. If people hear Healthy Inside and Fresh Outside, they will remember YOU C1000. ABSTRAKArtikel ini bertujuan untuk menginvestigasi and menyelidiki makna interpersonal dari produk ‘YOU C1000’ pada iklan televisi Indonesia. Penelitian ini dilakukan secara kualitatif dengan menggunakan pendekatan studi kasus untuk mengetahui bagaimana semiotik dan moda yang berbeda seperti musik, suara, ucapan, warna, tindakan, dan gambar, dapat berjalan bersama-sama dalam membentuk makna interpersonal. Penelitian ini mengulas makna interpersonal dalam ucapan dan musik, makna interpersonal dalam gerakan, dan makna interpersonal dalam gambar dan warna. Penelitian ini bertujuan untuk memberikan kontribusi pada studi semiotika sosial, televisi, atau iklan video. Iklan YOU C1000 berhasil menarik perhatian penonton. Pemilihan bintang iklan Miss Universe, Bali sebagai lokasi syuting, bahasa Inggris sebagai fungsi bahasa dan upacara pernikahan sebagai aktivitasnya adalah faktor-faktor yang signifikan untuk memperkenalkan produk tersebut ke pasar. Selain itu, ‘tagline’-nya juga merupakan faktor penting lainnya. Jika orang-orang mendengar ‘Healthy Inside and Fresh Outside’, mereka akan mengingat YOU C1000. How to Cite: Hidayat, D. N.., Abrizal., Alek. (2018). A Multimodal Discourse Analysis of the Interpersonal Meaning of a Television Advertisement in Indonesia. IJEE (Indonesian Journal of English Education), 5(2), 119-126. doi:10.15408/ijee.v5i2.11188

The current paper presents an overview of interpersonal conflicts in discourse area of character in modern English fiction discourse from the standpoint of multimodality theory, pragmalinguistics, and semiotics. In this respect semiosis is defined as the action of a sign, a dynamic process of meaning-making and meaning-interpretation realized through multimodal semiotic modes which collectively construct the meaning, communicated in these situations. This constructing is proceeded with the help of conflictives as emergent discursive constructs, the result of interactive constructing by means of verbal, nonverbal and graphic semiotic resources functioning in different stages of conflict communicative process. The linguosemiotic space of their realization is in the plane of disharmony of interpersonal relations of characters and its semiosis is built on cognitive, semiotic, communicative, and pragmatic specifics of conflictives as operational units of conflict discourse. Moreover, the appropriate inferences require understanding of cognitive, psychological, social, and cultural aspects accompanying narration.The aim of the study is also to establish a link between different approaches to the interpretation of conflict communication development and methods of their research in modern scientific studies. Multimodal nature of conflictives comprises several modes of multimodality for the analysis of conflict semiosis in fiction discourse: verbal, nonverbal, visual, auditory, kinetic etc. These patterns of meaning combination or meaning multiplication through different semiotic modes together construct the meaning, communicated and interpreted in the situations of interpersonal conflicts in discourse area of character in modern English fiction discourse. To achieve the objectives of research, a semiotic approach to the paradigm of conflict discourse approaches is applied, together with the elements of conversational analysis, multimodal discourse analysis, pragmatic analysis. The obtained results show the capacity of the semiotic approach to the conflict studies to enhance the effectiveness of linguistic research in the field of conflict studies.

This article offers a multimodal discourse analysis of the picturebook Daddy, Papa, and Me (2009), in which there is a two-father family with a child. The aim of this article is to analyse how the written text and the visual create meaning and to observe if the child has a symmetrical relationship with both fathers. We will follow the method of Visual Social Semiotics developed by Kress and van Leeuwen (2006) based on Halliday’s Systemic Functional Linguistics, in order to explore the representational, interpersonal and compositional meanings. The analysis will show that the child and his fathers are dynamic, as the actions represented in the visuals and the use of materials makes clear. However, the multimodal analysis does not encourage interaction with the readers, because most of the visuals are offers and the predominance of oblique angles. Images and written text are intergrated in the layout, but visuals are salient so that children can understand the meaning of the picturebook just looking at the images. The writer and the illustrator collaborate to tell the story in a simple way, with the aim of making the plot accessible to children and to invite them to identify with the actions being narrated.

One of the most significant shifts in Iraq's political regime was the parliamentary elections in 2021 with the election of a new prime minister. Massive demonstrations over government corruption and political parties' failure to enact measures to rein such corruption through legislative elections. It was possible that a new poll law might help the independent candidates in their efforts to secure a seat in the parliament. To explore this issue, the current study has employed two models: visual social semiotics of Kress and Van Leeuwen (2006) and Bednarek (2006) to analyze the visuals of Election Day and the evaluations of news values. The visuals convey representational, interpersonal, and compositional meanings, while the verbal texts are analyzed according to their evaluation characteristics. To demonstrate the different facets of election day that can be captured in the images chosen for the study on the BBC, the vocal text strives to explain how it relates to the visual style. Attempts have also been made to determine how the election day would be portrayed in the writer's news report for BBC. The results have demonstrated that the most significant aspects of election day could be reflected in the visual and spoken modes of BBC's less upbeat portrayal of Iraqi conditions which has been clear from this report.

With the advent of the information age, language is no longer the only way to construct meaning. Besides language, a variety of social symbols, such as gestures, images, music, three-dimensional animation, and so on, are more and more involved in the social practice of meaning construction. Traditional single-modal sentiment analysis methods have a single expression form and cannot fully utilize multiple modal information, resulting in low sentiment classification accuracy. Deep learning technology can automatically mine emotional states in images, texts, and videos and can effectively combine multiple modal information. In the book Image Reading, the first systematic and comprehensive visual grammatical analysis framework is proposed and the expression of image meaning is discussed from the perspectives of representational meaning, interactive meaning, and composition meaning, compared with the three pure theoretical functions in Halliday’s systemic functional grammar. In the past, people often discussed films from the macro perspectives of literary criticism, film criticism, psychology, aesthetics, and so on, and multimodal analysis theory provides film researchers with a set of methods to analyze images, music, and words at the same time. In view of the above considerations, Mu Wen adopts the perspective of social semiotics, based on Halliday’s systemic functional linguistics and Gan He’s “visual grammar,” and builds a multimodal interaction model as a tool to analyze film discourse by referring to evaluation theory.

This article presents a critical multimodal discourse analysis of how people make meaning through the semiotic practice of shooting digital self-portraits (selfies), adding captions and then sharing these texts on the social network site Instagram. Combining theories from social semiotics, critical discourse analysis and multimodal discourse analysis, the analysis focuses on the embedded ideological meaning in such digital communication. The analysis explores a data corpus of 100 selfies shared on Instagram. Despite the fact that digital texts shared on social media are generally regarded as personal communication, selfie makers seem to reproduce features of a commercial and global discourse. The typical way of representing oneself on Instagram appears to be surprisingly similar to visual representations in advertisements and image banks. The linguistic resources in use also appear globalized through a mix of languages combined with slang and abbreviations.

Movie Posters are visual media used to transmit cultural and commercial information through social semiotics such as image, text, font and color. One of the important techniques of poster design is to convey movie topics or focused information. Movie poster design has gone through two major periods in China. Between 1980s and 1990s, posters were apparently designed simple and color-vivid; entering the 21st century, with computer technology, Chinese movie posters have shown diversified expressions and propagandas, which seem more fresh and unique in artistic charm. Multimodality usually expresses meanings through the combination of text and other elements, and movie poster is one of the multimodal means. This paper discusses the findings of multimodal discourse analysis by Kress &Van Leeuwen carried out on a movie poster Little Big Soldier, of which the aims are, among others, to reveal how verbal and visual signs work together as social signs to interpret the representational, interactional and compositional meanings, thus viewers better understand how the movie poster realizes meaning co-construction and how it conveys the movie information. The main goal of poster message is to play a propaganda role and attract more audience to the movie.

This research is concerned with the compositional pattern of meaning in selected Nigerian military and paramilitary logos, how the logos presented their identity, and the appropriateness of visual and verbal elements. This paper is concerned with the analysis of patterns of meaning in visual and verbal components of selected Nigerian military and paramilitary logos. For visual analysis, ten logos were selected, and the texts (agency's name and motto), which accompanied the images , were grouped via 'below the clause.' Both visual and verbal components of the logos were analysed based on Kress and Van Leeuwen's social semiotics, and Halliday's systemic functional linguistics (henceforth, SFL). Our findings showed that the logo designers made use of animal (eagles, bat, horses, elephant and human eye) and object (flag, anchor, shovel, axe, flower, wheat leaves, passport, colours) participants. Analysis 'below the clause' presented Nominal Group (NG) with highest percentage which showed that the major focus of the communicators (military and paramilitary) is to persuade viewers. The verbal components of the logos were appropriately used to accompany the logos for easy understanding. Also, the selected colours are peculiar to Nigeria environment. Therefore viewers had no difficulty in getting the intended messages.

In the Philippines, “sex-sell” advertisements (ads), particularly the 2011 billboard by Bench featuring the Volcano Philippine Rugby Team, have been controversial. Community leaders view the ads as being offensive to the socio-cultural values of the community in which the ad is distributed. In response, producers of the ads contend that their works are merely creative options within the frames of the law. This paper approaches the arguments by investigating the linguistic functions of sex-sell ads through the grammatical analysis of its visual syntax. Using the framework of multimodal discourse analysis in visual semiotics, the paper demonstrates how grammar in visual language is compatible with the systemic functional language of Halliday (1985) as applied in the visual design theories of Kress (2006) and van Leeuwen (2005). The multimodal approach to the analysis of the linguistic functions of sexually implicit billboard ads seek to demonstrate the social semiotic perspective in the construction of meaning in print ads, how ambiguous visual lexicon can occur, and how a dissonant visual assemblage invites resistant reading. Through this discursive analysis, the paper hopes to promote a critical awareness among the passive interpreters of texts (i.e. general public information consumers) and the institutions (e.g. education, creative industry) that actively participate in the discourse, design, production, and distribution of meaning in advertising texts and images.

A good number of studies in the past have examined the language of sexism from the feminist perspectives, gender segregation and degradation, among others, using semiotics resources, discourse analysis, multimodal discourse, among other theories. This study looks at the linguistic and non-linguistic language features of sexist language as choices available to language users on the Facebook social media platform. Using the multimodal theory as the framework, the study examines 10 randomly selected Facebook posts with texted pictures and comments posted by Nigerians with elements of sexism. The study also engaged the descriptive research design to examine the ‘textedpictures’ used as sampled data. These sampled data were given in-depth analysis to reveal their usually hidden and laughed-about sexist messages. The analysis of data was considered by determining the existence of sexist communication on Facebook platform, examining the meaning making elements in sexist languages posts. This is precipitated on the discovery that less attention is paid on the signification of the communicative elements deployed to convey sexism on the Facebook platform. From the analysis, the study finds out that Facebook users engage linguistic and non-linguistic elements symbolising sexist language on Facebook postings; that the posts on Facebook rely predominantly on both written texts and pictures, combined to make the tagging or stereotyping concrete; that the sexist posts on Facebook platforms rely heavily on hasty or intentional generalisation in order to demean the sex they chose to target through texts, pictures and the combination of texts and pictures.

Two approaches to materiality (i.e. mediated discourse and agential realism) are compared to explore their usefulness in tracking literacies in action and artefacts produced during a play and design activity in a preschool makerspace. Mediated discourse analysis has relied on linguistic framing and social semiotics to make sense of multimodality. Can a multimodal lens grounded in embodied histories of meaning-making unpack sensory exploration, silly repetition and free-wheeling nonsense in children’s playdough play? Barad’s agential realism seems promising for unpacking the sensory and the emergent produced in the materiality, fluidity and messiness of entangled bodies and things in a makerspace. We compare key constructs of mediated discourse and agential realism, comparing interaction and intra-action in video excerpts from two weeks of play with playdough electronics kits in three early childhood classrooms in a US university childcare centre. Mediated discourse analysis of multimodality identified collaborative interaction among players in a small group and tracked a collective flow of materialized knowledge that moved through children’s sharing and collaboration. Agential realism tracked intra-actions among bodies, materials and spaces as transitory becomings and undoings that rupture definitions of sense-making as strategic design that manipulates materials into artefacts or as play that resemiotizes materials into roles and props in dramatized narratives.

Verbal language is vital in communication and is the most important means of interaction. By the significant role of verbal language, visuals are often less considered unnecessary, even though they could help to understand communication better. Verbal and visual language are interconnected and simultaneously build a strong, complete, structured text. This research focused on how a book writer communicates using two or more modes simultaneously, namely text and images, which is called multimodal discourse analysis, particularly social semiotics. This study analyzes the illustrated book "I Hate Diet" by Yulia Baltschun. Using a descriptive qualitative method that refers to the theory of reading images, Kress & Leeuwen's (2006) analysis relates to the existence of representational and interactive meanings with images divided into three aspects: information value, salience, and framing. In connection with the data analysis in the form of text or verbal, it is analyzed using the theory of seven development tasks by Gee (2011). The results indicated the meanings of the image or visual illustrations supported by verbal or text analysis. There are several representations of meaning in it, namely the meaning of motivational invitations and warnings or meanings that remind.

In the socio-political context of a strategic transformation of public diplomacy in China, non-governmental discourses such as intellectual discourses have been showing increasing visibility both online and offline, at home and abroad. Through a digital ethnographic approach, this study investigates the meaning-making of the political discourse that uses edutainment as a genre by foregrounding the media activities of a commercial media company branded by Guan Media and the media discourse of an involved intellectual. Social media are changing the normality of knowledge production and distribution and the traditional media communication logic. Whether it is market-based filter bubble and echo chamber, politically controlled censorship, or spontaneous grassroots engagement, what is important is why and how mainstream discourses are constructed because of these factors through social media as a new form of political communication. To show the complexity of media communication of political messages in China, micro-level close observation on highly visible forms of news production and distribution by non-government actors is necessary. There are two levels of analysis in this study: self-branding of researchers in the knowledge economy, and edutainment as a genre of political discourse. Multimodal discourse analysis is adopted to discover the specific discursive and media strategies through the theoretical lenses of knowledge-power structure and semiotics. The cooperation of commercial media companies and intellectuals from higher education is found to be promoting a new form of political communication, in which edutainment works as a genre for better media presentation. In the context of the knowledge economy, edutainment content adjusts to the ideological dynamics of the socio-political reality in China in the tide of globalization and digitalization. This study contributes to understanding the participation of non-governmental actors in political communication and public sentiment on politics when political communication has become more dynamic and better organized by adjusting to the new media age.

Bible Journaling is a trend of the past decade whereby readers make creative, visual interventions in their Bibles, using coloured pens and pencils, watercolours, stickers and stencils, highlighting texts of particular resonance. Journaling, in its more conventional written forms, has long been recognised as a pathway to spiritual development. Significantly, Bible journaling is almost exclusively practiced by women and has a high level of interpersonal interaction attached to it, through open and mutual sharing of these creations, through various online social media fora. Gleaned from the sharing of women who journal for spiritual support, this article examines the role Bible journaling plays in aiding recovery from drug addiction. Multimodal analysis is a methodological approach that provides a structured semiotic framework in which to closely examine every feature of a creation such as a journaled page of a Bible, to examine how the journaler has made meaning of a text through their interventions on the page. Appreciating every mark, choice and placement of image, colour, typography as a motivated sign revealing the interest of the creator, the sign-maker, a detailed multimodal analysis is conducted of one page of a recovered drug-user’s journaled Bible. As shall be demonstrated, profound insights into the appropriation of sacred texts for the spiritual life of a recovering addict can be gleaned in this process. Bible journaling reveals itself to be a highly valuable spiritual practice for those in addiction recovery. This interdisciplinary paper uniquely brings a methodological approach from the field of semiotics to the field of spirituality. Both the methodological approach and the subject of sacred text journaling may be of particular interest to spiritual directors, across many religions with a foundational sacred text, as a means whereby adherents can engage with a text in a deep, contemplative and creative practice that is personally, spiritually sustaining and motivating during a difficult phase of life.

<p><em> Los aprendices acceden al currículo escolar interpretando significados construidos a partir de una variedad de recursos semióticos (esquemas, fotos, dibujos, escritura, entre otros), este aprendizaje les permite incorporarse a una visión de mundo, forma de pensar y actuar en el mundo propia de una disciplina. Desde una mirada pedagógica y semiótica la interacción en el aula, se focaliza en el uso de las imágenes en la enseñanza, en relación a su potencial para crear significado y aprender en el aula escolar en clases de Ciencias Naturales y Ciencias Sociales. Este artículo es parte del Fondecyt 1130684 y sistematiza herramientas heurísticas de la Semiótica Social y la Multimodalidad para explorar el potencial semiótico de un conjunto de imágenes estáticas y dinámicas usadas por profesores de Educación Básica y Media en una escuela pública. A partir de un corpus audiovisual y visual de una unidad didáctica completa de Ciencias Naturales y otra de Ciencias Sociales de 3°, 6° básico y 1° medio, se lleva a cabo un Análisis Multimodal del Discurso. Usando los conceptos de metafunción ideacional o representacional y las categorías de la Gramática del Diseño Visual se presentan ejemplos de imágenes en soporte impreso y digital que explicitan los significados curriculares construidos en la mediación que realizan los profesores en el aula. Este análisis puede aportar a los profesores en función de aprovechar las imágenes seleccionadas e incorporadas en la mediación cara a cara o los materiales de enseñanza preparados para los aprendices.</em></p><p> </p><p> </p><p><em> </em></p><p><em>Abstract</em></p><p>Learners access the school curriculum interpreting meanings created among a variety of semiotic resources (diagrams, photographs, drawings, writing, etc.), this learning enables them to join a worldview, way of thinking and acting in the world as they do in the curricular discipline. From a pedagogical and semiotic gaze to classroom interaction, we focus on the use of images in teaching, in relation to their potential to create meaning and learn in science and social studies lessons. This article is part of Fondecyt 1130684 heuristic tools from Social Semiotics and multimodality are systematized to explore the semiotic potential of a set of static and dynamic images used by teachers of elementary and secondary in a public school. From a visual and audiovisual record of lessons of a complete curricular unit, we analyze a corpus of Science and Social Studies videos from the two subjects in 3rd, 6^th grade of elementary and 1st grade of secondary school. Through a Multimodal Discourse Analysis using the concepts of ideational or representational metafunction and the categories of Visual Grammar Design, we show examples of images in print and digital format which create curricular meanings constructed in teacher’s mediation in the classroom. This analysis should help teachers to select and deploy images in terms of improving the learning process.</p><p> </p><p> </p>

For researchers, early childhood educators, and art educators, the contribution of this article is to expand meaning in drawing activities. Perspective in reading pictures using visual culture theory, especially visual grammar. This study aims to share knowledge and experiences in reading early childhood pictures from different perspectives. This research method uses a qualitative descriptive approach through visual material data collection techniques and analysis of three metafunctions. The objects of research are three pictures of children aged 7-8 years, namely the works of winners of the I-III children's painting competition held by PP-IPTEK TMII in 2018. Aspects of the representation structure, interaction system, and composition of each image are analyzed. The research findings conclude that the ideational function of the three images shows a narrative structure of representation and raises the discourse of resistance to the actual situation. The interpersonal function of the three images places the image maker in the real world and as an object of display impersonally. The textual functions of the three images position social life on other planets as a reflection of hope for real social life.Keywords: children's drawings, visual culture, visual system, three metafunctions References: Butler, S., Gross, J., & Hayne, H. (1995). The Effect of Drawing on Memory Performance in Young Children. Developmental Psychology, 12. https://doi.org/10.1037/0012-1649.31.4.597 Creswell, J. W. (2015). Educational research: Planning, conducting, and evaluating quantitative and qualitative research (Fifth edition). Pearson. de Lautour, N. (2020). The Visual Arts and Children’s Thinking and Theorising in Early Childhood. Www.Elp.Co. Nz/Articles, 13. Driessnack, M., & Furukawa, R. (2012). Arts-based data collection techniques used in child research. Journal for Specialists in Pediatric Nursing, 17(1), 3–9. https://doi.org/10.1111/j.1744-6155.2011.00304.x Elliot W, E., & D. Day, M. (2004). Handbook of Research and Policy in Art Education (1st Edition). Routledge. Everts, H., & Withers, R. (2006). A Practitioner Survey of Interactive Drawing Therapy as Used in New Zealand. 16. Freedman, K. J., & Stuhr, P. L. (2004). Curriculum Change for the 21st Century: Visual Culture in Art Education. Funch, B. S. (1996). The aesthetic experience as a transcendent phenomenon. Nordisk Psykologi, 48(4), 266–278. https://doi.org/10.1080/00291463.1996.11863884 Gernhardt, A., Rübeling, H., & Keller, H. (2013). “This Is My Family”: Differences in Children’s Family Drawings Across Cultures. Journal of Cross-Cultural Psychology, 44(7), 1166–1183. https://doi.org/10.1177/0022022113478658 Hirsh-Pasek, K., Zosh, J. M., Golinkoff, R. M., Gray, J. H., Robb, M. B., & Kaufman, J. (2015). Putting Education in “Educational” Apps: Lessons from the Science of Learning. Psychological Science in the Public Interest, 16(1), 3–34. https://doi.org/10.1177/1529100615569721 Hwang, G.-J., Lai, C.-L., & Wang, S.-Y. (2015). Seamless flipped learning: A mobile technology-enhanced flipped classroom with effective learning strategies. Journal of Computers in Education, 2(4), 449–473. https://doi.org/10.1007/s40692-015-0043-0 Jolley, R. P. (2009). Children and Pictures: Drawing and Understanding. Wiley. https://books.google.co.id/books?id=QpGS9s9zqMoC Kellogg, R. (1973). Misunderstanding Children’s Art. Art Education, 26(6), 7–9. https://doi.org/10.1080/00043125.1973.11652137 Knight, L. (2008). Communication and Transformation through Collaboration: Rethinking Drawing Activities in Early Childhood. Contemporary Issues in Early Childhood, 9(4), 306–316. https://doi.org/10.2304/ciec.2008.9.4.306 Kress, G. R., van Leeuwen, T., & Van Leeuwen, D. H. S. S. T. (1996). Reading Images: The Grammar of Visual Design. Routledge. https://books.google.co.id/books?id=vh07i06q-9AC Kucirkova, N. (2017). IRPD—A framework for guiding design-based research for iPad apps. British Journal of Educational Technology, 48(2), 598–610. https://doi.org/10.1111/bjet.12389 Lowenfeld, V. (1949). Creative and Mental Growth. Macmillan. https://books.google.co.id/books?id=x7tRAQAAMAAJ Mamur, N. (2012). The Effect of Modern Visual Culture on Children’s Drawings. Procedia - Social and Behavioral Sciences, 47, 277–283. https://doi.org/10.1016/j.sbspro.2012.06.651 Moerdisuroso, I. (2017). Social Semiotics and Visual Grammar: A Contemporary Approach to Visual Text Research. International Journal of Creative and Arts Studies, 1(1), 80. https://doi.org/10.24821/ijcas.v1i1.1574 Nielsen, A. M. (2012). Forskeres arbejde med oplevelser af børns tegninger som forskningsmetode [The researcher’s work with children’s experiences of drawing as a research method]. Psyke & Logos. Papadakis, S., & Kalogianakis, M. (2020). A Research Synthesis of the Real Value of Self-Proclaimed Mobile Educational Applications for Young Children. In Mobile Learning Applications in Early Childhood Education (pp. 1–19). IGI Global. https://doi.org/10.4018/978-1-7998-1486-3.ch001 Quaglia, R., Longobardi, C., Iotti, N. O., & Prino, L. E. (2015). A new theory on children’s drawings: Analyzing the role of emotion and movement in graphical development. Infant Behavior and Development, 39, 81–91. https://doi.org/10.1016/j.infbeh.2015.02.009 Santrock, J. W. (2011). Educational Psychology. McGraw-Hill Medical Publishing. https://books.google.co.id/books?id=M8S4kgEACAAJ Vygotski, L. S. (2004). Imagination and Creativity in Childhood. Journal of Russian & East European Psychology, 42(1), 7–97. https://doi.org/10.1080/10610405.2004.11059210

Abstract Investigation into language seems to be never ending – not only in form and meaning but also in function and use in communication. In use, for example, recently, social semiotics as the development of semiotics has witnessed the move from the dominance of monomodality to multimodality. Newspapers and magazines, computers, and mass media in general have articulated multimodal texts (combinations of video, audio, written text, etc.) that aim to reinforce, render more complex, or produce different meanings. This means that the investigation is more than about language, but attends to the full range of communicational forms people use which covers image, gesture, gaze, posture and so on- and the relationships between them. This article presents a multimodal analysis on video advertisement entitled “https://www.google.com/?Funny, Commercial Coca-Cola, Producer, Director, Victor Pukkalsky- YouTube + 2014” accessed on January 13, 2016. This case study describes and interprets the video in order to give the concepts of description and interpretation in the perspective of multimodal semiotics as recommended by Anstey and Bull (2010). By applying the systems of multimodal recommended it was found that five aspects namely 1. Linguistic, 2. Visual, 3. Audio, 4. Gestural and 6 Spatial aspects of the advertisement in multimodal semiotics perspective are covered. Key Words: multimodal, discourse, analysis, advertisement, video, coca-cola

This paper aims to analyze the multimodal discourse of TED Talks in the healthcare domain. This analysis looks for opportunities for positive, negative, and neutral messages. The paper also offers ways to correct social inequalities and summarize best practices in health care. Studies examining verbal and nonverbal semiotics require a more fine-grained analysis since nonverbal parts play crucial roles in TED talks and cannot be neglected. The main features of the analysis are: to identify discourse patterns and message types, to overview the meanings of these patterns, and to propose ways to achieve social justice. With mixed qualitative and quantitative approaches, verbal and nonverbal elements are analyzed. They show a joint effort in meaning-making, which unravels how the speaker conveys ideas and affects the audience. The result shows the characteristics of different semiotic resources and ascertains how they generate meaning to reinforce the academic speaker’s message. It suggests that the adopted theoretical framework applies to the multimodal discourse analysis of public speaking. The speaker makes the speech more convincing and offers a comprehensive meaning delivered in public, academic speaking, establishing a rational and practical image.

AbstractThis paper argues that meaning-making with multimodal representations in science learning is always contextualized within a genre and, conversely, what constitutes an ongoing genre also depends on a multimodal coordination of speech, gesture, diagrams, symbols, and material objects. In social semiotics, a genre is a culturally evolved way of doing things with language (including non-verbal representations). Genre provides a useful lens to understand how a community’s cultural norms and practices shape the use of language in various human activities. Despite this understanding, researchers have seldom considered the role of scientific genres (e.g., experimental account, information report, explanation) to understand how students in science classrooms make meanings as they use and construct multimodal representations. This study is based on an enactment of a drawing-to-learn approach in a primary school classroom in Australia, with data generated from classroom videos and students’ artifacts. Using multimodal discourse analysis informed by social semiotics, we analyze how the semantic variations in students’ representations correspond to the recurring genres they were enacting. We found a general pattern in the use and creation of representations across different scientific genres that support the theory of a mutual contextualization between genre and representation construction.

This study attempts to carry out an analysis of the compositional and interpersonal metafunctions of the picture book Stella brings the family (2015). The analytical tools employed in this study are based on the model for reading visual narratives proposed by Painter, Martin and Unsworth (2013) and on Kress and van Leeuwen’s Visual Social Semiotics (2021), which are useful models for the analysis of multimodal texts. The compositional and interpersonal analysis intends to deepen the deconstruction of meaning in the picture book and how the characters express affection. The compositional analysis reveals that the visual and the written text are complementary and that the different focus groups highlight the characters. The types of themes and the layout foreground the protagonist Stella. In addition, from an interpersonal perspective, the analysis reveals that Stella has more physical contact with one of her fathers, which suggests that Stella has a closer relationship with him. The compositional and interpersonal meanings contribute to establishing a close connection with readers so that they identify with Stella’s problem of not having a mother to celebrate Mother’s Day.

AbstractTeaching and learning in science disciplines are dependent on multimodal communication. Earlier research implies that students may be challenged when trying to interpret and use different semiotic resources. There have been calls for extensive frameworks that enable analysis of multimodal texts in science education. In this study, we combine analytical tools deriving from social semiotics, including systemic functional linguistics (SFL), where the ideational, interpersonal, and textual metafunctions are central. In regard to other modes than writing—and to analyse how textual resources are combined—we build on aspects highlighted in research on multimodality. The aim of this study is to uncover how such a framework can provide researchers and teachers with insights into the ways in which various aspects of the content in multimodal texts are communicated through different semiotic resources. Furthermore, we aim to explore how different text resources interact and, finally, how the students, or authors of teaching resources, position themselves in relation to the subject. Data consist of one student text and one teaching resource text, both comprising drawn and written elements in combination with symbols. Our analyses of the student text suggest that the proposed framework can provide insights into students’ content knowledge and, hence, how construction of multimodal texts may be a useful tool for formative assessment. When it comes to teaching resources, the framework may be a useful tool for teachers when choosing resources, particularly in relation to students’ possibilities of meaning making when engaging with such texts, but also, as a basis for classroom discussions.