Siti Sarah Fitriani.
Nira Erdina.
Iskandar Abdul Samad.
Visual RepresentationsA
VISUAL REPRESENTATIONS AND COMPREHENSION: THE
EXPLORATION OF IMAGE-WORD RELATIONS AND TYPES OF
DESIGN
Siti Sarah Fitriani Universitas Syiah Kuala.
Indonesia ssfitriani@unsyiah.
Nira Erdiana Universitas Syiah Kuala.
Indonesia niraerdiana@yahoo.
Iskandar Abdul Samad Universitas Syiah Kuala.
Indonesia samad@unsyiah.
Abstract Visualisation has been used for decades as a strategy to help readers construct meaning from reading Teachers across the globe have introduced visualisation mostly to primary students with native language background.
They used the strategy to understand their own language.
Little is known how this strategy works for university students who learn foreign language.
Visualisation can be done internally .
y creating mental imager.
and externally .
y drawing visual representatio.
The product of visualising texts by using both models can be further investigated to find out if the meaning represented is appropriate to the meaning written in the text.
This study therefore aims at exploring meaning by analysing the visual representations drawn by 26 English Education Department students of Syiah Kuala University after they read a narrative text.
The exploration was conducted by looking at the image-word relations in the drawings.
To do so, we consulted Chan and Unsworth .
Chan .
and Unsworth and Chan .
on the image-language interaction in multimodal text.
The results of the analysis have found that the equivalence, additive and interdependent relations are mostly involved in their visual representations.
and these relations really help in representing meanings.
Meanwhile, the other three relations which are word-specific, picture specific and parallel are rarely used by the students.
In addition, most students created the representations in a form of a design which is relevant to represent a narrative text.
Further discussion of the relation between image-word relations, types of design and studentsAo comprehension is also presented in this paper.
Keywords: visual representation, image-word relations, types of design, reading for meaning.
INTRODUCTION Reading is an important English skill for Undergraduate students studying in Syiah Kuala University with English as a Foreign Language (EFL) background are expected to be proficient in reading different genres of English texts so that they can be part of the international community.
Learning to read English is important to the future of students who want to progress to higher levels of However.
English is a demanding International Journal of Language Education.
Vol.
3 No.
October 2019 pp.
language for EFL students, especially ones without access to additional support like tutorials, extra classes and opportunities to use this kind of English in their daily lives.
Those who are at the centre of this study need effective strategies for tackling English texts and they need to know more about the purpose, structure and language choices in texts of different genres and topics.
Being competent readers contributes to studentsAo ability to increase their English proficiency score.
This is certainly not an easy task, especially for the EFL students, because English is not their mother tongue.
English is not the language of everyday communication, and it is often used only during school hours.
In fact, on many occasions.
English is used only in English subject classrooms.
In this limited context, students read English texts mostly when they are assigned to do so in order to complete tasks.
To be good AocomprehensionAo in reading, students are required to be strategic in a process of constructing meaning.
Although it is only one aspect of development, the final reading scores attained by students is a proxy for good reading.
In this study, it was necessary to focus on this area simply because the attainment of a final score for reading influences the academic fate of these students.
Preparing students to meet this goal successfully is a key to successful academic Reading process is identic to obtaining meaning from a text (Palincsar, 2.
Without understanding the meaning, the reading process is not successfully done.
To assess whether students already understand the meaning of a certain text, they commonly use reading comprehension test, usually consisting of multiple choice questions like the one found in TOEFL, fill in the blank questions such as in IELTS, or open-ended questions that can be found in other reading comprehension tests.
These types of questions are also used by school teachers to assess studentsAo reading Other than these types, teachers can also assess whether or not students understand the meaning of the text they are reading by asking them to visualise reading texts and by deeply exploring the image-word relations in the representations.
This way is in fact important to try.
but is unfortunately beyond teachersAo attention.
Reading needs complex cognitive capacity (Randi.
Grigorenko, & Sternberg, 2005.
Talebi & Seifallahpur, 2.
which leads to the need of metacognition.
Visualisation is one of metacognitive strategies that can help students in reading (Israel.
Bauserman, & Block, 2.
Since few decades ago, visualisation has been used by scholars to help readers comprehend reading passages .
Anderson & Kulhavy.
Levin.
Davidson.
Wolff & Citron, 1973.
Pressley, 1.
Current scholars have also introduced visualisation to promote reading comprehension .
Park, 2012.
Wolley, 2010, 2007, 2006.
Hobbs, 2.
Most scholars introduce visualisation to school students.
There is still little attention given for the use of this strategy at university level.
Visualisation is divided into two types:
internal and external (Cohen & Hegarty, 2007.
Gilbert.
Nakhleh & Reiner, 2.
The internal visualisation is famously known as mental imagery that readers create in their mind as a part of meaning construction process (Trafton.
Trickett, & Mintz, 2.
The images that they had in their mind is invisible.
put simply, we are not able to see them.
When they do internal visualisation, we can examine their understanding of the reading passage being read by asking them to do think aloud, so they can share what they have already understood to But still, the mental imagery that they create remains invisible.
To make the internal visualisation visible, readers can draw a visual representation to represent their mental imagery as well as representing the meaning of a text.
This is what we call as external visualisation (Cohen & Hegarty, 2.
The external visualisation is seen as a multimodal text that involves semiotic resources including Siti Sarah Fitriani.
Nira Erdina.
Iskandar Abdul Samad.
Visual RepresentationsA for example visual and linguistic resources like images and words.
Currently, there is still little attention given by scholars to explore the meaning of a text represented by EFL university studentsAo through the analysis of visual representation by looking at its semiotic resources and how these resources are related.
We have the assumption that the types of image-word relation explain the meaning of a text that they visualise which also show studentsAo understanding to the text.
To find out this, the question that we pose is AuWhat are the types of relation between the images and words and types of design involved in studentsAo visual representations? Do these resources reflect studentsAo comprehension?Ay.
verb related to the text need to also be included in the representations.
Studies on Visualisation The visualisation strategy has been used in various fields.
In medical science, visualisation has been used since the nineteenth century as a therapy .
Mason, 2.
It is also used in the area of science, where scientists explicitly draw their mental images to assist with further exploration of current activities by Faraday.
Newton, and Maxwell .
Trafton.
Trickett, & Mintz, 2.
Also in language learning, visualisation has recently been used in the literacy area, especially in reading .
Hobbs.
Narkon & Wells, 2013.
Onofrey & Theurer, 2007.
Park, 2012.
Rader, 2009.
Woolley, 2007.
Woolley, 2.
to mainly support learners in comprehension.
Research on visualisation in literacy has been undertaken for a few decades, but the result of this research is still controversial among the researchers in the literature.
Some researchers argue that readers or students with comprehension difficulty are usually unable to generate visual images when they are reading a text (Hibbing & Rankin-Erickson, 2.
, while proficient readers automatically visualise what they are reading (Onofrey & Theurer, 2.
Comprehending a text by using visualisation that focuses on mental imagery has also been reported by educational language researchers .
Anderson & Hidde, 1971.
Anderson & Kulhavy, 1972.
Levin, 1973.
Levin.
Davidson.
Wolff, & Citron, 1973.
Pressley, 1.
Other researchers, however, have found that visualisation can disturb readers while they are interpreting meaning in a text.
for example.
Brooks .
Knauff and Johnson-Laird .
, and Mast and Kosslyn .
While most previous studies focused on the relation between visualisation .
s a process of the activation of cognitive capacit.
and reading comprehension, little attention is given to see how the resources in a visual representation, for example image-word Rhetorical Pattern of a Reading Text Each text type has its own purpose, language features and also generic structure.
These components are combined to convey meanings to readers.
For example, a procedural text has a purpose to tell readers how to do/make something.
This text type mainly use imperative mode to direct readers to the steps they have to do.
Readers are usually introduced to the goal of the procedure in the first stage of the text and then given the sequential steps in the second stage.
Different to this type, an explanation text aims at informing readers the process of the formation of a natural a declarative mode is used to explain the processes.
General statement and sequential explanation are the two stages used to develop the explanation text.
When students do the external visualisation, the rhetorical pattern of the visual representations needs to be related to the generic structure of the text being If students draw a representation of a procedural text, then the patterns are suggested to follow the stages of this text type so that the purpose of the text to deliver meaning can be achieved.
In addition, some language features of the text such as the use of technical vocabulary and appropriate mode of International Journal of Language Education.
Vol.
3 No.
October 2019 pp.
relations and types of design, can be related to reading comprehension.
present meaning alone.
in other words, meaning needs to be presented together.
For the analysis of image-word relations in studentsAo visual representations, we adapted the literature by Unsworth and Chan .
Chan .
, and Chan and Unsworth .
The analysis of image-word relations in viusal representations is important to understand the meanings represented by them.
The meanings that we understand from their drawing will also tell us the studentsAo comprehension of a text they read.
The analytical framework for the analysis is provided in this paper under the research method section.
Image-word Relations in a Visual Representation The way to interpret meaning in multimodal texts such as in images and picture books has been proposed by Unsworth .
by looking at the connection between the images and words in the texts.
Unsworth and Chan .
and Chan and Unsworth .
suggest two fundamental types of image-word Elaboration occurs when an image describes the accompanying word, or a word describes the accompanying image.
This type has two sub-types, which are called equivalence .
hen the image and word represent a similar meanin.
and exposition .
hen the image and word re-express or re-formulate meanin.
The second type of image-word relation, extension, is also divided into two types: additive .
hen the image or the word adds meaning to the othe.
and distribution .
hen the image and word are pairing in developing meanin.
Chan .
discusses seven types of image-word relation that are identified by McCloud .
Word specific is the first This type can be found when words mostly manage a multimodal text.
and the image existing in the text does not provide meaning to the language.
The second type is picture specific where images mostly dominate a multimodal text.
In other words, this second type is in contrast with the first type.
The next one is duo-specific.
In this type, both images and words represent the same meaning.
The fourth one is called additive.
In this type, either images or language elaborate on the meaning of the other.
Another type is called parallel where each image or each word communicates each meaning without crossing at any point.
montage relationship can be found when language becomes important for the image.
the last relation, which is called interdependent, both image and words cannot The Design of Visual Representations Joyce and Gaudin .
suggest that there are two kinds of visual representations:
narrative and non-narrative.
Narrative images commonly include a story that illustrates a causal process in a mentioned time.
The participants in the narrative images can be people, animals or objects that are doing something .
, or they may have something done to them .
So, the important parts of the narrative images are the participants and the process.
We conclude that within a narrative design, a visual representation can be drawn as problematic event sequence which is presenting some images .
he participants in a tex.
that show the problematic sequential events involving the orientation, complication and resolution parts of a story as mentioned in a text.
This can be used to represent a narrative text.
Meanwhile, a visual representation can also be drawn as a character organisation by involving the images of the participants involved in a text as to present the events that happened to the This one is used as a representation to the other types of story genres: recount and news item.
In non-narrative representations, graphs, diagrams or charts are usually involved.
The purpose of the non-narrative images is to show the relationship between different objects, and Siti Sarah Fitriani.
Nira Erdina.
Iskandar Abdul Samad.
Visual RepresentationsA to present a process for the combined elements.
Non-narrative images involve classificatory processes about how to connect one object to the other and analytical processes about objects of a larger whole.
The participants can be a thing, people or an animal.
In relation to text genres, non-narrative representations can be used to represent factual genres such as the explanation and procedural texts because these genres involve some processes/steps to respectively show readers, for example, how a natural phenomenon occurs and how to do Types of visual representations mentioned by Joyce and Gaudin .
are important for the current study in order to see in what formation or design students drew their visual representations.
The shapes students drew were related to the genre of the academic For example, students with a good understanding of the explanation genre tended to draw the visual representation as a diagram of processes, or they drew a problematic event sequence for a narrative text.
participants was done purposively.
These participants are one group of students who undertook Interpretive Reading subject at the English Education Department of Syiah Kuala University.
They were chosen because they have passed two required reading subjects in the last two semesters which means that they have been introduced to reading strategies and are used to applying them in a reading process.
thus, they are also ready to use visualisation strategy to get meaning in English texts.
Before the drawing activity, the students were firstly given the narrative text and asked to read it The students were encouraged to create mental imagery in their mind in the process of constructing meaning to understand the information written in the text.
They were also asked afterwards to think-aloud and share their understanding of the text to other This was followed by doing the external visualisation which is the drawing of their visual representation of the text.
The analysis was done following the theory of image-word relation in multimodal texts by Unsworth and Chan .
Chan .
, and Chan and Unsworth .
, as explained previously in this paper.
The analytical framework can be seen in Table 1
METHOD
This qualitative study was conducted by collecting and analysing visual representations drawn by 26 student-participants after they read a narrative text entitled AuThe Three Feather StoryAy.
The technique of choosing the Picture Table 1.
The Analytical Framework on The Image-Word Relations Image-word relations Word specific Equivalence Additive Parallel Interdependent In addition to the analysis, we consulted Joyce and Gaudin .
on the types of design to explore the formation of visual representations produced by the students.
From the discussions that we have presented in the literature review section of this paper, we concluded the framework of analysis as follows:
Table 2.
The Analytical Framework on The Types of Design Types of Design Non-narrative Narrative International Journal of Language Education.
Vol.
3 No.
October 2019 pp.
FINDINGS AND DISCUSSION
Image-Word Relations and Types of Design in Visual Representations For the purpose of discussing the results, we present three visual representations drawn by Nani.
Juni and Rini .
ll pseudonym.
These three were chosen for the discussion because they include detailed combination of images and words to represent meaning that the students had read in the narrative text.
Figure 1 below is the visual representation drawn by Nani.
Figure 1.
NaniAos visual representation of the narrative text Nani drew images of the story characters in her visual representation.
These characters are labeled with their names, and the characters of the kingAos sons are accompanied by their characteristics.
Other images appearing in her visual representation include images of carpets, handkerchiefs, feathers and The information presented in her Siti Sarah Fitriani.
Nira Erdina.
Iskandar Abdul Samad.
Visual RepresentationsA drawing is divided into four parts, which are separated by using two long horizontal lines and one short vertical line.
These lines are considered to be text organising images, which organise the order of the story events.
The first part of the image presents the characters, the kingAos idea to choose the next king, and the first event that happened in the In this part.
Nani identified the problem of the story, which was to find the next king.
Picture Word specific She writes this problem clearly in a conversation balloon.
The second represents the choice of feathers .
east vs.
, with Simpleton choosing the middle one.
The third part presents the second event.
The last part is about the third event and the resolution after all the events had occurred.
Based on the analysis, the image-word relations involved in Figure 1 is concluded in the following table.
Image-word relations Equivalence Additive In NaniAos drawing, there are three imageword relationships found.
these are equivalence, additive and interdependent.
The equivalence relation is presented by both images and word that communicate similar For example, the image of the king is named with AokingAo.
Both the king image and its term convey a similar meaning.
The additive relationship was used by Nani to clarify or add information to the image that the language follows.
For example, the image of the third son is labeled with AosimpletonAo, which is then described with Aodid not speak muchAo to add information that Simpleton did not speak too much, which is the feature of this The interdependent relation is presented, for example, by the use of the speech balloon next to the picture of the king.
Parallel Interdependent which expresses Aothink and confuse who will be the next king?Ao This speech balloon cannot communicate the expression alone without the image of the king, because the one who said the statement was the king.
So both the image and the language should together convey the The relations appearing in NaniAos viusal representation has made the meaning understandable and appropriate to the meaning found in the text.
In terms of the type of design.
Nani drew her visualisation as a narrative drawing presenting problematic event sequence, which is relevant to the genre of the narrative text.
This can be seen from the images of the characters and other participants drawn to present problematic events of the story.
The design is concluded in the table below.
Types of Design Non-narrative Narrative Now we see the second one drawn by Juni.
International Journal of Language Education.
Vol.
3 No.
October 2019 pp.
Figure 2.
JuniAos visual representation of the narrative text JuniAos visualisation is the most detailed production for the narrative text of all the studentsAo visualisations.
This can be seen, for example, in the way she drew an image of a This king has a crown and is clothed.
The other examples include detailed images of the three sons, three feathers, a box, doors, toads, carpets, rings and women.
Each event is connected to the other events using arrows.
The use of arrows as text organising images Picture Word specific has helped in ordering the events More importantly, she introduced the problem of the story to find the next king at the beginning of her drawing inside a speech balloon.
The image-word relations found in JuniAos representation are presented in the table Image-word relations Equivalence Additive Parallel Interdependent Siti Sarah Fitriani.
Nira Erdina.
Iskandar Abdul Samad.
Visual RepresentationsA For example, the image of the king is followed by a speech balloon, within which she wrote Aowhich son should become the kingAo.
The language inside the speech balloon cannot communicate information alone without the image of the king to show that the words are used by the king to express his thinking.
Thus, this should indicate an iterdependent relation where neither the words nor the image can convey the information The involvement of these relations is important and appropriate to communicate the story events for better meaning making.
As can be seen, there are three types of image-word relations found in JuniAos drawing of the narrative text.
The first one is equivalence, or similar relationship.
This relationship was used by Juni to name the images she drew.
For example, the image of the feather is named with its term AofeatherAo, and the image of the kingAos third son is called AoSimpletonAo.
The second relation is additive, or clarifying.
For this one.
Juni used words to clarify or add more information to the image that the words follow.
For example, she wrote Aofrom shepherdAos wivesAo next to the image and its term Aocoarse handkerchiefAo to clarify that the coarse handkerchief was given by the shepherdAos wives.
The third relationship is The following table conclude the tyep of design of JuniAos representation of the narrative Types of Design Non-narrative Diagram of Event sequence processes/steps Narrative Problematic event sequence JuniAos external representation is drawn in the form of a narrative design showing the problematic event sequence of the narrative text, which is relevant to the genre of the narrative text.
It presents detailed images of the story characters along with language to develop the story.
Now we present the last visual representation by Rini.
Rini drew a two-page visualisation for the narrative text .
ee Figure 3 and Figure .
She drew a long visualisation because she tried to include all the events of the story in detail.
She Character began by drawing the characters of the story, including the king and his three sons.
The representation then shows the first event that happened in the story, followed by the second and third events.
The last part of her visualisation is the resolution of the story.
Each event in the drawing presents detailed information with the use of detailed images and language.
In addition.
Rini also used arrows as text organising images to organise and connect one event to another in her International Journal of Language Education.
Vol.
3 No.
October 2019 pp.
Figure 3.
RiniAos visual representation of the narrative text .
Siti Sarah Fitriani.
Nira Erdina.
Iskandar Abdul Samad.
Visual RepresentationsA Figure 4.
RiniAos visual representation of the narrative text .
The analysis of her visual representation based on the image-word relations is indicated in the table below.
Picture Word Image-word relations Equivalence Additive The relation between the images and the words appears to be equivalence and additive.
The equivalence relation can be seen from the image followed by word.
, which both communicate a similar meaning, for example, the image of the king that is accompanied by the language AokingAo.
The additive relationship is seen, for example, in the second event presented in the visualisation, which is demonstrated with the images of three sons and Parallel Interdependent three feathers accompanied by the language Aoflew three feathers into the air which they were to followAo.
The use of this language has clarified or added information about what happened to the three sons and the three The existence of these relationships has strengthened the visual representation for communicating the meaning in the story.
International Journal of Language Education.
Vol.
3 No.
October 2019 pp.
The type of design RiniAos drawing can be classified as a narrative one representing the problematic event sequence, which is indicated in the table below.
It is classified so because it presents the problematic story events by drawing the images of the story characters along with their accessories, such as the crown, ring and castle, which are organised following the stages of the text starting from the orientation to the resolution.
This formation is relevant to represent a narrative text.
prediction about the characteristics of a productive visual representation of the narrative text can be connected to studentsAo appropriate thinking of the text.
In other words, if students can appropriately think and understand the source text, there is a high probability that they can include all the productive characteristics in their visual representation, for example, the appropriate relations of image and word, as well as the relevant type of design.
This is also connected to studentsAo success in generating mental imagery, which they presented externally Successful visualisation is related to successful comprehension.
This successful productive characteristics being included in studentsAo visual representation.
Secondly, studentsAo familiarity with the narrative genre, including the purpose and stages, is suggested to help in understanding the development of the events in the story.
Students are also familiar with the topic This also relates to studentsAo background knowledge of other similar fiction stories that helped them to connect to the story in the narrative text.
In general, most students indicated good comprehension of the narrative text through their visual representation, as can be seen from the involvement of relevant image-word reations and type of design.
The relation between image-word relations, types of design and studentsAo reading The other 23 visual representations drawn by the students mostly involve combination of images and words in representing meaning.
found that it is more interesting and meaningful if a visual representation combines images and words where the relations of these resources are helpful in presenting meaning i.
equivalence, additive and interdependent We also reveal from the analysis that the relations between images and words in visual representations can be explored to understand the meaning being represented.
This is in line with Unsworth and Chan .
Chan .
and Chan and Unsworth .
that we can interpret meaning from images and words by looking at their relations.
Moreover, the meaning that the students represent in their visual representations also show their understanding of the text being visualised.
Therefore, literacy teachers can explore studentsAo understanding of a text by lookng at their visual representations.
To summarise, most students have successfully represented complete and chronological events of the narrative story using different features.
Most of their drawings are considered productive, and two of them (Juni and Nan.
were able to draw powerful and productive visual representations of the narrative text.
Some other points are also concluded from the analysis.
Firstly, the CONCLUSION Among the relations discussed in the interdependent relations are mostly used in the visual representations.
These relations help in communicating meaning based on the one found in the text.
The other three relations which are word specific, picture specific and parallel do not appear in any of studentsAo visual representations, which is based on the result of the analysis.
We also found that students can better communicate meaning in their visual representations if they include the use of images and words that have equivalence.
Siti Sarah Fitriani.
Nira Erdina.
Iskandar Abdul Samad.
Visual RepresentationsA additive and interdependent relationships.
also assume here that the other three relations, which are word specific, picture specific and parallel Aeif they are used in visual representations- need to be accompanied by the equivalence, additive and/or interdependent relationships to represent the appropriate Therefore, we suggest other researchers to investigate this assumption.
Moreover, the type of design of studentsAo visual representation has a relation to their understanding of the genre knowledge of the text they read.
the pertinent use of a design also reflects to their understanding of the text.
Gilbert.
Nakhleh.
, & Reiner.
Visualization: Theory and practice in science education.
Models and Modeling in Science Education .
Hibbing.
, & Rankin-Erickson.
A picture is worth a thousand words: Using visual images to improve comprehension for middle school struggling readers.
The Reading Teacher.
Hobbs.
Improving reading comprehension by using media literacy Voices from the Middle, 8.
Israel.
Bauserman.
, & Block.
Metacognitive Thinking Classroom, 6.
, 2128.
Joyce.
, & Gaudin.
Interpreting the Visual: A Resource Book for Teachers.
Phoenix Education: Putney.
Australia.
Knauff.
, & Johnson-Laird.
Visual Memory & Cognition, 30.
Levin.
Inducing comprehension in poor readers: A test of a recent Journal Educational Psychology, 65.
, 19.
Levin.
Davidson.
Wolff.
, &
Citron.
A comparison of induced imagery and sentence strategies paired-associate Journal Educational Psychology, 64.
, 306.
Mast.
, & Kosslyn.
Visual mental images can be ambiguous:
Insights from individual differences in Cognition, 86.
, 57-70.
McCloud.
Understanding Comics:
The Invisible Art.
Northampton.
Mass.
Narkon.
, & Wells.
Improving reading comprehension for elementary students with learning ACKNOWLEDGEMENTS The authors would like to thank Mary Macken-Horarik for her support in completing this research.
REFERENCES