SciELO - Scientific Electronic Library Online

 
vol.48 número89Hacia una normalización de la frecuencia de los corpus CREA y CORDEReconocimiento auditivo de palabras, léxico pasivo y comprensión de textos descriptivos orales en preescolares índice de autoresíndice de materiabúsqueda de artículos
Home Pagelista alfabética de revistas  

Servicios Personalizados

Revista

Articulo

Indicadores

Links relacionados

  • En proceso de indezaciónCitado por Google
  • No hay articulos similaresSimilares en SciELO
  • En proceso de indezaciónSimilares en Google

Compartir


Revista signos

versión On-line ISSN 0718-0934

Rev. signos vol.48 no.89 Valparaíso dic. 2015

http://dx.doi.org/10.4067/S0718-09342015000300003 

Revista signos. Estudios de Lingüística
ISSN 0718-0934
© 2015 PUCV, Chile
48(89) 332-354

ARTíCULOS

The impact of a linguistic intervention on rhetorical inferential comprehension and metacognition in EFL academic reading: A quasi-experimental, mixed-methods study*

Impacto de una intervención lingüística en la comprensión inferencial retórica y la metacognición durante la lectura académica en inglés lengua extranjera: Un estudio cuasi-experimental con métodos mixtos

Moisés Damián Perales Escudero
Universidad de Quintana Roo, México
mdperales@uqroo.edu.mx

María del Rosario Reyes Cruz
Universidad de Quintana Roo, México
rosreyes@uqroo.edu.mx

Edith Hernández Méndez
Universidad de Quintana Roo, México
edith@uqroo.edu.mx


Abstract: The connections between metacognition and metalinguistic awareness are under- researched and poorly understood in the field of foreign language reading. This paper uses a linguistic intervention to investigate the connections between rhetorical inference making, linguistic awareness inspired in Systemic-Functional Linguistics (SFL), and forms of metacognition. An intervention inspired in SFL was designed and implemented using a quasi-experimental design. The target genres were the introduction of the empirical research article and the discussion article. The control group received traditional instruction focusing on background knowledge and vocabulary. Answers to a pre-/post-test were analyzed quantitatively; answers to a written, self-report questionnaire were analyzed qualitatively. The quantitative results show gains in inferential comprehension for both groups, but a greater effect size for the experimental group. The qualitative results show gains along several meta- comprehension dimensions for the experimental group. Students’ self-reports suggest that metalinguistic awareness promotes increased monitoring, raised standards of coherence, and willingness to engage in effortful processing.

Key Words: Reading comprehension, language awareness, metacognition, Systemic- Functional Linguistics, English as a Foreign Language.


Resumen: La investigación sobre las relaciones entre la metacognición y la conciencia metalingüística en el campo de las lenguas extranjeras es limitada. Este trabajo reporta una intervención lingüística que se usó para investigar las conexiones entre las inferencias retóricas, la metacognición y la conciencia metalingüística inspirada por la Lingüística Sistémico- Funcional (SFL). Se siguió un diseño cuasi-experimental. Los géneros meta fueron la introducción del artículo de investigación empírica y el artículo de discusión. El grupo de control recibió instrucción tradicional centrada en el conocimiento previo y el vocabulario. Las respuesta al pre-test/post-test fueron analizadas cuantitativamente; las respuestas a un cuestionario fueron analizadas cualitativamente. Los resultados cuantitativos muestran mejoras en la comprensión inferencial de ambos grupos, con un tamaño del efecto mayor para el grupo experimental. Los resultados cualitativos muestran mejoras en varias dimensiones de la metacomprensión para el grupo experimental. Los resultados indican que la conciencia metalingüística promueve un mayor monitoreo, estándares de coherencia aumentados, y deseo de hacer esfuerzos para procesar el texto.

Palabras Clave: Comprensión lectora, conciencia lingüística, metacognición, Lingüística Sistémico-Funcional.


INTRODUCTION

Reading comprehension in general, and L2 reading comprehension in particular, are complex processes that involve multiple sub-processes acting in different configurations according to the type of reading task. Two key components of reading comprehension are inference-making and comprehension monitoring, an aspect of metacognition. Inference-making plays a pivotal role in this process as no text, or local segment within it, can contain all the information that would be necessary to comprehend it. The ability to make inferences is particularly useful in domains and tasks for which readers have little background knowledge, such as cases when undergraduate-level, foreign language (FL) readers read for academic purposes. Comprehension monitoring has only recently started to be investigated by L2 scholars, but research results suggest that this ability distinguishes less proficient from more proficient FL readers (Kolić-Vehovec & Bajšanski, 2007; Han, 2012).

Within the general acknowledgment of the role played by inferences, a less investigated issue in both L1 and L2 reading has to do with rhetorical inferences, or inferences pertaining to unstated or implied authorial motivations, intentions, arguments target audiences and persuasive strategies (Perales Escudero, 2011, 2013). As Perales Escudero (2011, 2013) has suggested, the product of rhetorical inferences is a pragmatic kind of meaning that emerges from the interaction between specific forms of metalinguistic activity and metalinguistic knowledge of a rhetorical nature. However, this connection between inferences and metalanguage/metalinguistic activity has received little scholarly attention, particularly in EFL academic reading (Han & D’Angelo, 2007). Further, as claimed by Nagy (2007) and Brantmeier and Dragiyiski (2009) the connections between metalinguistic activity and comprehension monitoring in reading comprehension are under-researched in both L1 and L2 comprehension scholarship.

Investigations addressing the intersection between metalinguistic knowledge, comprehension monitoring and rhetorical inferential comprehension have the potential to shed light on our understanding of the reading process and produce useful pedagogical insights. This is particularly true in light of findings suggesting that metacognitive monitoring makes a significant contribution to L2 reading (Kolic-Vejovec & Bajsanski, 2007; McKeown & Gentilucci, 2007), and that better comprehenders are also better at learning new vocabulary (Pulido, 2009).

The research project reported in this paper locates itself at that intersection. Specifically, it investigated the effectiveness of an intervention inspired in Systemic- Functional Linguistics (SFL) and rhetoric on the ability of advanced, adult EFL readers to make rhetorical inferences from disciplinary texts. At the same time, it addressed the following questions pertaining to the metalinguistic activity-metacognition connection.

1. What metalinguistic terms are taken up by the learners in anonymous written answers to a questionnaire?

2. What kinds of metalinguistic and metacognitive activity do the learners show awareness of in anonymous written answers to a questionnaire?

Addressing these questions from a SFL paradigm also helps to specify the ways that SFL contributes to reading comprehension, which is an ongoing research program in SFL’s pedagogical branch, the Sydney School (Rose & Martin, 2012; Schleppegrell, 2013). Scholars in this tradition have so far not addressed this connection between SFL metalanguage and metacognition. From the answers to these questions and the testing of the intervention’s effectiveness, we attempt to theorize the connections across rhetorical inferences, metalinguistic activity and metacognition, with a particular focus on metacognitive monitoring. Our results suggest that metalinguistic activity and metalanguage assist in building kinds of procedural and declarative knowledge that enable the learners to set up higher standards of coherence (Van den Broek, Virtue, Everson, Tzeng & Sung, 2002, explained below) and engage in effortful processing. We discuss pedagogical implications that may help the field break up with current-traditional practices that focus on pre-reading and post-reading, but have limited impact on what students actually do while reading (Gibbons, 2002; Hazelrigg, 2009).

1. Theoretical framework

A basic distinction between inference types divides inferences between those that require retrieving information from other textual segments than the immediate one being read, or bridging inferences, and those requiring the addition of background knowledge, or elaborative inferences (Grabe, 2009). Kintsch (1998) proposed a different, more cognitive distinction based on whether the information required to repair a gap is retrieved from memory or requires a reasoning process because it is not previously known. Inferences of the first type, or retrieval inferences, include both retrieval from other textual segments or from background knowledge (long term memory). Inferences of the second type, or generative inferences, do not involve just retrieval but processes of reasoning because the information needed is not present in the text, short term memory, or long term memory.

Perales Escudero (2011) has proposed that rhetorical inferences are a kind of generative inference as long as the rhetorical dimension of a text is not encoded explicitly in the propositions that make up the text, i.e. the text won’t say explicitly what the author’s motives, intentions or targeted audiences are. In texts using evaluative language, inferring that kind of information involves semantic-functional parsing of that kind of language (i.e. determining who evaluates what or whom in what way and in what contexts) and then considering the textualized interpersonal relations in light of one’s knowledge about the social, communicative functions of genres and interpersonal parameters such as authority, social distance, power, and so on. Perales Escudero (2011, 2013) Perales Escudero y Reyes Cruz (2014) uses the term ‘rhetorical genre knowledge’ to label this kind of pragmatic metalinguistic knowledge.

Comprehension monitoring has been defined as:

“a metacognitive process… essential for competent reading, which directs the reader’s cognitive processes as he/she strives to make sense of incoming textual information” (Wagoner, 1983: 328).

According to Han (2012), it involves three related processes: evaluation (assessing one’s understanding), planning (selecting relevant strategies to address a difficulty), and regulation (implementing strategies). In the well-known three-part division between kinds of metacognitive knowledge-declarative, conditional, and procedural- these activities would fall under procedural knowledge. The first two kinds of knowledge were proposed by Flavell (1979, 1985) and the third by Paris, Lipton and Wixon (1983). According to Peronard (2002), the basic distinction between declarative and procedural knowledge when it comes to reading strategies is that the former involves knowing that they exist and the latter involves knowing how to enact them. Following Peronard (2002) and Baker and Brown (1984), we posit that any verbalization of procedures, including monitoring ones, is metacognitive knowledge of the procedural kind as it shows evidence that the reader knows how to enact the procedures; it is verbalization which indicates conscious awareness of cognition.

Drawing on Peronard and Velásquez (2003), we propose that the three kinds of knowledge are underpinned by knowledge of the structural, semantic-pragmatic, and functional features of written texts, or textual knowledge. Metalanguage is a component of this knowledge, and it can consist of non-technical words such as ‘text’, ‘paragraph’, or ‘main idea’, or more technical words such as ‘rhetorical moves’, a term taken from Swales (1990). Readers vary in the quality and extent of textual and metalinguistic knowledge, partly as a function of the kinds of linguistic theories that have informed their formal literacy education. This latter aspect also bears on the kinds of metalinguistic activity that readers commonly engage in.

To explicate the connection between metacognition and textual knowledge, Nagy (2007) has put forward the Metalinguistic Hypothesis (MH), which proposes that a) metalinguistic awareness is a subset of metacognition, and b) metalinguistic awareness makes direct contributions to the metacognitive aspects of comprehension and, specifically, to monitoring. Extending Nagy’s MH, we suggest that metalinguistic awareness has an impact on readers’ standards of coherence for specific reading tasks.

Van den Broek et al. (2002) use the term ‘standards of coherence’ to designate a kind of metacognitive knowledge/activity involved in reading comprehension. ‘Standards of coherence’ refers to a readers’ idea of what good comprehension is (a kind of declarative knowledge) and what it necessitates (a kind of procedural and perhaps also conditional knowledge) for a specific reading task. For example, a reader holding a low standard of coherence for a specific reading task may think that good comprehension involves understanding a few keywords in a document’s title.

Standards of coherence associate with the amount of cognitive effort a reader is willing to expend on textual processing and on monitoring their own comprehension. A low standard of coherence would signal low chances of engaging in monitoring because, for the reader, comprehension is simple and there are little chances s/he might make mistakes. A high standard of coherence could predict higher chances of engaging in planning, monitoring, and self-regulation because the reader is willing to engage more cognitive resources and consider the possibility of needing to repair his/her comprehension. A higher standard also predicts greater chances for critical evaluation of one’s own reading. We suggest that metalinguistic awareness enhances declarative and procedural knowledge, thus building cognitive resources that the reader can then apply to comprehension monitoring. Because of greater knowledge, the reader raises her standards of coherence and also her willingness and ability to engage in effortful processing and evaluation thereof in order to achieve those standards.

A final element of our theoretical framework concerns the distinction between metalanguage and metalinguistic activity. We adhere to Camps, Guasch, Millán and Ribas’ (2004: 104) definition of the latter as:

“discursive activity on language… both as developing in an interactive situation (interpsychological) and also as an internal activity (intrapsychological) in the form of verbal thought.”

Explicit metalinguistic activity in social interaction can coincide with the use of a specific metalanguage coming from the language sciences, or technical metalanguage. Alternatively, it can involve the use of everyday, non-technical metalanguage (e.g. ‘story’, ‘paragraph’, ‘word’). It may also involve circumlocutions to describe language or discourse phenomena for which everyday language has not developed a term.

This distinction is necessary as we do not wish to suggest that metalanguage is a necessary condition to produce the kinds of heightened metacognitive monitoring and standards of coherence above. What we see as a necessary condition is metalinguistic activity. Nonetheless, metalinguistic activity can be aided by technical metalanguage, and the kinds of technical metalanguage infused in an intervention are likely to have an influence in the kinds of metalinguistic activity and awareness that learners and teachers develop.

2. Methodological framework

Our study is informed by a non-dichotomous view of research paradigms. We agree with Maxwell (2010) that a combination of both ways of thinking, the quantitative and qualitative, is helpful in trying to account for a phenomenon in richer, more complex ways. To study inferential comprehension, we used a quasi-experimental design with the intervention being the independent variable and the results of a rhetorical inferences and main idea comprehension test being the dependent one. The experimental group (N=14) and the control group (N=12) consisted of students in the eighth semester of an undergraduate degree in English Language Teaching at a Mexican public university. All the participants were NS of Mexican Spanish. Per the curriculum of this degree, students enrolled in the eighth semester should have achieved the B2 level of the Common European Framework for Languages. Thus, their English proficiency was assumed to match that level and was not tested.

The treatment consisted in the exposure to the instructional design presented below during 12, two-hour sessions for a total of 24 hours, with one weekly session. The control group received traditional instruction, which scholars have characterized as consisting of activating previous knowledge, presenting vocabulary before reading, and asking post-reading questions to evaluate comprehension (Grabe, 2009; Hazelrigg, 2009; Han & D’Angelo, 2009). Both syllabi were implemented by one of the authors of this paper. The classes were videorecorded, watched, and commented on by a colleague in order to ensure fidelity to the instructional designs. This colleague earned a MA in Applied Linguistics by writing and defending a thesis on argumentative text analysis. He also holds a PhD in Linguistic Anthropology and teaches academic reading and writing at the tertiary level. In addition, he participated in the pedagogical design of our intervention, although he is not an author of this paper. We think his credentials and participation in the pedagogical design qualified him to be an informed and reliable observer.

The same test was used to measure inferential and main idea comprehension before and after the treatment. It was designed by us and was based on reading the introduction to Bateman (2008) and the full text of Belcher (2007). Both texts require background knowledge about EFL teaching methods and L2 acquisition. We assumed that students had this knowledge from their previous courses, which included Methods and SLA. Neither Bateman (2008) nor Belcher (2007) is excessively technical and their topics are not related to the papers read during the intervention. The test can be seen in Appendix 1. The results were analyzed using t-tests. The null hypothesis was that the treatment would not produce significant differences or effects across the results of the experimental and control groups.

Unlike other studies, we did not use think-aloud protocols to study cognitive monitoring. Instead, we analyzed qualitatively verbal data produced in response to a written questionnaire that asked the following questions: 1) would you recommend this class to a friend? Please discuss why or why not; 2) what are your general impressions on this class?; 3) do activities in this class relate in any way to your out- of-class reading? If so, please explain how. The students were told to answer the questions in the language in which they felt most comfortable; all but one chose Spanish. Our reasons for using a questionnaire are as follows.

First, we focus on cognitive monitoring as procedural knowledge, not on regulation as an online process. Under this conceptualization, what is verbalizable is what is accessible to consciousness. What the students are able to verbalize in a delayed, post-intervention condition is likely to be what has caused a deeper, long- lasting impression in their consciousness and is accessible for recall and analysis in long term-memory, not only in short-term memory as is the case with think-aloud protocols. In other words, if students’ verbalizations in a post-intervention, open- ended questionnaire that does not focus explicitly on metacognition include statements about comprehension monitoring, those statements are evidence that comprehension monitoring activities have entered their conscious awareness.

Second, a previous study of a similar intervention that focused on journalistic opinion articles (Perales Escudero, 2011) indicated that an open-ended, qualitatively analyzed questionnaire asking for self-reports of learning yielded rich evidence of metacognitive gains. Although we could have developed a quantitative instrument based on that questionnaire, we did not want to do so because this study focuses on a different genre, and we did not know if the analytic categories emerging from the previous study would apply. That is, we feared that constructing a questionnaire based on a different intervention with a different genre would leave out important insights, as it has been shown that genre deeply affects the kinds of comprehension processes engaged by readers (Parodi, 2011). For similar reasons, we did not want to use existing metacognitive strategies inventories, such as the MARSI (Mokhtari & Reichard, 2002) as our set of strategies focuses specifically on rhetorical, inferential skills and two specific genres.

We analyzed students’ answers to this questionnaire using directed qualitative coding (Hsieh & Shannon, 2005). That is, we looked for evidence of occurrence of the phenomena of interest, namely forms of metalanguage, metalinguistic activity, and metacognition. We then developed theory-based codes that we used to systematically code the students’ written statements.

3. The instructional design

3.1. The syllabus

The curriculum that was designed for the experimental group was based on the teaching of metatextual knowledge and metacognitive monitoring. The target genres were the introduction to the empirical research paper and the discussion article. The texts that were read as part of this curriculum are shown in in Table 1.

Table 1. The texts in the experimental syllabus.

Empirical research papers

Discussion papers

Kupferberg & Olshtain (1996)

Larsen-Freeman (2000)

Bardovi-Harlig (1999)

Belcher (2007)

Collins (2002)

Kumaravadivelu (2006)

Besides the fact that they incorporate complex intertextual relations within their macro-structure, these texts are also intertextually related to one another because they deal with controversial issues in applied linguistics. In some cases, they even take opposite stances on those issues. The students are familiar with the topics thanks to the courses they’ve taken in previous terms, such as EFL Methods and Second Language Acquisition. Nevertheless, we verified through informal interviews that the specific papers in Table 1 had not been included in the syllabi of previous courses. As a matter fact, it was found through these interviews that those courses did not incorporate research papers in their syllabi. The only reading materials that the instructors reported using were textbook chapters. The participants in this study confirmed that information.

The strategies that were used can be understood as comprising three parallel axes. First, there was a meta-strategic axis at which the emphasis was on self-regulation, comprehension monitoring and effortful processing. Second, there was a strategic axis with two components: main idea comprehension and rhetorical inferences. Third, there was a micro-structural strategy axis where the strategies were focused on language-processing. These should not be understood as discrete and separate, but as integrated and connected to one another via multiple processes of emergence. Table 2 outlines the specific strategies followed for each level. The relevant strategies are explained and exemplified in the paragraphs below. Due to space limitations, grammar-focused and self-explanatory strategies are not explained.

Table 2. Strategic axes of the intervention.

Meta-strategic axis

Macro-structural axis

Micro-structural axis

Treating one’s local rep- resentations as emerging, hypothesis.

Reconstructing one’s syntactic and semantic processes.

Locating and comprehending main ideas

Processing complex nominal groups

Unpacking nominalizations (who does what to whom with what in what context)

Predicting content from titles

 

Making rhetorical inferences: motivation, intention, arguments, target audience

Identifying rhetorical moves and relating them to motivation, intention and arguments.

Identifying and parsing evaluative language (who evaluates whom in what way)

Identifying and parsing patterns of Engagement (attributions, counters, denials) and connecting them to arguments, intentions, and target audience

3.2. Treating one’s local interpretations as emerging hypotheses

This meta-strategy is a component of comprehension monitoring and involves a shift from assuming that one’s understanding of a local segment is right, to treating such understanding as an emergent mental representation whose validity is contingent upon further verification aided by either deeper reading of the same segment, or confirmation from other, adjacent or non-adjacent segments. An example of the use of this strategy comes from reading the intro to Larsen-Freeman (2000: 2), reproduced below.

“The topic of this paper is language teaching methodology -and the matter of its appropriateness. A number of writers in our field have criticized language teaching methods during the last decade. Some have noted that the search for the best method is ill-advised (see, for example, Bartolome, 1994). They say that there can be no one best method. Others have maintained that teachers do not think in terms of methods when planning their lessons (for example, Long, 1991). Teachers are much more likely to think in terms of activities. Still others assert that methodological labels tell us little about what really occurs in classrooms (for example, Katz, 1996)… If these criticisms were not serious enough, there has been an additional concern registered during this past decade, which has attracted a great deal of attention”.

In this paragraph, the author presents a variety of concerns and critiques focusing on language teaching methods. None of those are proffered or endorsed by the authorial voice. On the basis of the evidence, two possible hypotheses can be formulated regarding the author’s own position toward the appropriateness of methods: she shares the concerns and critiques (hence the extensive quoting of critical voices), or she does not and is quoting those critical voices in order to argue against their stand. Students were engaged in formulating these hypotheses dialogically, and were explicitly told that they should hold these cognitions as hypotheses to be confirmed rather than factual information.

3.3. Reconstructing one’s syntactic and semantic processing

This is another component of comprehension monitoring. It involved a dialogic process between the students and the teacher that occurred whenever an incongruent interpretation on the part of student, usually one involving non-target like input parsing, was identified by the teacher in the course of text-focused conversations.

3.4. Comprehending main ideas

For this strategy, ‘main idea’ was operationalized as the paper’s main contribution to knowledge, in the case of the empirical research articles, or the author’s position on the issue being discussed, in the case of the discussion papers. In both cases, the identification of main ideas was aided by an analysis of a paper’s rhetorical move structure. For the empirical research papers, we used the well-known CaRS model (Swales, 1990). This empirically-based model describes a three-stage move structure for empirical research paper introductions, which we present here in a simplified fashion. Move 1, ‘establishing the territory’, states the overall topic and its relevance. Move 2, ‘creating the gap’, carves a niche for the study by discussing limitations of previous research and/or situating the study within a tradition. Move 3, ‘filling the gap’, shows how the study fills the niche created in move 2 by, for example, summarizing results and implications. In other words, an empirical research paper’s contribution can be found in move 3, ‘filling the gap’. For the discussion papers, the authorial position is a move in itself and usually comes after the opposing or contrasting positions have been reviewed.

3.5. Making rhetorical inferences

Rhetorical inferences are those inferences that pertain to authorial motivations, the audiences authors textualize for their texts, and authors’ persuasive intentions. Academic authors are usually motivated by a desire to change the state of a field’s knowledge on a specific topic, theme, question, or issue. We used the construct of rhetorical exigence (Bitzer, 1968) to signal that authorial motivations are structured by perceptions of situations that are not as they should be and can be modified by verbal action. Academic authors’ intentions are structured by epistemic exigencies pertaining to the state of knowledge in a field; they seek to align readers with their views and persuade of the value of their contributions and/or positions (Hyland, 2005). Rhetorical inferences can be made on the basis of parsing evaluative language and interpreting the resulting patterns of interpersonal positioning in light of a set of discursive conventions (Perales Escudero, 2011, 2013). For example, the inference regarding Larsen-Freeman’s (2000) position on methods requires parsing the patterns of evaluation (who evaluates what/whom in what way) and then considering that, in position papers, it is a common practice to first represent the viewpoint that the author will ultimately oppose or question. Inferences about authorial motivations can also be made by identifying moves 1 and 2 in the CaRS model; they explain what the state of knowledge is that the author wishes to alter.

3.6. Identifying rhetorical moves and connecting them to motivations, intentions and arguments

The CaRS model (Swales, 1990) was used to teach the rhetorical move structure of research article introductions. While not originating in SFL, move analysis has been integrated within the SFL framework of ideational an interpersonal meanings in several studies (Lewin, Fine & Young, 2001; Englander, 2006). To make the metalanguage more transparent, we changed the names of the CaRS moves to Theme, Debate and Contribution. The generic stages of discussions proposed by Martin and Rose (2008) were used to teach the rhetorical structure of discussion papers.

3.7. Identifying and parsing evaluative language

In English, evaluative language plays a critical role in the textualization of authors’ positions vis-à-vis disciplinary issues and other authors in a field (Hyland, 2005; Hood, 2006). For that reason, we chose to use discourse organizer inspired in Appraisal Theory (Martin & White, 2005) and developed by Wallace (2003) and Perales Escudero (2011). For each evaluative segment, this discourse organizer prompts students to identify the source of the evaluation, or Appraiser, and the focus of the evaluation, or Appraised. We didn’t use the SFL metalinguistic terms because a previous study (Perales Escudero, 2011) has shown that ‘evaluation’ is much more readily grasped by the target kind of learners than ‘Appraisal’ or ‘Attitude’ are. Table 3 shows an example of this discourse organizer filled for the following paragraph in Bardovi-Harlig (1999: 472).

“Studies of individual learners carried out in the 1980s provided preliminary support for both hypotheses, although such studies were later treated as anecdotal by Meisel (1987). Important early studies included those by Andersen (1986) and Kumpf (1984). Recent studies have tested the hypotheses on large corpora with increasingly rigorous and quantified analyses (Bardovi-Harlig, 1992)”.

Table 3. Discourse organizer for a paragraph in Bardovi-Harlig (1999).

Evaluation

Source

Focus

Anecdotal (-)
Important (+)
Rigorous (+)
Quantified (+)

Meisel
The author
The author
The author

Studies of the 1980s
Some studies of the 1980s (Andersen; Kumpf)
The author’s own studies
The author’s own studies

Parsing of this kind enables readers to see how the author constructs a complex positional structure where she first represents Meisel’s criticisms of other studies, then aligns with some such studies by qualifying them as important, but then takes up Meisel’s basis for critique by qualifying her own studies as rigorous and quantified. Tellingly, this parsing depends on knowing the attitudinal polarities associated with academic register words such as ‘anecdotal’ and ‘quantified’.

3.8. Identifying patterns of engagement and connecting them to rhetorical inferences

For this strategy, we use Engagement, a sub-system in Appraisal Theory (Martin & White, 2005) that models the linguistic resources used by authors to construct their positions and those of others, including other authors and the idealized reader. Engagement includes options that describe argumentative language, such as Denials (propositions denying others’ claims), Concede (propositions conceding the validity or thruthfulness of others’ claims), and Counter (propositions presenting alternatives -usually one’s own- to the views of others). These engagement options textualize complex patterns of argumentation that appear in move 2 of research article introductions and all over discussion papers. These metalinguistic terms were used without simplification.

4. The delivery method

As a delivery method, we followed direct teaching (Duffy, 2003). This method consists of five steps: 1) understanding the strategies conceptually and meaningfully; 2) understanding the motivation to use the strategies and how their use contributes to learning; 3) learning how to use the strategies step by step; 4) understanding how to deploy the strategies differently depending on textual and contextual factors; 5) evaluating the use of strategies with the goal of monitoring and improving comprehension.

For step 1, the class built knowledge about sociocultural practices of comprehension and their textualization and meaningfulness to the students. First, the students were asked to reflect on how knowledge moves forward in applied linguistics and how those advances are textualized. Through dialogic, conceptual scaffolding, the instructor and the students concluded that knowledge is built through dialog with previous studies. This dialog includes pointing at limitations and signaling conceptual or methodological disagreements. The students concluded that these disagreements and discussions are not generally present in textbook chapters (with exceptions), which are their only disciplinary reading material.

Using textual evidence, the instructor explained that empirical research papers and discussion papers do textualize disagreements and discussions, and it is important to reconstruct them to understand the processes of knowledge production and the changes in what constitutes ‘accepted knowledge’ across time. The students concluded that the activities of knowledge production by experts are like events occurring backstage in a theater play: textbooks tend to present the play to the public as stable knowledge, whereas the journal articles are the backstage events that are known only to the actors, that is, the scholars, and these events present knowledge as unstable and contingent.

The instructor explained that the journal articles are not addressed to undergraduate students, which makes comprehension difficult for this group. However, comprehension is necessary and useful for their training as researchers and professionals capable of keeping up to date with the state of knowledge in applied linguistics. This was step 2. The knowledge thus constructed about the instability of knowledge was connected with the constructs of ‘exigence’, ‘rhetorical move’, ‘engagement’ and ‘evaluative language’ through guided readings along the 12 weeks of the treatment (step 3). This step included the explicit presentation and practice of the strategies through the collective reading and analysis of a text projected electronically on a screen in the classroom. The collective reading and analysis was followed by group reading and analysis and then by individual reading and analysis. To address step 4, the class built knowledge about the rhetorical and structural similarities and differences between empirical and discussion papers. Step 5 was addressed with the application of the meta-strategies.

5. Results

A paired samples t-test was conducted for each pair of pre- and post-tests. As shown in Table 4, there was a significant difference in the comprehension scores for both groups (p<.05). However, Table 4 also shows that the effect size was larger for the experimental group (d = 2.43), than it was for the control group, where it was moderate (d = 0.7).

Table 4. Paired samples t-tests.

  N M SD t df Effect size a
Experimental groups       -16.971* 13 2.43
Pre-test 14 4.36 1.55      
Post-test 14 7.39 1.25      
Control group
        -5.745* 11 0.7
Pre-test 12 4.75 1.42      
Post-test 12 5.75 1.28      

* p<0.05, a = Cohen’s d

In addition, an independent samples t-test revealed that there were no significant differences between the groups in the pre-test, whereas there were significant differences in the post-test (p<.05), with a large effect size (Cohen’s d = 1.6). Taken together, these statistical results indicate that the intervention had a positive impact on students’ main idea comprehension and rhetorical inferential comprehension.

With regard to metacognition, the questionnaire yielded rich data in terms of statements referring to aspects of metacognition as anticipated. We found evidence of the following phenomena: metalanguage, metalinguistic activity, comprehension monitoring, standards of coherence, and processing effort. Example 1 below shows evidence of the latter four processes:

(1) Sí, ahora cuando leo tiendo a aplicar varias estrategias que aprendimos, como cuestionarme, no leer de manera lineal/Now, when I read I tend to apply several of the strategies we learned, like questioning myself, not reading linearly.

The assertion that this leaner questions him/herself (‘questioning myself’) while reading is evidence of the evaluation aspect comprehension monitoring, which was addressed by the two meta-strategies include in the instructional design. Reading in a non-linear way appears to refer to the strategy that we named ‘processing complex nominal groups’ and also to the need to make anaphoric and cataphoric links, which emerged during the intervention and which the instructor emphasized during text- based conversations.

(2) A veces me desanimaba ver la longitud de los textos a leer pero luego era fácil cuando leías e identificabas el tema, debate y contribución./Sometimes I felt discouraged after seeing how long the texts to be read were, but then it was easy once you read and identified the theme, debate and contribution.

In example 2, we see evidence of a heightened ability and willingness to engage in effortful processing. The leaner contrasts her feelings of anxiety at the difficulty involved in reading lengthy texts with a feeling of increased competence and ease that came from using the target metalanguage (topic, debate, contribution). It appears then that this metalanguage and its associated metalinguistic activity enabled the reader to build procedural knowledge that provided her with cognitive resources. These eased her cognitive load by enabling effortful processing and diminishing the anxiety associated with reading a long, difficult text. His/her ability to verbalize this also indicates an impact on conscious awareness that makes this knowledge declarative. A recurring theme that was present in all the questionnaires was a perceived change between the state of prior reading ability and post-intervention reading ability as shown by example 3.

(3) Al menos yo he aprendido a comprender mejor un texto. Es decir, a veces sólo leía y no me detenía a leer frase por frase, es por eso que antes no entendía bien. Ahora cada vez que leo hago un análisis de lo que estoy leyendo./ I for one have learned to understand a text in a better way. I mean, sometimes I just read and did not stop to read each phrase, and that’s why before I didn’t understand well. Now, I analyze what I’m reading every time I read.

The student in this example reflects on her prior reading practices, which evidence a low standard of coherence (‘I just read and did not stop to consider each phrase’), and also indicates a transition to higher standards of coherence, effortful processing and monitoring (‘I now perform an analysis of what I am reading every time I read’). These kinds of references are present in all the other experimental group participants, as in examples 4 and 5.

(4) Antes me daban mucha flojera ese tipo de textos y no entendía nada, ahora ya no y al menos puedo identificar de qué se tratan./Before, I felt discouraged by that kind of texts and didn’t understand anything, now that’s not the case anymore and at least I can identify what they’re about.

(5) Anteriormente leía por leer, no pensaba ni trataba de inferir de qué se trataba el texto, no me concentraba en aspectos tales como: a qué público va dirigido, a qué quiere llegar el autor, no sabía encontrar palabras clave para entender mejor la lectura./ Before I just read for the sake of reading, I did not think or try to infer what the text was about, I did not focus on aspects like the target audience, the author’s intention, I didn’t know how to find the key words to understand the text better.

The learner in example 5 evinces procedural knowledge of the targeted rhetorical inferences (‘target audience’, ‘author’s intended point’) and metalinguistic activity (‘key words’). The fact that the learner reports that she now knows how to deploy strategies is evidence of the planning aspect of comprehension monitoring.

One learner was more precise in her uptake of metalinguistic terms and her ability to relate those to rhetorical inferences:

(6) El dividir el texto en tema, debate y contribución me ha ayudado a hallar más fácil lo que busco en los textos. Entender la posición del autor y diferenciarla de otros. Para ser más específica, me ha facilitado al escribir ensayos./ Dividing the text in topic, debate and contribution helped me find what I’m looking for in texts more easily. To understand the author’s position and distinguish it from the positions of others. To be more specific, it has made essay writing easier.

This learner takes up the metalinguistic terms issuing from the CaRS model (‘topic, debate and contribution’) and explains that locating these helps to make rhetorical inferences (‘understand the author’s position and distinguish it from those of other authors’), which she reports has had a favorable impact on her essay writing. However, only this learner and the learner in example 2 showed any uptake of metalinguistic terms. Other learners show declarative knowledge of rhetorical inferences without connecting them explicitly to the metalanguage or metalinguistic activity. Tellingly, one learner’s answer reveals some impact on cognitive monitoring while simultaneously giving evidence that metalanguage is not a necessary condition for it to occur:

(7) Trato de ser más analítico al leer un texto cualquiera, aunque no siempre me acuerdo de los términos utilizados para la comprensión y la definición de estos términos./ I try to be more analytic when I read any text, although I don’t always remember the terms used for comprehension or their definition.

Importantly, the questionnaire responses of the control group allude to the impact of conscious manipulation of background knowledge and parsing titles carefully to predict content, which was emphasized by that kind of instruction

(8) Hay ciertas estrategias que trato de poner en práctica como desglosar el título y plantearse preguntas o tratar de recordar mi conocimiento previo. Creo que ayudan a entender mejor el texto, lo cual no hacía antes./ There are certain strategies that I try to use, like parsing the title and asking myself questions or trying to remember my prior knowledge. I think they help to understand the text better, which is something I didn’t do before.

Another control group student wrote the following about parsing titles and predicting content from there:

(9) Parsing the title and then imagining what the text would be about by using my previous knowledge has helped me a lot to understand. The funny thing is I think I would have been able to do it before the class, but it either hadn’t occurred to me or I was lazy and didn’t do it.

This student’s statement adds to the body of evidence suggesting that background knowledge and prediction can be powerful strategies. This learner’s assertion that she was lazy to apply the strategies suggest that traditional instruction as implemented with the control group had an impact on the learner’s willingness to engage in effortful processing and, perhaps, raise their standards of coherence, which may assist monitoring. These verbalizations also indicate that these participants may have not been exposed to these traditional strategies or, if they had, the strategies had not been properly implemented with academic texts, or had not been shown to require effort and engagement. Statements of this kind, however, were restricted to only this learner; other learners only made general statements about being aware of the strategies, which shows impact on declarative knowledge.

CONCLUSIONS

Quantitative data analysis shows that both traditional instruction and the intervention had an impact on participants’ ability to make rhetorical inferences with texts in the targeted genres. However, the experimental group’s effect size was larger. With regard to question one concerning the participants’ uptake of metalinguistic terms, the only terms that were clearly taken up by two of the experimental group participants were the adapted names for the CaRS model moves: topic, debate, and contribution (examples 5 and 6). There is no evidence of uptake of any other metalinguistic terms. As for question two focusing on awareness of metalinguistic and metacognitive activity, the use of metalanguage already suggests awareness of rhetorical move parsing; however, other learners show awareness of rhetorical inferencing strategies without connecting them to metalanguage. All learners in the experimental group report metacognitive gains in terms of increased knowledge of the strategies (declarative knowledge), ability to engage in effortful processing and comprehension monitoring (procedural knowledge), and increased standards of coherence. Participants in the control group report metacognitive gains in declarative terms (awareness that the strategies exist) with only one participant showing some impact on standards of coherence (example 9).

From a theoretical standpoint, our results appear to confirm the Metalinguistic Hypothesis: the group exposed to forms of metalanguage and metalinguistic activity that are tailored to match the rhetorical features of the target genres was the group that experienced greater gains in inferential comprehension and metacognition. The students’ statements also suggest a connection between increased metalinguistic awareness, raised standards of coherence and comprehension monitoring abilities, and an emerging willingness to engage in effortful processing. It appears, then, that the quality of readers’ metalinguistic awareness allows readers to exercise conscious activity on their developing mental representation of the text, or textbase. Having words to describe this emerging textbase and/or engaging in procedures that require conscious attention to it results in greater focalization, which enables monitoring and effortful processing, and the emergence of higher standards of coherence. From the perspective of Communicability Theory (Parodi, 2011), promoting metacomprehension of specific genres would require different forms of metalinguistic activity and metalanguage, as textbase features would vary across genres.

Metalanguage uptake was very limited; it seems that what was most important was metalinguistic activity, thus confirming Svalberg’s (2007) claim that the specific forms of metalanguage are less important than the kinds of metalinguistic activity they promote. However, we doubt that the same results could have been attained without the target metalanguage, or some form of metalanguage that closely represented and allowed the negotiation of the targeted types of metalinguistic activity.

These results have implications for current instructional practices in EFL teaching of reading. While a role for background knowledge and text structure awareness is well-established in the literature, our results suggest that these practices could be usefully combined in the classroom with careful parsing of evaluative language and moves in combination with rhetorical genre knowledge. Specifically, a case can be made that some forms of metalanguage at the discourse/rhetorical level is beneficial. While our study suggests that move-related metalanguage was taken up, we do not know if the learners developed passive knowledge of the other metalinguistic terms or whether that knowledge is related to their inferential comprehension and metacognition abilities.

Our study is limited because it involved a small sample and the same instructor taught both the experimental and control groups. It also did not use think-aloud protocols or elicited passive understanding of metalinguistic terms. Further, the cognitive mechanisms involved in the learners’ increased willingness to engage in effortful processing remain unclear, although they could be due to an increased sense of self-efficacy, and a simultaneous building and freeing of attentional resources. Future studies should investigate this issue, as well as the transferability of the design with other instructors in different contexts. The connections between measurements of metacognition and metalinguistic awareness could be further explored with quantitative instruments that measure passive knowledge of metalinguistic terms.


* ACKNOWLEDGMENTS

We would like to thank Professors Gerardo Bañales Faz and Norma Alicia Vega López of Universidad Autónoma de Tamaulipas for their helpful comments during the design and execution of this study.


REFERENCES

Baker, L. & Brown, A. L. (1984). Cognitive monitoring in reading. In J. Flood (Ed.), Understanding reading comprehension: Cognition, language and the structure of prose (pp. 21-43). Newark, Delaware: International Reading Association.         [ Links ]

Bardovi-Harlig, K. (1999). Narrative structure and lexical aspect: Conspiring factors in second-language acquisition of tense-aspect morphology. Studies in Second Language Acquisition, 20, 471-508.         [ Links ]

Bateman, B. (2008). Student teachers’ attitudes and beliefs about using the target language in the classroom. Foreign Language Annals, 41(1), 11-28.

Belcher, D. (2007). A bridge too far? TESOL Quarterly, 41(2) 396-399.         [ Links ]

Bitzer, L. (1968). The rhetorical situation. In W. A. Covino (Ed.), Rhetoric: Concepts, definitions, boundaries (pp. 1-14). Boston, M.A.: Allyn and Bacon.         [ Links ]

Brantmeier, C. & Dragiyski, B. (2009). Toward a dependable measure of metacognitive reading strategies with advanced L2 learners. In C. Brantmeier (Ed.), Crossing languages and research methods: Analyses of adult foreign language reading (pp. 47-72). Charlotte, NC: IAP.         [ Links ]

Camps, A., Guasch, O., Milian, M. & Ribas, T. (2000). Metalinguistic activity: The link between writing and learning to write. In A. Camps & M. Milian (Eds.), Metalinguistic activity in learning to write (pp. 108-123). Amsterdam: Amsterdam University Press.         [ Links ]

Collins, L. (2002). The roles of L1 influence and lexical aspect in the acquisition of temporal morphology. Language Learning, 52(1), 43-84.         [ Links ]

Duffy, G. (2003). Explaining reading: A teacher’s resource for teaching concepts, skills and strategies. New York: Guilford Press.

Englander, K. (2006). Revisions of scientific manuscripts by non-native English speaking scientists in response to journal editor’s criticism of the language. Journal of Applied Linguistics, 3(2), 129-161.

Flavell, J. (1979). Metacognition and cognitive monitoring. American Psychologist, 34(10), 906-911.         [ Links ]

Flavell, J. (1985). Cognitive development. Englewood Cliffs, N. J: Prentice-Hall.         [ Links ]

Gibbons, P. (2002). Scaffolding language, scaffolding learning. Portsmouth, NH: Heinemann.         [ Links ]

Grabe, W. (2009). Reading in a second language: Moving from theory to practice. Cambridge: Cambridge University Press.         [ Links ]

Han, Z. & D’Angelo, A. (2009). Balancing between comprehension and acquisition: Proposing a dual approach. In Z. Han, N. J. Anderson & D. Freeman (Eds.), Second language reading research and instruction: Crossing the boundaries (pp. 173-191). Ann Arbor, MI: The University of Michigan Press.

Han, F. (2012). Comprehension monitoring in reading English as a foreign language. New Zealand Studies in Applied Linguistics, 18(1), 36-49.         [ Links ]

Hazelrigg, A. C. (2009). Second language reading research: A critical review [on line] Retrieved from: http://www.eric.ed.gov/ERICDocs/data/ericdocs2sql/ content_storage_01/0000019b/80/3e/c4/72.pdf        [ Links ]

Hsieh, H. F. & Shannon, S. E. (2005). Three approaches to qualitative content analysis. Qualitative Health Research, 9(15), 1277-1288.         [ Links ]

Hood, S. (2006). The persuasive power of prosody-radiating values in academic writing. Journal of English for Academic Purposes, 5, 37-49.         [ Links ]

Hyland, K. (2005). Metadiscourse: Exploring interaction in writing. New York: Continuum.         [ Links ] Kintsch, W. (1998). Comprehension: A framework for cognition. New York: Cambridge University Press.         [ Links ]

Kolic-Vehovec, S. & Bajsanski, I. (2007). Comprehension monitoring and reading comprehension in bilingual students. Journal of Research in Reading, 30(2), 198-211.         [ Links ]

Kupferberg, I. & Olshtain, E. (1996). Explicit contrastive instruction facilitates the acquisition of L2 forms. Language Awareness, 5(3-4), 149-165.         [ Links ]

Kumaravadivelu, B. (2006). TESOL methods: Changing tracks, challenging trends. TESOL Quarterly, 40(1), 59-81.         [ Links ]

Larsen-Freeman, D. (2000). On the appropriateness of language teaching methods in language and development. In J. Shaw, D. Lubelske & M. Noullet (Eds.), Proceedings of the fourth international conference on language and development (pp. 65-71). Bangkok, Tailandia: Asian Institute of Technology.         [ Links ]

Lewin, B. A., Fine, J. & Young, L. (2001). Expository discourse: A genre-based approach to social science research texts. New York: Continuum.         [ Links ]

Martin, J. R. & Rose, D. (2008). Genre relations: Mapping culture. London, Inglaterra: Equinox.         [ Links ]

Martin, J. R. & White, P. R. R. (2005). The language of evaluation: Appraisal systems in English. New York: Palgrave Macmillan.         [ Links ]

McKeown, R. & Gentilucci, J. L. (2007). Think-aloud strategy: Metacognitive development and monitoring comprehension in the middle school second language classroom. Journal of Adolescent and Adult Literacy, 51(2), 136-147.         [ Links ]

Maxwell, J. (2010). Using numbers in qualitative research. Qualitative Inquiry, 16(2), 475-482.         [ Links ]

Mokhtari, K. & Reichard, C.A. (2002). Assessing students’ metacognitive awareness of reading strategies. Journal of Educational Psycholog y, (94)2, 249-259.

Nagy, W. (2007). Metalinguistic awareness and the vocabulary-comprehension connection. In R.K. Wagner, A. Muse & K.R. Tannenbaum (Eds.), Vocabulary acquisition: Implications for reading comprehension (pp. 52-77). New York: The Guilford Press.         [ Links ]

Parodi, G. (2011). La Teoría de la Comunicabilidad: Notas para una concepción integral de la comprensión de textos escritos. Revista Signos. Estudios de Lingüística, 44(76), 145-167.         [ Links ]

Paris, S. C., Lipton, M. Y. & Wixson, K. K. (1983). Becoming a strategic reader. Contemporary Educational Psycholog y, 8, 293-316.         [ Links ]

Perales Escudero, M. D. (2011). Teaching and learning critical reading at a Mexican university: An emergentist case study. Unpublished doctoral dissertation, University of Michigan, Ann Arbor, EUA.         [ Links ]

Perales Escudero, M.D. (2013). Attractor states, control parameters and co-adaptation in L2 inferential comprehension: A design-based study. Revista Brasileira de Linguística Aplicada, 13(2), 463-492.         [ Links ]

Perales Escudero, M. D. & Reyes Cruz, M. R. (2014). Enseñanza de estrategias de comprensión inferencial en inglés L2 y su impacto en español L1: Implicaciones teóricas y prácticas para la lectura en las disciplinas. Revista Mexicana de Investigación Educativa, 61, 599-626.         [ Links ]

Peronard, M. (2002). Conocimiento de estrategias de lectura y metacomprensión. Onomazein, 7, 95-105.         [ Links ]

Peronard, M. & Velásquez, M. (2003). Desarrollo del conocimiento metacomprensivo. Revista Signos. Estudios de Lingüística, 36(54), 89-101.         [ Links ]

Pulido, D. (2009). Vocabulary processing and acquisition through reading: Evidence for the rich getting richer. In Z. Han, N.J. Anderson & D. Freeman (Eds.), Second language reading research and instruction: Crossing the boundaries (pp. 65-82). Ann Arbor, MI: The University of Michigan Press.         [ Links ]

Rose, D. (2006). A reading-based model of schooling. Pesquisas em Discurso Pedagogico 4(2) [on line] Retrieved from: http://www.maxwell.lambda.ele.puc-rio.br        [ Links ]

Rose, D. & Martin, J. R. (2012). Learning to write/reading to learn: genre, knowledge and pedagog y in the Sydney School: Scaffolding democracy in literacy classrooms. London: Equinox.         [ Links ]

Schleppegrell, M. (2013). The role of metalanguage in supporting academic language development. Language Learning, 63(1), 153-170.         [ Links ]

Svalberg, A. (2007). Language awareness and language learning. Language Teaching, 40, 287-308.         [ Links ]

Swales, J. (1990). Genre analysis. Cambridge: Cambridge University Press.         [ Links ]

Van den Broek, P., Virtue, S., Everson, M., Tzeng, Y. & Sung, Y. C. (2002). Comprehension and memory of science texts: Inferential processes and the construction of a mental representation. In J. Otero, J. A. León & A. C. Graesser (Eds.), The psycholog y of science text comprehension (pp. 131-154). Mahwah, NJ: Lawrence Erlbaum.         [ Links ]

Wagoner, S. (1983). Comprehension monitoring: What it is and what we know about it. Reading Research Quarterly, 18(3), 28-346        [ Links ]

Wallace, C. (2003). Critical reading in language education. New York: Palgrave Macmillan.         [ Links ]

Recibido: 23-X-2013 / Aceptado: 05-XII-2014


APPENDIX

Pre-/Post-test

Preguntas sobre Bateman (2000)
1. ¿Cuál es la motivación del autor para escribir este artículo?
2. ¿Cuál es el mensaje principal que pretende transmitir?
3. ¿Cuál es la posición del autor con respecto a la cita de Macaro (1997)?
4. ¿Cuál es la posición del autor con respecto a la cita de Turnbull (2001)?
5. ¿Cómo describirías el tipo de lector se dirige el autor en la sección de introducción?

Preguntas sobre Belcher (2000)
1. ¿Cuál es la posición de Han (1999) presentada por el autor?
2. ¿Cuál es la posición del autor con respecto a Han (1999)?
3. ¿Cuál es la motivación del autor para escribir este artículo?
4. ¿Cuál es el mensaje principal del autor?
5. ¿Cómo describirías el tipo de lector al que se dirige el autor?

Creative Commons License Todo el contenido de esta revista, excepto dónde está identificado, está bajo una Licencia Creative Commons