PublicationsSystematic reviewsMFL strategy trainingMFL strategy training - summary
Strategy training in language learning - a systematic review of available research. Summary

Background

Strategy training in language learning has been topical since the 1970s, and a large amount of work has been done on identifying the strategies used by both successful and less successful learners, and by users of modern languages. The strategies have generally been classified as metacognitive (to do with awareness of the learning), cognitive (to do with the behaviours and mental process of the learning) and socioaffective (to do with personality traits and interactions with others). It is generally held that the skills we develop for learning in general and for learning our first language do not automatically transfer to other learning situations or to other languages - hence the rationale for research into strategies and their potential benefit if the skills are trainable.

Strategy training is defined in this review as any intervention which focuses on the strategies to be regularly adopted and used by language learners to develop their proficiency, to improve particular task performance, or both; a simple example of a strategy in the area of reading skills is brainstorming a theme and making notes in the margin before and during the reading of the text; an intervention that might help language learners to read and understand better.

Aims

The aims of the review are to identify and evaluate the primary research on strategy training in order to gather together, present and comment on the strength of evidence about its effectiveness in teaching and learning of languages.

Rationale

No previous systematic overview of strategy training has been carried out, and literature reviews in the area have not evaluated findings in terms of research type or robustness. At a time when the UK education system, for example, is looking at the possible introduction of strategy training, it is important to get as complete a picture as possible of its effectiveness as revealed through research. This review's systematic searching for study reports, not generally a feature of traditional reviews, aimed to ensure that all relevant research evidence was considered, not just that which is most easily accessible. This approach was taken since it is known that the most easily accessible studies may often represent a biased subset of the whole.

Review questions

As its primary question, the review asks the following:

What is the effectiveness of strategy training?

Within this, we also wanted to uncover evidence of differential effectiveness for different languages, different learners (school, university, adult), different stages of learning (beginner, intermediate, advanced), and different language skills (reading, writing, listening, speaking, overall ability, etc). In doing so, we hoped to explore why different types of strategy training might or might not work.

Methods

A peer-refereed protocol, detailing the steps for the review, was published via the EPPI-Centre website. A brief description of key stages is given below, but the main steps in the process include extensive searching for reports of studies, application of inclusion and exclusion criteria to references found, production of a descriptive map of included studies, an in-depth review describing and appraising the quality of a subset of the mapped studies, and a synthesis of findings from these studies. At least two reviewers independently carried out each step of the work, and quality assurance was provided by the EPPI-Centre.

User involvement

The core review group was constituted so as to involve people with experience of research, language teaching at compulsory and post-compulsory levels, and language centre leadership. In addition, consultation events were held with tertiary level students, Government policy-makers and educational researchers expert in the field.

Inclusion criteria

To be included in the map, reports needed to be:

  1. of a strategy training intervention in language learning
  2. of an intervention carried out in a formal setting such as groups of learners in schools, universities and language centres
  3. a study not primarily involving bilingual learners
  4. of primary, empirical research
  5. of research carried out since 1960.

Studies were included in the in-depth review if they met all the inclusion criteria for the map and were also experimental in design, testing the effect of an intervention against either another intervention, standard practice, or no intervention.

Search strategy

In summer 2002, 17 electronic databases were searched for studies dating back to 1960 using a range of terms for strategies, strategy learning, or strategy training, in combination with terms for language learning or teaching.

Characterising studies, in-depth review and weight of evidence

To produce the map, studies were screened using the first set of inclusion criteria described above. Relevant studies were classified according to a standardised 'core' keywording system developed by the EPPI-Centre. Review-specific keywording, drawn up by the review group, was also applied so as to describe studies further in terms of the type of strategy training provided and outcomes measured.

Studies included in the in-depth stage of the review were subject to examination using EPPI-Centre and review-specific data-extraction tools. Key elements (such as aims, methodology, context and results) were described and, at the same time, judgements were made about the quality of the reported study. These descriptions and judgements were used to determine a 'weight of evidence' composed of three sections: the trustworthiness of the reported study, the appropriateness of design and analysis for the review question, and the relevance of the focus of the study to answering the review question. A narrative synthesis was drawn up, with each study described and weighted alongside others focusing on similar areas of language learning.

Results

Mapping of all included studies

A total of 567 potentially relevant references were found, and following application of selection criteria, 38 studies (in 40 reports) were finally included in the descriptive map. Most of the studies were on learning English as a foreign or second language (24) but Spanish, French, German, Russian, Japanese, Italian and Latin were also studied. Overall, most of the studies were carried out in the USA (16), but other studies were carried out in Canada, Australia, Hong Kong, Japan, Singapore, Egypt, Germany, Korea, the Netherlands, New Zealand, Poland, Turkey and the UK. Only 11 of the studies involved school students, and the rest were of university, higher education or adult students of languages.

Of the 38 studies in the map, 28 were controlled or randomised controlled trials (RCTs) of interventions, while the rest were descriptive intervention studies, case studies, ethnographic, action research, and one interrupted time series study. Sample sizes varied greatly, and cluster randomisation was frequently used (rather than randomisation of individuals to intervention or control groups). Intervention length also varied greatly from short one- or two-hour awareness-raising interventions to year-long programmes of study.

The outcomes measured included accuracy of language output, asking and answering higher order questions, attitude, awareness, comprehension, strategy use, writing ability and vocabulary recall among others. The effects of the interventions on these outcomes were measured using a wide variety of tests and instruments designed specifically by the researchers (interview, survey, self-report, questionnaire, multiple choice and other tests), but in some cases standard end-of-term tests and grade systems were also used.

Studies selected for in-depth review

A subset of 25 of the 38 mapped studies were experimental in design and available in full within the timetable of the in-depth review and these were included in the in-depth review.

Areas of language learning covered by the 25 studies were speaking, reading, writing, overall language ability, vocabulary and listening. Only 10 of these studies explicitly classified the training as focused on metacognitive, cognitive or socio-affective strategies, although the majority (14) were considered by reviewers to be concerned with cognitive strategies or a mixture including cognitive (5). Several of the studies (5) looked at training in semantic and structural mapping of texts (that is, uncovering the meaning and form of a text before or while studying it in detail) in order to improve comprehension. Other interventions involved training in strategies to ask and answer 'higher order' questions to improve overall and speaking ability; mnemonics or keywords for learning vocabulary; focusing on specific grammar items either at the input or output stage; and strategies to improve writing. Of interest to this review is the effectiveness of strategy training, rather than the specific strategies.

Is strategy training effective?

There is sufficient research evidence to support claims that training language learners to use strategies is effective, but it is not possible to say from this evidence whether the effect of training is long-lasting or not. Furthermore it is not really known to what extent the specific mechanics of different training interventions are responsible for the effect, or if it is due to improved awareness that a broad range of training might engender in the learner.

Most studies (N=17) report only positive results. Fewer (N=6) report mixed results, such as reporting a positive finding for one outcome and a negative outcome for another. Only two studies report only negative findings.

While most studies report positive findings to a greater or lesser extent, the weight of evidence varies across the different language skills. For speaking ability, for example, training learners to use certain strategies appears successful but the evidence is not compelling (small number of studies, varied relevance, varied reliability) while training in semantic and structural mapping to improve reading comprehension is supported by more robust evidence. Only two studies were found for strategies to improve writing ability (both based on learners revising and/or rewriting first drafts) and both reported positive results - so, despite the small number of studies, the strength of the evidence in them is relatively strong. Interventions to improve overall language ability gave mixed results, and the differences between training interventions plus the design and execution of the studies made it difficult to draw strong conclusions. Similarly, the evidence for the effectiveness of interventions to improve listening ability and vocabulary ability was weakened by methodological characteristics of the studies.

It should be borne in mind that weak evidence does not mean that strategy training does not work - only that the evidence is weak. This is self-evident perhaps, but important as regards the implications: more evidence, or evidence of greater strength, might demonstrate the effect of training more clearly but one should not discount the possibility of findings in the other direction. Only two out of the 25 studies report negative findings, although a further six show mixed results, with training appearing to work for some aspects of language learning and not for others, or there being no clear difference in outcomes between two strategy training interventions.

Questions that need to be addressed through deeper analysis and by further development of the review include the extent to which the effectiveness of strategy training depends on particular aspects of any given teaching/learning situation: for example, the level and stage of learning, the language in question, the first language of learners, the age of learners, prior learning experience, and the similarity of one strategy training context to another.

What sort of studies were found?

All the studies included in the in-depth review were comparative studies looking at one or more interventions compared with another (which might have been an alternative training intervention or a control/usual practice one). Most of them used groups of learners, either in pre-existing classes or randomly formed, but very few of the studies randomised individual learners to the intervention group. Usually the learners were tested before and then immediately after the intervention, and none of the studies retested all the learners again after 6 or 12 months or longer (a few studies did some selective delayed post-testing). Sometimes the effect on the learners was measured through their regular grades or end of term tests, while in other studies, particular tests were constructed and used. The importance of this is that some of the studies were more 'naturalistic' than others, and this has a bearing on the generalisability of the findings. Some studies related evidence of improvement in performance or ability to increased strategy use, with corroboration of the findings between both performance and frequency of strategy use. There was great variety in the interventions over time and place.

How reliable, relevant and strong were the studies?

The review found that in terms of the total of 27 findings about learning outcomes (two studies reporting on two findings each), three were highly reliable and 15 were of medium reliability - together over half of the included studies' findings - leaving nine of (relative) lesser reliability. This should be seen in the light of all the studies being experimental and a priori more reliable than other forms of research in determining the effect of an intervention on groups of learners (rather than on individuals).

Twenty-five studies were highly relevant to the review question that we were asking. Without prior knowledge of exactly how much research existed, a good number of experimental research studies were found and this provides a convincing example and counter argument to claims that much educational research is not applied and is irrelevant.

The review found that 16 of the studies were considered to be of high or medium weight, and nine of low weight in the evidence they provided: that is to say in terms of overall relevance to the question of the review, appropriateness in their design, and reliability in how they were carried out. This incorporated, in addition to the standardised EPPI-Centre guidelines for evaluation, findings from review-specific questions including indicators in the studies of clear definition of the strategies, clarity in how the strategy is expected to lead to improved learning or performance, and a relationship between the intervention and the eventual autonomous language learning behaviour of the learners.

Conclusions

Implications for teaching practice

Strategy training for language learning is effective but a number of conditions restrict the universality and usefulness of this finding. Since there have been no direct comparisons, it is not clear whether relatively simple programmes to raise awareness are any more or less effective than longer, expensive, complex interventions. That is to say, it may be possible to make a positive impact on the learner with one or two sessions that 'open their eyes' to something. It is unclear what additional impact may be gained through longer interventions that model the behaviours and then seek to have learners imitate and autonomously adopt them. Furthermore, the long-term benefits, or otherwise, of strategy training interventions have yet to be evaluated sufficiently in experimental studies.

Strategy training works for reading comprehension and writing skills, and the research evidence for this is stronger than it is for listening, speaking and overall proficiency.

Practitioners might select from the strategy training interventions found in this review but should also assess carefully their learners and the pedagogic situation in question in estimating the likelihood of applicability and benefit of the outcomes. Additional aspects to consider include whether long or short intervention programmes are required, how similar or different the learners are in relation to those in the experiments, and whether the level and stage of learners are important variables in the particular intervention of interest.

Implications for policy-makers

The evidence shows that it is worth considering strategy training programmes for language learning on a policy level, as research shows that it is effective in certain situations. As the evidence found is primarily for its effectiveness with adult and higher education learners, more information is needed before straightforward evidence is available for school-level learners. Evidence of a high and consistent standard is still needed, to get a clearer picture in terms of designing policy, and of different levels and stages of learners. Further evidence should be sought to demonstrate the longer-term effects (say after one year) of training; and research needs to be carried out to investigate the differences and relative effectiveness of awareness-raising and more intensive strategy modelling programmes - these elements have a bearing on the cost-effectiveness, both financially and pedagogically, of programmes that might be delivered on a wider scale.

Recent discussions between three EPPI-Centre systematic review groups (Science, Thinking Skills and Modern Foreign Languages) and policy-makers discovered a number of emerging common points that are relevant, including the hypothesis that a combination of behaviour modelling (cognitive strategies) and awareness-raising (metacognitive strategies) rather than either alone, may be at the heart of the improvements observed.

Implications for research

The review has been able to draw some clear conclusions and some equally clear ideas for future research.

Concerning strategy training first, not enough is known about the processes and mechanisms that are in operation, and therefore it remains unclear as to exactly what is making the difference when a strategy training intervention is said to be effective. Future studies should focus on uncovering these mechanisms by tightly controlling the variables in order to isolate the associations between cause and effect, by observing more closely the incidence of strategy use following the intervention, and by seeking in particular to differentiate between the effect of awareness raising and behaviour modelling. In addition, the difference between discrete strategies and 'packages' of strategies should also be investigated further.

The research identified in this review has been predominantly with non-school populations, and more evidence of strategy training for school learners of modern languages is needed, particularly if large-scale training programmes are to be considered.

Naturalistic evaluations are desirable - that is, studies that attempt to test the intervention with as little disturbance as possible in the day-to-day patterns of the learners in question: this includes using regular tests and grades/scores to measure the effects, and perhaps crossover trials so all participants receive the same treatment overall. If specialised measurement tests and instruments are used, these need to be validated and standardised where possible. It would be good to harmonise approaches across the research community to seek greater synthesis of findings in different studies.

Other enhancements to research designs include carrying out delayed post-testing of the intervention (to see how long the effect lasts) and incorporating case study and ethnographic methods into RCTs to assist with corroboration of overall findings for groups of learners and the effect on individuals within the groups. Clearer randomisation procedures, planned sampling strategies and better reporting will make the research more reliable and generalisable.

Strengths and limitations of the review

This is the first such systematic review of research evidence in relation to the question. It demonstrates clearly that evidence exists in support of the effectiveness of strategy training in language learning (i.e. that it works), although the caveats constraining this broad statement are important. The review brings to light research evidence that was not previously in the mainstream body of knowledge, and it highlights areas of need in terms of future research, research method and quality.

In terms of limitations, the review looks at an average overall question and does not examine in detail the processes and mechanisms of what is happening in a successful intervention. Meta-analysis of findings has not been attempted and only 'crude' syntheses of the evidence based on directions of effect were carried out. The evidence needs to be interpreted carefully by practitioners, policy-makers and researchers for specific contexts (e.g. schools versus tertiary sector, beginner versus advanced learner, etc.), and the findings are not immediately transferable to all language learning situations. Furthermore, it was not possible for this review to handsearch journals or to cover many non-English language databases systematically. It is possible that further evidence remains to be identified and included in updates of the review.

This report should be cited as: Hassan X, Macaro E, Mason D, Nye G, Smith P, Vanderplank R (2005) Strategy training in language learning: a systematic review of available research. In: Research Evidence in Education Library. London: EPPI-Centre, Social Science Research Unit, Institute of Education, University of London.

  
Copyright 2019 Social Science Research Unit, UCL Institute of Education :: Privacy Statement :: Terms Of Use :: Site Map :: Login
About::Projects::Publications::Research Use::Courses & seminars::Resources::Databases::Blog