Other Researchers

The key researchers differ significantly when their top strategies are compared.

1. Hattie, Marzano, and Dylan Wiliam:

The most influential researchers differ in so many areas although formative assessment and feedback are common.

Hattie's Top StrategiesMarzanoProf Dylan Wiliam
1. Self-Reported GradesIdentifying Similarities and DifferencesFormative Assessment
2. Piagetian programsSummarizing and Note-takingFeedback
3. Formative evaluationReinforcing Effort+Providing RecognitionPeer Tutoring
4. Micro-teachingHomework and PracticeSelf-regulated learning
5. AccelerationNonlinguistic RepresentationsDeliberate practice
6. Classroom behavioralCooperative LearningTeacher Mindset
7. Interventions-learning disabledSetting Objectives and Providing FeedbackTeachers as critical friends
8. Teacher clarityGenerating and Testing Hypotheses
9. Reciprocal teachingCues, Quest, and Advanced Organizers
10. Feedback

Marzano has been criticised by Dr J Becker and Matthew Fitzpatrick for similar reasons to Hattie - poor research and conflicts of interest.

Hattie's top strategies have changed a little recently, I have not had time yet to review them.

I also think Professor Wiliam's strategy of influencing teacher's mindsets is important. Teachers must want to improve or everything else is secondary.

2. The U.S Dept of Education Recommendations:

The U.S Education Department fund a research group consisting of 8+ distinguished professors and 15+ PHD research scientists. They review research with a focus on the quality of research. So far nearly 12,000 individual educational research studies have been reviewed. Their detailed recommendations are here.

As already stated, most of Hattie's studies would fail the U.S. Educational Department quality standards.

It is significant that the U.S. Educational Department provide some caution in their recommendations with the 'Level of Evidence' rating. This is at odds with Hattie's confidence about his own conclusions. Also, one of the strong features of these practice guides is that they are subjected to rigorous external peer review (pvii). Hattie rarely responds to the critique from numerous peer reviews of his work.

U.S. Ed Dept Recommendations (p2):

U.S. Ed Dept Recommendation versus HattieLevel of EvidenceHattie's Rank
Space learning over time.Moderate70?
Interleave worked example with problem-solving exercises.Moderate20,30,118?
Combine graphics with verbal descriptions.Moderate104
Connect and integrate abstract and concrete representations.Moderate82?
Use pre-questions to introduce a new topic.Minimalnot ranked
Use quizzes to re-expose students to key content.Strongnot ranked
use delayed judgments of learning to identify.Minimalnot ranked
Tests and quizzes to identify content that needs to be learned.Minimal3?
Ask deep explanatory questions.Strong53?

3. The Education Endowment Fund:

Is an independent English organisation with a funding of around $250 Million. It has 5 full-time researchers and has published an evidence guide similar to the US Education Dept, but adds a helpful costing feature.

Most of its findings contradict those of Hattie - click here for full list:

4. The Sutton Trust:

With a focus on good quality research, the two factors with the strongest evidence of improving pupil attainment are:

a. Teachers’ content knowledge, including their ability to understand how students think about a subject and identify common misconceptions - a direct contradiction to Hattie's work where this is ranked #125 with a small effect size = 0.09).

b. The quality of instruction, which includes using strategies like effective questioning and the use of assessment.

5. The Deans For Impact:

Produced a document The Science of Learning which details the key strategies for effective teaching, e.g.,

a. Students learn new ideas by reference to ideas they already know.

b. Information is often withdrawn from memory just as it went in. We usually want students to remember what information means and why it is important, so they should think about meaning when they encounter to-be-remembered material.

c. Practice is essential to learning new facts, but not all practice is equivalent.

d. Self-determined motivation (a consequence of values or pure interest) leads to better long-term
outcomes than controlled motivation (a consequence of reward/punishment or perceptions of self-worth).

e. Effective feedback is often essential to acquiring new knowledge and skills.

6. Professor Jo Boaler:

Jo Boaler's focus is on visual representations of abstract ideas which seems to directly contradict Hattie's rankings where 'simulations' and 'visual representations' are ranked very lowly.

And Jo Boaler's example of simulations:

7. Professor Michael Fullan:

In ACEL Monograph 52 Fullan states,

'there was one finding that stood out as twice as powerful as any other factor in 'effect size' - principals who participated as learners working with teachers to make improvements had twice the impact on school-wide student achievement compared to any other factor' (p3).

8. The Grattan Institute:

Professor Wiliam also provides some extra useful strategies on the teacher. He suggests teachers must want to improve and teachers need to act as critical friends. The Grattan Institute analysed the high performing international educational systems and concluded that one of the reforms responsible for improving student achievement across the four high-performing education systems in East Asia was teachers acting as critical friends. Note the amount of time devoted to feedback, lesson planning, etc. Although, one caveat in these systems is that class sizes are higher (Grattan Report p14).

9. PISA 2015:

David Didau in his excellent blog on feedback pointed out that even though most researchers have feedback as an important teaching strategy, PISA has feedback NEGATIVELY correlated with Science performance.

From PISA 2015 Volume 2 (page 228)

The Australian Government Productivity Commission.

In their latest report (2017) they make a number of recommendations regarding educational research.

Firstly, a focus on the quality of research:

" ... the gold standard techniques are meta analyses of randomised controlled trials and individual trials. Such approaches are widely used in health research, but are not routinely used in Australian education research" (p17).

Verifying the quality of the research (p19):

"A range of processes can be used to ensure the findings from completed research are robust. These include independent validation of the findings, peer review of research, publication of all outputs to enable scrutiny and debate (irrespective of findings), and the provision of project data for secondary analysis."

Their recommendations (p28):

In assessing whether to improve the quality of existing education data, governments should examine on a case by case basis whether:
the existing quality of the data is fit for purpose
data quality improvements are feasible given the context of data collection
other options are available
the benefits of improving data quality exceed the costs.

The Australian Government should request and sufficiently fund the agencies that conduct the Longitudinal Study of Australian Children to establish new cohorts at regular intervals. 

U.S. Ed Dept Levels of Evidence guide:

The US Education department state, 'In classifying levels of empirical support for the effectiveness of our recommendations, we have been mindful not only to the issue of whether a study meets the “gold-standard” of a randomized trial, but also to the question “Effective as compared to what?” Virtually any educational manipulation that involves exposing students to subject content, regardless of how this exposure is provided, is likely to provide some benefit when compared against no exposure at all. To recommend it, however, the question becomes “Is it more effective than the alternative it would likely replace?” In laboratory studies, the nature of instruction in the control group is usually quite well defined, but in classroom studies, it is often much less clear. In assessing classroom studies, we have placed the most value on studies that involve a baseline that seems reasonably likely to approximate what might be the ordinary practice default' (p3).

No comments:

Post a Comment