Other Issues

Conflict of Interest:


It is difficult to get a man to understand something, when his salary depends on his not understanding it.” Upton Sinclair

Professor Ewald Terhardt (2011), "A part of the criticism on Hattie condemns his close links to the New Zealand Government and is suspicious of his own economic interests in the spread of his assessment and training programme (asTTle). Similarly, he is accused of advertising elements of performance-related pay of teachers and he is being criticised for the use of asTTle as the administrative tool for scaling teacher performance. His neglect of social backgrounds, inequality, racism, etc., and issues of school structure is also held against him. This criticism is part of a negative attitude towards standards-based school reform in general. However, there is also criticism concerning Hattie’s conception of teaching and teachers. Hattie is accused of propagating a teacher-centred, highly directive form of classroom teaching, which is characterised essentially by constant performance assessments directed to the students and to teachers" (p434).

Hattie is once again promoting asTTle in his collaboration with Pearson - What Works In Education but fails to divulge his financial interest in the program (p13).

Professor John O'Neill wrote a timely warning, in 2012, about Hattie's influence on Education policy and his financial interest in the solutions proposed:

The "discourse seeks to portray the public sector as ‘ineffective, unresponsive, sloppy, risk-averse and innovation-resistant’ yet at the same time it promotes celebration of public sector 'heroes' of reform and new kinds of public sector 'excellence'. Relatedly, Mintrom (2000) has written persuasively in the American context, of the way in which ‘policy entrepreneurs’ position themselves politically to champion, shape and benefit from school reform discourses" (p2).

Hattie quotes Cohen (1985) "New and revolutionary ideas in teaching will tend to be resisted rather than welcomed with open arms, because every successful teacher has a vested intellectual, social, and even financial interest in maintaining the status quo" (p252).

This is a disappointing inference directed at teachers, given Hattie's collaboration with Pearson; who paid Hattie for the intellectual rights to Visual Laboratories and Hattie's financial interest in the solutions provided to schools.

Dr Jonathan Becker, similarly critics Marzano, for his lack of independence, due to his financial arrangement with Promethean in his research.

Nick Rose goes into more detail regarding financial conflicts of interest and research.

Joshua Katz's YouTube presentation went viral regarding financial conflict of interest in Education.

Contradictions & Inconsistencies:


Hattie defines "influence" as any effect on student achievement. But this is too vague and leads to many contradictions & inconsistencies.

In his preface, Hattie states that "The book is not about classroom life, and does not speak of its nuances ..."

However, his influences consist of a large number classroom nuances: behaviour, feedback, motivation, ability grouping, worked examples, problem-solving, micro teaching, teacher-student relationships, direct instruction, vocabulary programs, concept mapping, peer tutoring, play programs, time on task, simulations, calculators, computer assisted instruction, etc.

Also in his preface he states, "It is not a book about what cannot be influenced in schools - thus critical discussions about class, poverty, resources in families, health in families, and nutrition are not included."

Yet he has included these in his rankings:

Home  environment, d = 0.57 rank 31
Socioeconomic status, d = 0.57 rank 32
Pre-term birth weight, d = 0.54 rank 38
Parental involvement, d = 0.51 rank 45
Drugs, d = 0.33 rank 81
Positive view of own ethnicity, d = 0.32 rank 82
Family structure, d = 0.17 rank 113
Diet, d = 0.12 rank 123
Welfare  policies, d = -0.12 rank 135

Passion

In Hattie's 2012 update to VL he states, "Throughout Visible Learning, I constantly came across the importance of ‘passion’; as a measurement person, it bothered me that it was a difficult notion to measure – particularly when it was often so obvious" (preface).

Passion is not included in Hattie's list of influences, yet he raises it as one of the most important influences!

Teacher Training and Experienced Teachers 

Hattie uses his own research on 65 teachers comparing NBC with Non-NBC teachers and reports this in the last chapter of VL. But, Hattie is using the research for a very different purpose, to demonstrate the difference between expert versus experienced teachers. Hattie makes the arbitrary judgement that NBC certified teachers are 'Experienced Experts' while Non-NBC teachers are 'Experienced'. He does not use student achievement but rather arbitrary criteria as displayed in the graph below.

Podgursky (2001) in his critique, describes them as "nebulous standards". Podgursky is also rather suspicious of Hattie's rationale for not using student achievement, “It is not too much of an exaggeration to state that such measures have been cited as a cause of all of the nation’s considerable problems in educating our youth. . . . It is in their uses as measures of individual teacher effectiveness and quality that such measures are particularly inappropriate" (p2).

Hattie concludes that expert teachers (NBC) outperform Non-NBC teachers on almost every criterion (p260).


Harris and Sass (2009) report that the National Board for Professional Teaching Standards (NBPTS) who administer the NBC generate around $600 million in fees each year (p4). Harris and Sass's much larger study 'covering the universe of teachers and students in Florida for a four -year span' (p1) contradict Hattie's conclusion, 'we find relatively little support for NBC as a signal of teacher effectiveness' (p25).

It is interesting that much of Hattie's consulting work to schools involves measuring teachers on the arbitrary categories listed on the graph, a significant omission is Teacher Subject Knowledge.

Yet, using the same type of research, e.g. Hacke (2010), comparing NBC with Non-NBC teachers, he uses the low effect size to conclude that Teacher Education is a DISASTER. See Hattie's slides from his 2008 Nuthall lecture.




Hattie's Rankings are Not Helpful to Teachers:


Professor Dylan William explains the problem in 'Inside the Black Box' (2001), 
"Teachers will not take up attractive sounding ideas, albeit based on extensive research, if these are presented as general principles which leave entirely to them the task of translating them into everyday practice - their classroom lives are too busy and too fragile for this to be possible for all but an outstanding few. What they need is a variety of living examples of implementation, by teachers with whom they can identify and from whom they can both derive conviction and confidence that they can do better, and see concrete examples of what doing better means in practice" (p10).

Then commenting on research again, William says,
"despite the many and varied reports of successful innovations, they fail to give clear accounts on one or other of the important details, for example about the actual classroom methods used, or about the motivation and experience of the teachers, or about the nature of the tests used as measures of success, or about the outlooks and expectations of the pupils involved" (p12).

Averaging:

Hattie's averaging hides much of the complexity, for example, Emeritus Professor Ivan Snook, et al, on Homework:
"There is also the difficulty which arises amalgamating a large number of disparate studies. When results of many studies are averaged, the complexity of education is ignored: variables such as age, ability, gender, and subject studied are set aside. An example of this problem can be seen in Hattie’s treatment of homework: does homework improve learning or not? 

Overall, Hattie finds that the effect size of homework is 0.29. Thus a media commentator, reading a summary might justifiably report: “Hattie finds that homework does not make a difference.” When, however, we turn to the section on homework we find that, for example, the effect sizes for elementary (primary in our terms) and high schools students are 0.15 and 0.64 respectively. 

Putting it crudely, the figures suggest that homework is very important for high school students but relatively unimportant for primary school students. 

There were also significant differences in the effects of homework in mathematics (high effects) and science and social studies (both low effects). Results were high for low ability students and low for high ability students. The nature of the homework set was also influential. (pp 234-236). All these complexities are lost in an average effect size of 0.29" (p4).

Similar detail is lost in the averaging Hattie uses for class size and ability grouping.

Hattie's interpretation of the average d=0.40 representing a year's progress is TOTALLY misleading. This is obvious when the table of the USA effect size benchmarks for each year level is viewed.

Many of Hattie's researchers warn about averaging:
Mabe and West (1982) "considerable information would be lost by averaging the often widely discrepant correlations within studies" (p291).

Slavin (1990), "In pooling findings across studies, medians rather than means were used, principally to avoid giving too much weight to outliers" (p477).

Professor Maureen Hallinan. (1990)  "The fact that the studies Slavin examines show no direct effect of ability grouping on student achievement is not surprising. The studies compare mean achievement scores of classes that are ability grouped to those that are not. Since means are averages, they reveal nothing about the distribution of scores in the two kinds of classes. Ability grouping may increase the spread of test scores while leaving the mean unchanged" (p501).

Hattie attacks the easy target - The Teacher:

Hattie summarises his book “the devil in this story is not the negative, criminal, and incompetent teacher, but the average, let's get through the curricula… teacher“ (p258).

This is an amazing critique and represents Hattie's focus throughout the book. He seems oblivious to systemic and political influences and seems all too eager to focus the blame on the easy target - the teacher.

Yet Hattie says, "Educating is more than teaching people to think – it is also teaching people things that are worth learning." (p27) This is the realm of politicians and senior bureaucrats, who mostly decide what is worth learning by designing and enforcing a curriculum. So if following the curricula is the issue, why not focus on those who decide the curricular? They are most often not the teachers.

 In Hattie's jurisdiction, the state of Victoria, Australia; teachers can get dismissed for not teaching the curricula - click here for examples.

The whole may be more than the sum of the parts:

Bruce Springsteen says in his tribute to U2 using their song Vertigo - "Uno, dos, tres, catorce" - translated 1,2,3,14! - the correct maths of rock-n-roll, and maybe for classrooms too.

Also, Hattie's rankings distract us from some of the more useful teaching initiatives. For example, Professor Jo Boaler and Charles Lovitt, who focus on combining a number of influences together: problem-based learning, simulations, time on task, inquiry, and visual methods. Yet, Hattie rates these individual influences very lowly and ignores the major effect and usual classroom dynamic of combining a number of influences together.