Looking at the historical academic data (see below), the school improved around 2008-2013 then plateaued and then decreased a little in 2015 (the year Hattie was there).
The show displayed many difficult and disruptive students. Also, apparent was the passion, enthusiasm, care, patience and hard work of the teachers. The staff also displayed a desire to improve their teaching, a key mindset, according to a different expert, Prof Dylan Wiliam.
Ironically, the qualities like passion, etc are the invisible influences that Hattie argues AGAINST in his book Visible Learning. But, in the 2012 update, Visible Learning for Teachers, he states,
"Throughout Visible Learning, I constantly came across the importance of 'passion'; as a measurement person, it bothered me that it was a difficult notion to measure – particularly when it was often so obvious" (preface).At the same time Prof George Vaillant (2012), takes this a step further,
"Perhaps the greatest problem faced by the academic social sciences is that what is measurable is often irrelevant, and what is truly relevant often cannot be measured."I am concerned that Hattie seems to be given the credit for the improvement from 2008-2013, yet Hattie was only there in 2015, when the Year 12 and NAPLAN results decreased!
However, I am more concerned about Hattie's financial conflict of interest in promoting his own software (e-Astle and the Visible Classroom app) as a solution to the 'so-called' problems he identifies with public education. Professor John O'Neill (2016) details this conflict of interest.
Examples of the promotion of Hattie's App - Episode 4 - 26 minutes. Janet Clinton demonstrates the app, she also receives royalties from the app and is also Hattie's wife. Neither is divulged in the program.
Also, O'Neill wrote a timely warning in 2012, about Hattie's strategy,
The "discourse seeks to portray the public sector as ‘ineffective, unresponsive, sloppy, risk-averse and innovation-resistant’ yet at the same time it promotes celebration of public sector 'heroes' of reform and new kinds of public sector 'excellence'. Relatedly, Mintrom (2000) has written persuasively in the American context, of the way in which ‘policy entrepreneurs’ position themselves politically to champion, shape and benefit from school reform discourses" (p. 2).This particularly applies to Revolution School - Hattie diagnosis:
Hattie's comments that education in Australia is "DIRE" since it is slipping down the world rankings, directly contradicts his 2005 ACER lecture where he exhorts to politicians & school leaders,
"It’s NOT about numbers, assessment literacy, national ability tests."McKnight & Whitburn (2018) in Seven reasons to question the hegemony of Visible Learning are also concerned about Hattie's portrayal in this TV series as,
"the potential saviour of public education and redeemer of recalcitrant teachers" (pp. 2 ff).They also question the financial conflict of interest of Visible Learning,
"Where are the flows of capital around Visible Learning? Where is capital and what kinds of capital are accruing for those producing “Visible Learning” as a brand? What material and financial benefits flow on to teachers and students?" (p. 6).Professor Ewald Terhardt (2011) also comments on Hattie's conflict of interest,
"A part of the criticism on Hattie condemns his close links to the New Zealand Government and is suspicious of his own economic interests in the spread of his assessment and training programme (asTTle)." (p. 434)Compare Hattie diagnosis with 2017 Australian of Year Maths teacher - Eddie Woo:
Was there a revolution?
Nilholm (2017),
"... he [Hattie] does not give reasonable answers to how his theses are to be translated into practical work and I have definitely not seen any study that critically examined what happens when municipalities and schools try to base their work on Hattie's work" (p. 3).The Maths curriculum for all Victorian schools including Kambrya has detailed the following criteria for ALL students to achieve by the end of Year 10:
"Evaluate statistical reports in the media and other places by linking claims to displays, statistics and representative data." Mathematics Statistics and Probability Levels 7-10A.We expect our students to evaluate claims but we rarely do this ourselves, particularly with regard to educational statistics and in particular to Hattie's claims.
The program displays this graphic indicating phenomenal improvement in Year 12 achievement, click here for short video clip from TV show.
Hattie's Preponderance to Exaggerate
In Hattie's 2017 presentation @researchEd in Melbourne, Hattie states,
Kambrya was in the bottom 10% of all schools in the state, now its in the top 10% in ANY measure! (@25mins here)
However, When I checked the Actual Results I got...
These results show a reasonable improvement from 2008 to 2010, but they don't indicate the phenomenal improvement implied by the graph on TV. Also, it appeared that when Hattie consulted with the school in 2015, the results slightly decreased!
Perhaps the graph is showing the change in ranking and the School got into the top 25% of Schools. But, this is not correct either as the school ranking in 2015 was 273rd out of 528 schools. Berwick College (nearby) was ranked higher at 260th - check rankings here.
Similar spectacular English results were displayed:
But these cannot be Y12 results so they must be the Y7 and Y9 NAPLAN results. When I checked the NAPLAN results (see- https://www.myschool.edu.au) I found these (Y9 are the higher line):
These NAPLAN results do not show phenomenal improvement.
Hattie Continues to Exaggerate the Improvement of Kambrya
"I was involved in the television show ‘Revolution Schools’ where a TV crew followed a high school here in Melbourne that went from the bottom 10% to top 10% in the state." (Hattie & Larsen 2020, p. 282)Hattie is completely misleading when he says the school improved to the top 10% in the State. The School did improve from 2008 to 2011 but the key NAPLAN results show a moderate decline from 2011 & the Year 12 scores show a leveling off from 2011.
Looking at the actual Year 12 scores it would be more accurate to say the School improved from the bottom 25% of schools to the bottom 35% of schools in the state.
An improvement but not a revolution.
Comparison with other local schools:
A comparison of Y12 results with other schools nearby shows a revolution is also happening, e.g., Berwick SC is 1km away from Kambrya.
Comparison with other local schools:
A comparison of Y12 results with other schools nearby shows a revolution is also happening, e.g., Berwick SC is 1km away from Kambrya.
But, if you look at the results of other private schools in the area they achieve much higher results!
Also, Nossal High (select entry) is a few km from Kambrya & Box Hill High, seems to achieve much better results. If there is any revolution, I would be looking at these 2 schools.
The 2016 results have just been released Kambrya - Median = 29 and '% of 40+' = 2.20%
Berwick SC - Median = 29 and '% of 40+' = 3.8%
The other spectacular results are the parent, student and staff surveyed perceptions of the school. I have not got access to all that data. Small segments of the surveys are available in the School's annual reports. These brief snapshots of the surveys do show improvement in the student, parent and staff attitudes. Whilst these are not the phenomenal improvements implied by the graphs, they are still worthwhile.
So what caused these improvements? Was it the leadership? Was it the passion and other qualities fostered and demonstrated by the teachers? Was it the strategies they employed? Was it due to Hattie's involvement? Or was it something else?
Hattie was very prominent on the show. He expressed some controversial views like 'public schools perform as well as private schools' and 'your postcode does not determine your VCE score'. But, the only strategy I could identify that he was directly involved with was the recording of teacher-directed instruction and determining the % of the time the teacher spoke versus students. It seemed Hattie was coaching staff to get down to 50%.
Given Hattie's position of pushing for influences with high effect sizes (d), values of d > 0.40 and his associated mantra of 'know thy impact', it seemed contradictory to me that the other strategies used by the school were strategies which Hattie ranks very lowly. Hattie has regularly described influences where d < 0.40 as 'going backward', 'distractions' or 'disasters'!
Strategies Kambrya used and their effect size (d):
Principals / school leaders: d = 0.36
Maths teacher told to make videos: audio/video, d = 0.22
Daraby (difficult) boys class reduced to 18 kids: class-size, d = 0.21 Hattie has called a focus on a class-size a disaster!
Daraby boys taken on outdoor-ed camp: co/team teaching d = 0.19, out of school d = 0.09, extra-curricula d = 0.17
Gender seperation: Daraby boys and girls: gender d = 0.12
Psychologist brought in to teach well-being, respect: Values/moral d = 0.24 (Prof. Lea Waters)
Full-time welfare worker: Welfare d = -0.12 (NEGATIVE!!!!)
Use of restorative practice: Decreasing disruptive behavior d = 0.34
Classroom management (Dr Rogers): Decreasing disruptive behaviour d = 0.34, but, Classroom behavioral d = 0.80
Reading programs: d = 0.36 (literacy consultant Di Snowball)
Debutante ball: out of school d= 0.09, extra-curricula d = 0.17
Home visitation: d = 0.29
Mentoring: d = 0.15
From Hattie's 2008 Nuthall lecture:
The School's choice to use these strategies (classified as poor by Hattie) questions Hattie's interpretation of the research. Also, Dan Haesler in his excellent commentary about the program mentions Hattie's input would cost the school about $60,000.
In Hattie's recent collaboration with 16+ experts; the head scholar, Professor Peter Blatchford, concludes by asking for a new direction of research - the aim is move beyond the rather tired debates about whether these individual influences affect pupil performance and instead move things on by developing an understanding of the complex nature of classrooms and teaching with important practical benefits for education worldwide (p. 102).
Whilst it may be simpler and easier to see teaching as a set of discreet influences, the evidence shows that these influences interact in ways in which no-one, as yet, can quantify. It is the combining of influences in a complex way that defines the 'art' of teaching.
Math's Teaching:
Firstly, class size DOES MATTER and the evidence shows this. In addition many of the strategies I use, e.g. problem-based learning, Computer-assisted instruction, Creativity, Simulations, Inductive teaching, Inquiry-based teaching, Visual/audio-visual methods, Web-based learning, Within-class grouping, Student control over learning, kinesthetic awareness and Teacher subject matter knowledge, work for large numbers of students yet they fall below Hattie's "worthwhile" point of d = 0.40.
Other, initiatives which contradict Hattie's interpretations are the Maths 300 project, The Maths Task Centre, and Prof James Tanton. As does Boaler's work on visualisations.
We also need to spend more time promoting and congratulating good teaching, like Maths Guru, Prof Steven Strogatz tweet about an Australian maths teacher.
Private versus Public Schools:
Throughout the program, Hattie constantly proclaimed that there is no difference between private and public schools here in Victoria. The actual Year 12 results of schools show a completely different picture. Only five public schools make the top 50, but four of those are select entry via standardised tests, which demonstrates how powerful prior knowledge, home environment, socioeconomic status, and even ability grouping at a school level are.
Also, Hattie contradicts his colleague at Melbourne University, Professor Richard Teese, who in his book 'Undemocratic Schooling' presents compelling data showing private schools outperform public schools consistently in Year 12 scores. Teese also identified that schools need to offer students a broader range of pathways (e.g., vocational and apprenticeship) rather than just an academic pathway.
This snippet, from the Kambrya website, shows the improvement of the school in 2007. But, it also shows the difference compared to the top schools, which are all private apart from Mac.Robertson which is a select entry school.
The difference in achievement is significant. Perhaps Hattie means there is no difference in teacher quality? If that is the case then, what is causing the difference between the schools? It appears to be other factors besides the teacher, perhaps, home background, socioeconomic status, parent involvement, the culture of the school, selective entry, homework, ability grouping, ???
Whatever it is, this totally contradicts the main contention of Hattie's book, that it is the teacher that most influences student learning.
Victorian School Rankings:
Scores are out of 50, so the "Scores of 40+ (%)" indicate the percentage of students in that school, who get 40 or above out of 50. In other words the A+ students. This to me indicates the academic culture of the school.
These rankings indicate that the kinds of schools that are revolutionary (if focusing on academic success) are the government high schools such as Box Hill High School who get comparable scores with private schools but are able to do it with $7,000 - $30,000 less funding per student/year compared to private schools.
However, as Hattie points out schools are not just about academic results,
Many schools are trying to provide for a broad range of student outcomes. I am proud to belong to one such school who provides pathways for academic, artistic, vocational, apprenticeship and sport.
At the time of writing (June 2016) one of our students, Ben Simmons has been picked Number 1 in the NBA draft. That was his intended outcome all along, yet his success and others like it are lost in the myopic scrutiny of academic results which people like Hattie tell us to focus on.
Kambrya has these sorts of broad programs too, but that was lost in the documentary.
Other news about Kambrya.
Berwick SC - Median = 29 and '% of 40+' = 3.8%
The other spectacular results are the parent, student and staff surveyed perceptions of the school. I have not got access to all that data. Small segments of the surveys are available in the School's annual reports. These brief snapshots of the surveys do show improvement in the student, parent and staff attitudes. Whilst these are not the phenomenal improvements implied by the graphs, they are still worthwhile.
So what caused these improvements? Was it the leadership? Was it the passion and other qualities fostered and demonstrated by the teachers? Was it the strategies they employed? Was it due to Hattie's involvement? Or was it something else?
Hattie was very prominent on the show. He expressed some controversial views like 'public schools perform as well as private schools' and 'your postcode does not determine your VCE score'. But, the only strategy I could identify that he was directly involved with was the recording of teacher-directed instruction and determining the % of the time the teacher spoke versus students. It seemed Hattie was coaching staff to get down to 50%.
Given Hattie's position of pushing for influences with high effect sizes (d), values of d > 0.40 and his associated mantra of 'know thy impact', it seemed contradictory to me that the other strategies used by the school were strategies which Hattie ranks very lowly. Hattie has regularly described influences where d < 0.40 as 'going backward', 'distractions' or 'disasters'!
Strategies Kambrya used and their effect size (d):
Principals / school leaders: d = 0.36
Maths teacher told to make videos: audio/video, d = 0.22
Daraby (difficult) boys class reduced to 18 kids: class-size, d = 0.21 Hattie has called a focus on a class-size a disaster!
Daraby boys taken on outdoor-ed camp: co/team teaching d = 0.19, out of school d = 0.09, extra-curricula d = 0.17
Gender seperation: Daraby boys and girls: gender d = 0.12
Psychologist brought in to teach well-being, respect: Values/moral d = 0.24 (Prof. Lea Waters)
Full-time welfare worker: Welfare d = -0.12 (NEGATIVE!!!!)
Use of restorative practice: Decreasing disruptive behavior d = 0.34
Classroom management (Dr Rogers): Decreasing disruptive behaviour d = 0.34, but, Classroom behavioral d = 0.80
Reading programs: d = 0.36 (literacy consultant Di Snowball)
Debutante ball: out of school d= 0.09, extra-curricula d = 0.17
Home visitation: d = 0.29
Mentoring: d = 0.15
From Hattie's 2008 Nuthall lecture:
The School's choice to use these strategies (classified as poor by Hattie) questions Hattie's interpretation of the research. Also, Dan Haesler in his excellent commentary about the program mentions Hattie's input would cost the school about $60,000.
Whilst it may be simpler and easier to see teaching as a set of discreet influences, the evidence shows that these influences interact in ways in which no-one, as yet, can quantify. It is the combining of influences in a complex way that defines the 'art' of teaching.
Math's Teaching:
Firstly, class size DOES MATTER and the evidence shows this. In addition many of the strategies I use, e.g. problem-based learning, Computer-assisted instruction, Creativity, Simulations, Inductive teaching, Inquiry-based teaching, Visual/audio-visual methods, Web-based learning, Within-class grouping, Student control over learning, kinesthetic awareness and Teacher subject matter knowledge, work for large numbers of students yet they fall below Hattie's "worthwhile" point of d = 0.40.
Other, initiatives which contradict Hattie's interpretations are the Maths 300 project, The Maths Task Centre, and Prof James Tanton. As does Boaler's work on visualisations.
We also need to spend more time promoting and congratulating good teaching, like Maths Guru, Prof Steven Strogatz tweet about an Australian maths teacher.
Private versus Public Schools:
Throughout the program, Hattie constantly proclaimed that there is no difference between private and public schools here in Victoria. The actual Year 12 results of schools show a completely different picture. Only five public schools make the top 50, but four of those are select entry via standardised tests, which demonstrates how powerful prior knowledge, home environment, socioeconomic status, and even ability grouping at a school level are.
Also, Hattie contradicts his colleague at Melbourne University, Professor Richard Teese, who in his book 'Undemocratic Schooling' presents compelling data showing private schools outperform public schools consistently in Year 12 scores. Teese also identified that schools need to offer students a broader range of pathways (e.g., vocational and apprenticeship) rather than just an academic pathway.
This snippet, from the Kambrya website, shows the improvement of the school in 2007. But, it also shows the difference compared to the top schools, which are all private apart from Mac.Robertson which is a select entry school.
The difference in achievement is significant. Perhaps Hattie means there is no difference in teacher quality? If that is the case then, what is causing the difference between the schools? It appears to be other factors besides the teacher, perhaps, home background, socioeconomic status, parent involvement, the culture of the school, selective entry, homework, ability grouping, ???
Whatever it is, this totally contradicts the main contention of Hattie's book, that it is the teacher that most influences student learning.
Victorian School Rankings:
Scores are out of 50, so the "Scores of 40+ (%)" indicate the percentage of students in that school, who get 40 or above out of 50. In other words the A+ students. This to me indicates the academic culture of the school.
These rankings indicate that the kinds of schools that are revolutionary (if focusing on academic success) are the government high schools such as Box Hill High School who get comparable scores with private schools but are able to do it with $7,000 - $30,000 less funding per student/year compared to private schools.
However, as Hattie points out schools are not just about academic results,
'Of course, there are many outcomes of schooling, such as attitudes, physical outcomes, belongingness, respect, citizenship, and the love of learning. This book focuses on student achievement, and that is a limitation of this review' (p. 6).Ken Robinson, in the most viewed TED talk, 'Do Schools kill creativity?' answers YES. He asks for another type of revolution, one that values and fosters the individual talents of the student, whether they be academic, athletic, creative or otherwise.
Many schools are trying to provide for a broad range of student outcomes. I am proud to belong to one such school who provides pathways for academic, artistic, vocational, apprenticeship and sport.
At the time of writing (June 2016) one of our students, Ben Simmons has been picked Number 1 in the NBA draft. That was his intended outcome all along, yet his success and others like it are lost in the myopic scrutiny of academic results which people like Hattie tell us to focus on.
Kambrya has these sorts of broad programs too, but that was lost in the documentary.
Other news about Kambrya.
No comments:
Post a Comment