Piagetian Programs

Hattie used ONE meta-analysis for the second highest ranked influence with d=1.28 (Hattie's Rank=2)

Jordan, V & Brownlee, L., (1981) Meta-analysis of the relationship between Piagetian and school achievement tests. Presented to American Educational Research Association, Los Angeles, CA.

I have not been able to get a full copy of the paper but was able to get a copy of the summary:
'The relationship between Piagetian and school achievement tests was examined through a meta-analysis of correlational data between tests in these domains. Highlighted is the extent to which performance on Piagetian tasks was related to achievement in these areas. The average age for the subjects used in the analysis was 88 months, the average IQ was 107. Mathematics and reading tests were administered. Averaged correlations indicated that Piagetian tests account for approximately 29% of variance in mathematics achievement and 16% of variance in reading achievement. Piagetian tests were more highly correlated with achievement than with intelligence tests. One implication might be the use of Piagetian tests as a diagnostic aid for children experiencing difficulties in mathematics or reading.'
So once again a proper experiment is not used but rather a correlational analysis which is then converted by Hattie into an effect size.

Professor Pierre-Jérôme Bergeron highlights the issue:
'Hattie confounds correlation and causality when seeking to reduce everything to an effect size. Depending on the context, and on a case by case basis, it can be possible to go from a correlation to Cohen’s d (Borenstein et al., 2009):
but we absolutely need to know in which mathematical space the data is located in order to go from one scale to another. This formula is extremely hazardous to use since it quickly explodes when correlations lean towards 1 and it also gives relatively strong effects for weak correlations. A correlation of .196 is sufficient to reach the zone of desired effect in Visible Learning. In a simple linear regression model, this translates to 3.85% of the variability explained by the model for 96.15% of the unexplained random noise, therefore a very weak impact in reality. It is with this formula that Hattie obtains, among others, his effect of creativity on academic success (Kim, 2005), which is in fact a correlation between IQ test results and creativity tests. It is also with correlations that he obtains the so-called effect of self-reported grades, the strongest effect in the original version of Visible Learning. However, this turns out to be a set of correlations between reported grades and actual grades, a set which does not measure whatsoever the increase of academic success between groups who use self-reported grades and groups who do not conduct this type of self-examination.'
I've created an example of the problem with correlation here using a class of 10 students.

NOTE: Correlation studies do not satisfy The U.S. Department of Education's design or quality criterion.

Mike Bell, who runs the Evidence-Based Teacher Network (very pro-Hattie) says :
"the research Hattie cites only refers to the fact that if you assess students using a Piagetian test of thinking level (not on the topic), and then give them a subject/topic test, there is a close correlation. It does not refer to the effectiveness of a Piagetian program."
From docendo
"I have made a few enquiries and will update this post if I get hold of the full text but it seems quite close to my assumption that it’s about a correlation between tests of Piagetian stages and achievement. I don’t think that’s of any direct use since it doesn’t tell us anything about how we accelerate progression through the stages."
Once again there are significant doubts about the validity of this meta-analysis and therefore the high effect size. It does not appear to measure what Hattie implies it measures.

VERDICT: the quality of evidence Hattie presents for this influence is very poor. I do not believe it is valid to make the claims that Hattie does. So will this advance a child's achievement by 3 years? I very much DOUBT it!

No comments:

Post a Comment