Does Khan Academy Increase Student Growth?
Thursday, June 2, 2011 at 8:16AM Yes? Maybe? I still have no idea! I think so though!
I analyzed my 5th grade students' end of year NWEA data and compared it with their Khan Academy Energy Points. Students who were in the top 1/3 of the class for Energy Points grew, on average, a whopping 2.5 years during this past school year. Those in the bottom 1/3 for Energy Points still grew, but only 0.7 years. The below graph shows this striking relationship between Khan Academy activity (in terms of Energy Points earned) and student growth.
Student growth calculated from Spring 2010 - Spring 2011 NWEA data. n = 63
Please question the methodology and validity of this data as you please. I'm interested in hearing your thoughts and plan on building this post based on the discussion via comments/emails.
Reader Comments (8)
Hi Harsh,
All you've done is shown a correlation between Energy Points and Growth, not a cause and effect. Think about it the other way, students who have grown more are therefore able to earn more energy points. Correlation, not causation.
What is the bottom third was forced to achieve the same number of energy points as the top third? Would the show the same growth?
I'd be more interested in seeing this as a scatter plot of Growth vs. Points, with each student as one dot on the graph. Is there a direct relationship? Do you have students with low energy points and high growth? Students with high energy points and low growth? What is the correlation coefficient?
I would love to see the original data.
I want to applaud your effort for being looking at data and being open about it. Thanks for sharing!
Frank
Are there other classes in your school with a similar student composition (maybe similar test scores from last year)? Would be interesting to compare overall performance of your students to theirs (although if yours outperform, it could just as well be due to having a great teacher than due to having Khan Academy). It does sound like 2.5 years of growth is dramatic by any measure. Is this type of thing even seen in other classes at all?
I don't think Harsh was actually trying to "show" anything more than that he saw growth in his student's math skills and that growth, at least in his class, had some correlation with using KA. KA is not the magic bullet, but I do believe it really does motivate students to at least try and do the math. I certainly saw that in my classes. I had students coming in at lunch and break to work on their KA. I can tell you they didn't do that when I gave the worksheets.
Thanks for the info Harsh.
Joe
Frank,
I have no data to back this up, but if the bottom third was forced to work up to the same energy points as the top third, I strongly believe they would see similar growth. At this age (9-11) the foundations of math are what significantly increases student growth. They start to skyrocket once they master multiplication, division, subtraction, and addition--which is what KA helps with--rote practice.
Similar to what Joe said, the KA exercise modules are basically worksheets on fire. They increase student motivation to practice, and there's no doubt that more practice = more growth.
Also, there's no control group because I didn't want to deprive one class of KA for two reasons: 1) I believe KA is great practice for students both low and high, so I just couldn't resist not allowing one of my classes to use KA. 2) My teacher evaluation is strongly tied to student performance, so I couldn't afford to hold back a class of 32 kids to run this experiment. Bill, I'll look into other classrooms, hopefully they will share their data and I can find another teacher whose data has been similar in the past to compare with.
I'm not at all closed to sharing the original data, I'll post as much as I can as soon as I finish end-of-year requirements (I'm moving grades, and moving rooms).
When using KA, how do you're students sign-in to the system? Do they use gmail or some other login? Thanks!
Jenn,
They use a different login. I assigned them e-mail accounts using this domain (Ex. "Tim" would log in using Tim@harshpatel.net), for the sake of privacy, and control on my end.
Harsh, what was average growth for you classes before you implemented KA? That might be a very rough way to get a figure for comparison.
I love what you're doing and this blog is very interesting, but I'll agree with Frank - the heading then "Yes?" suggest that the correlation is meant to be inferred as causal, when there is no reason to believe so. Instead of "Yes?" it should be "We have no idea"! :P
Great work though, keep it up!
Andrew,
I implemented KA after the Winter (5th Grade) NWEA. The following are the breakdowns between Fall, Winter and Spring scores.
Fall (5th Grade) to Winter (5th Grade): 0.5 grade levels
Winter (5th Grade) to Spring (5th Grade): 1.4 grade levels (Total of 1.9 from beginning of 5th to end of 5th)
Spring (4th Grade) to Spring (5th Grade): 1.7 grade levels (Total of 1.7 from end of 4th to end of 5th)
The above data also shows the summer dip. Students' scores fell between Spring (4th grade) to Fall (5th Grade).
I also edited the "Yes?", I agree with you. I meant the "Yes?" as more of a "ehhh... I think so? I hope so? Not too sure, but here look!"