Khan Academy Vision and Social Return

Voiceover: What I want to go over in this video is what Khan Academy is in its current form and where I see it going, what I hope it will grow into. Most of you are probably already familiar with the video library that's available at Khanacademy.org. There already about 1,400 videos covering everything from basic mathematics to college level calculus, biology, chemistry, differential equations, there's even a piece on economics, but I really see it growing well beyond the video library and I've already started working on the software piece. This right here, this map structure you see, this is the Khan Academy knowledge map, (writing) Knowledge, knowledge map. and you can access it at Khanexercises.appspot.com You just need a Google ID and you can log in, it's completely free and hopefully it always will be. If you click on the knowledge map tab you'll see something like this for yourself. This shows you, essentially, the current curriculum of the software. I want to be very clear, the videos, there's 1,400 videos, far more extensive than the software piece. The software piece right now, because it frankly is more labor intensive, or since I've been the only labor, maybe I should say since it's more Sal intensive, than the video piece there's only about 70 software modules right now and this covers from basic arithmetic through about Algebra II. The idea here is to start everyone at the most basic level, a level that anyone can start at. This module right here is 1 + 1 = 2 or 2 + 3 = 5 and as they get streaks of 10 in a row right it keeps moving them up, at their own pace, to more and more advanced topics. Over here we have level 4 linear equations, systems of equations, multiplying expressions, so on and so forth. Just to get an idea of what these look like and if you log in you can just click on the modules and start working on them, but this is a screen shots of one of the modules. This is a screen shot of multiplying decimals. The paradigm that I think this type of software introduces, although it might seem very subtle, I think it's quite transformational to how we traditionally teach and learn. The traditional model is you have a lecture, you have a lecture that normally occurs while everyone's in a room together at some scheduled time. Then you have practice in the form of homework that normally goes on at home, usually on your own and if you get stuck there's not much help and you get very little feedback. The book might have every other problems answer, but you have no idea why you got it right or wrong. Then you have some type of assessment, some type of quiz. You maybe get 80 or 90% right on that quiz, you are labeled a B student or an A student and then you move on to the next topic. (writing) Next topic. And because of this structure, and it's no individuals fault, it's more mainly a bi-product of the system, everyone has to learn at the same pace, the lecture is 1 to 30 people at their pace. It doesn't matter if half the class is lost or half the class is bored, that's what everyone's getting. No matter how good of a teacher you are you have to cater to everyone, so some percentage will be lost and some percentage will be bored. Then no mater whether you get 10% wrong or 20% wrong or 30% wrong as long as you are "passing" you move on to the next topic. Anyone familiar with math or science knows that if you have some core weakenesses, if there's 10% of exponent problems you didn't know how to do, later on when you move to more advanced topics that's going to be a major hole in your foundation, it's going to make it almost impossible to master more advanced topics. So, this is the traditional model. What this software is advocating or maybe what I am advocating, is a model where all of this is happening simultaneously in a very data rich environment. You start off with instructions, that's the whole point of lectures. When you do it within this framework you're getting your instruction from the videos. (writing) You're getting your instruction from the videos. Let's say you're seeing the multiplying decimals module for the first time in your life, so you're right here, right here on the knowledge map. You've just gotten 10 in a row right on level 4 multiplication, so it is now suggesting that you work on multiplying decimals, this will be orange in your knowledge map. So you're seeing this for the first time, you have no idea how to do it, you can click on watch video and this window will pop up and you can watch one of, well in this case, the multiplying decimals video from Khanacademy.org it will be pumped directly in. Once you watch it you can start doing some practice problems. Once again the paradigm here is very different. If you don't know how to do this particular problem you can click hint and it will give you the exact steps, step by step for this problem. So, when the person, or the viewer first went here they just saw the problem and all of this highlighting and all of that, that happened after they started clicking hint. I encourage you to experiment on it, with it on Khanexercises.appspot.com. Once you start getting good at it, you don't move on after only getting 90% or 80% and it labels you as a B+ student or as an A- student. This says I want you to keep doing these multiplying decimal problems and I, the software, will keep generating multiplying decimal problems for you, even if it takes you a thousand problems until you get 10 in a row. (writing) 10 in a row. The paradigm or the idea that I'm trying to advocate is for you not to move on until you have complete proficiency in a concept. Right here on this streak as you get problems right, you can try it out, the stars fill up until you get 10 in a row. Once you get 10 in a row it will say, "Hey you know what you're doing, we think you should move on to the next concept." The other area where this is kind of ... this is the user experience, but behind the scenes here we have a super, super data rich environment. Just to give you an idea of the potential here ... Everything you're seeing, this is already built, but I think this can all be taken to even another level. This right here is data I get from YouTube. (writing) That right there is data I get from YouTube. This really is, on some level, the educational holy grail. This is the actual data on the Khan Academy balancing chemical equations videos. This is the video here, but right here YouTube shows me the attention span as someone watches videos. (writing) This is attention, this is attention. They're measuring it against the average YouTube video of this length. Let's say this is 7 minutes, so at 7 minutes into the video I'm doing pretty well, I'm still above the average. There's always a natural drop off rate, but my drop off rate is doing better and actually it improves relative to the average as we go further and further through the video. So you can imagine the potential here. You can keep recording videos and see which ones have the best audience attention. I've seen some of my videos where this graphic looks more like this. (writing) Where the graphic looks more like this. I can go back and say gee, something must have happened right there in the video, at maybe minute 8, that caused people to drop off, maybe I said the word orthogonality without defining it or I skipped a step in an equation and I can either annotate that video or I could re-record the video so that the new version does something like ... (writing) does something like that. Just off of the viewer data, off of YouTube, you can do tremendous things, things that were never possible in education before. When you couple that, when you actually couple that with the exercises, when you actually know when they watched the video, in the context of what exercise, how they did on the exercises before and after watching the videos, how many times they clicked on hint, then all of the sudden you can do far more powerful things. This right here is actual data collected for a teacher. This is a daily spreadsheet that is automatically generated for a teacher using Khan Academy at a summer camp. These are all of the students. I've blurred them out for their privacy. Each column here is one of the exercises in the Khan Academy and green means that the student has already gotten 10 on a row in that module. Purple means that they're working on it right now, but it seems like everything is good. Red means that they're working on it, but they've done an awful lot of problems without having gotten 10 in a row yet, so this might be a problem area. The model that, I'm at least advocating is, everyone works at their own pace, at home and in the classroom using the software and the teacher gets these daily reports, and it tells them, look teacher everyone is doing fine, everyone is progressing, except for these 2 kids right here are stuck on level 2 division. Why don't you take them aside, have a very focused 30 minute, 1 to 2 sessions with them kind of a tutorial with them and then move on to the next 2 students. There's a couple of students who are having trouble with level 4 subtraction, so that way the student is always being directly catered too and the teachers time is always spent directly, directly hitting pain points for the student. Just to highlight the power, at least in my mind, of this asynchronous learning. This is the actual data of that same cohort of students as they work through the summer camps. Just to understand what this chart is showing, this vertical axis right here is the modules that they completed and the horizontal axis right here is days on the summer camp. The black line is the class mean. This is saying 5 days into the camp the average student had completed 20 modules. These green lines above and below that are standard deviations below and above the mean. I've plotted some, what I consider interesting students trajectories here. I could plot all of the students here, but it would be a very messy graph. What I want to show you in particular is one student right here. On day 5 if you were to do an assessment using a traditional learning model, if you were to do an assessment on day 5 right here, you would say that this student, this student in purple, that she is more than one standard deviation below the mean. You would say that maybe she needs some type of remediation. She is a C student or a D student, maybe she shouldn't be in this class, but when you do it with an asynchronous learning model you allow that student to work at their own pace, you don't have to slow down the rest of the class for him or her, and this was a her. It turns out that she just needed a little bit extra time on negative numbers. This was a cohort of rising 8th graders. Everyone else, it was a little bit of a review for her, she needed extra time on negative numbers and we were collecting data on everything. Each of these columns represent a problem she did. This right here, this first column was the very first problem she did. She got it wrong, that's why it's red and it took her 33 or 35 seconds. The next one she got right, maybe she guessed it, or maybe it was 2 positive numbers being added and then the next two she got wrong, then she got one right, maybe she guessed, who knows. Then this right here, actually, I don't plot it, but this is actually where she watched the video, that's why it took her so long. Then after that she started to get some problems right, that's what the blue bars represent. Not only did she get some ... she got a few wrong here or there, but at the same time, the time it took her to do problems also went down, you see the height of these bars are going down. This is actually a distribution of the times it took her. As you can see, you can actually get granular date of what peoples' actual pain points are, what remediation they need without slowing anyone else down and then directly tackle that and you get all of the data on it. At the same time, once she was able to get through this, this one pain point, not only was she not below the average, but she was able to rocket forward after she got through this and she actually, at the end of ... this was actually only a month and a half, she finished at the 2nd or 3rd best student of the class. Just to give an idea, these judgements that we place on students are so arbitrary, back here you might have called her a remedial student, out here you would say wow she just rocketed straight through level 3 linear equations, which are equations of this form right here, which is really above her grade level and would say, hey she's maybe a gifted student. When you let every student work at their own pace, at their own time, not pushing people forward before they're ready or holding people back, you really do see amazing results out of students. Now, everything I've talked about so far, everything I've talked about so far, we've talked about the video library. (writing) we've talked about the video library, we've talked about the exercises. (writing) we've talked about the exercises and how that could be, the combination here, could be transformational for a classroom, everyone working at their own pace and even if the software isn't being used, there have actually been a few teachers who've already emailed me, that they've flipped the model, instead of doing lectures in the classroom and homework at home, they're now doing homework in the classroom and using the Khan Academy videos at home which makes a lot of sense because you do the videos at your own time, you can pause, repeat, look up stuff, you can review videos at your own pace, at your own time, as you need them and then when you're actually doing problems in the classroom when you're surrounded by your peers and the teacher who can help you along and really kind of give you assistance and feedback when you need it. That's already happening, even before the software, but I think when you have the software it becomes even more rich, especially with the data collection, but there is a situation of, what if one of these students were getting stuck. Let's say they're not working in the framework of a classroom that has already adopted this paradigm, what do they do, at least my view point of the answer, that you have peer-to-peer instruction. (writing) peer-to-peer, peer-to-peer instruction, peer-to-peer instruction. So you've watched the videos, we know you've watched the video, the software knows you've watched the video, you've done the exercises, we know that you've done so many that you're having a pain point here. What we can automatically do is set up a peer-to-peer instruction with some other teacher, student, tutor, it could be in the same city, it could be on the other side of the planet and we could do it virtually; the same way that I started off with my cousins where you have a pen tablet, you're doing this on a computer but you have some type of internet telephone connection whether it's Skype or Google voice or whatever else and you're using a shared whiteboard, kind of something like what I'm writing on right now, but it's live and in real time. You could actually even record. (writing) You could record the interaction. One, that's valuable from a safety point of view, but it's also valuable to have this huge library of other interactions that maybe if I can't find a tutor immediately, I can dig through the recordings of other people who've had the same problem and I can get my question answered. The power here is not only are you able to leverage all of the student body of this virtual academy to teach each other, but you also are doing it in a very data rich environment because you know what happens. Each of these, in isolation, I think, is already a huge value add, but when you do them together it becomes a super-duper value add because not only do you have the peer-to-peer instruction, you use the exercises to figure out exactly what type of peer-to-peer instruction you needed, you know who to pair someone with because you know maybe Sal is having trouble with level 1 exponents, Bill, we already know has mastered level 1 exponents. So, let's set up that interaction and then you have data on before and after. (writing) Before and after. You will even be able to assess how good was that interaction. You can have a qualitative rating from both the tutor and the tutee, I guess you could say or from the student. You can also have data on what happened before and after that interaction and is it statistically significant compared to other similar interactions. When you couple this, I really think you are approaching the holy grain of education. This right here is the virtual school that I see Khan Academy growing into. Obviously these first two pieces, this piece has been built to a large degree. I intend to continue to build it and continue to add, really all topics. This is already starting to be built, but I envision, right now it goes through Algebra II, if you look at the knowledge map. I completely see more and more of these, you can see physics problems budding off of this, you can see calculus budding off of this, you can see chemistry budding off at certain points. You can see genetics problems budding off of this, probability, you can imagine. Right now there are 70 modules, I imagine a world where you can have 500 or 5,000 modules. That piece can obviously built up, you could have other types of exercises that are complimentary, maybe emersive type exercises. Then the part that I think has a lot of value that has yet to be built is really the peer-to-peer piece, but all of the technology already exists today, I think, to do it. Just to give, I think, a hint of where I think the social value is here and I'm going to do a quick back of the envelope calculation because I think there is a huge amount of ... it seems appealing to a large percentage of people but just to get an idea of how scalable, how much leverage this type of a concept could have, or how much social value it can create. Let's just talk about the videos alone, let's not even talk about the exercises or the peer-to-peer interactions and let's just talk about the videos only in English although they are already being translated into Spanish and Tamil, and there's a group in Poland. They could very easily be translated to every major language in the world. Let's just talk about English, let's say we have about 500 million, (writing) There's about 500 million English speakers world wide. That includes, I think there's about 350 million native speakers and 150 million people who are speaking English as a second language. Let's just say, this is a back of the envelope calculation. Let's just say that 10% could potentially use these type of videos, that they are at some point in their life where they want to learn. Maybe they're teachers who want a new way to teach something or there are people going back to college, or they are your traditional K-12 or college student, so 10% of the population could stand to benefit. This is, you could kind of view this as the market, but I speak of this in kind of a social enterprise point of view, as the market, people who could benefit from it. We're already talking about 50 million, I actually think this number is much larger because you should include parents and teachers and really anyone who wants to learn anything. Let's just say 10% of the population, so you have a 50 million addressable market, and let's say that for whatever reason only 10% adopt the videos, I'm only talking about the videos here, so let's just cut that down. We're talking about 10% of the 10% addressable market of the English speaking world, which is really only 10% of the world, I think making very conservative assumptions here. Sp, 10% penetration into the market that, for whatever reason, it's free and it's easily available and probably the most extensive resource available, let's say just for assumption only 10% adopt it, we're talking about 5 million learners on a regular basis using it. Let's say that they each watch a video roughly a week. We're talking about times say 50 video a year per person, they're just watching one video a week. The data I'm seeing right now is people are on average watching a video every 3 days. Actually, some of them are watching 2, 3 videos a day, but if we average all of the unique viewers out, it looks like they're watching a video once every 2 or 3 days, so once a week, I think, is a very conservative assumption. Remember, we're only talking about the videos right now, we're not talking about even the exercises or the peer-to-peer instruction or all of that. If you multiply that out, you have 250 million video views. (writing) views per year. Khan Academy isn't there right now because it doesn't have quite that, really that mind share yet. I don't think most people are even aware that it even exists, but I'm talking about the reality where people do know it exists. You have 250 million video views a year, we're talking about only the videos, only in English, we're making some conservative assumptions. Let's say that a video view, the social value of a video view. (writing) Let's say the social value, so the social value. Let's say, I don't know, it's $1. I'm just making an assumption here. My personal opinion, I think it's far more than that. Private tutoring in any of these topics can cost anywhere from $10 to $50 and this, I think, is frankly better than private tutoring. My cousin's have even told me that I'm better on YouTube than I am live because it's a less stressful environment, they can pause, they can repeat, but let's say that the social value is $1 which I think is very conservative. Then the total social return, that just the videos, just the videos using these assumptions would produce, just in English ... (writing) The social return in this scenario would be $1 times 250 million video views a year, which would be $250 million, $250 million annually, annually. I'm not doing some type of present value where I'm looking at all of the future value created over the next 25 or 50 years, this is on an annual basis. This is using these conservative estimates. If you had translated to the top 10 languages you can multiply this number by 10. If you make this 10% into 20% you can multiple this by 2, it will be 500 million. If you present value even this number you're going to get something in the billions, depending on how you present value and your discount rate and all of that. You're going to get a number that's in excess of multiple billions of dollars. This is all, in my mind, for a very, very, very small relative investment to support all of this. Remember, the social return is calculated just from the video views. If you actually think about it replacing actual education, you compare it to what's actually spent on a per pupil basis in the developed world, a $10,000 per pupil then the numbers become really ridiculous, but if you just look at the $250 million all of this, that's the return just on the videos, but all of this could be built, I think, by a team of 5-10 highly motivated engineers. (writing) 5-10, built and maintained, motivated, 5-10 engineers including myself. So, you're really talking about an operating budget depending on the scale you do it at on the order of a million dollars a year. The videos, obviously I can keep going and I would pay myself far less than a million dollars, so just there you're getting a huge social value, but even here if you're looking at an operating budget of a million dollars a year you really are talking about something that has the impact to transform education on a world wide basis and have a social return well, well in excess of what I think of this as kind of the base case. Even if this was the best case, a million dollar a year getting a $250 million a year payback is pretty tremendous. The last thing I want to highlight is that this is all evergreen, this is all evergreen, these videos and these modules they don't have to be changed every year. Math and Science does not change that fast. Once some of this, once most of this content is built out, if I, God forbid, got hit by a bus tomorrow, this social return will still exist. It's not like as soon as these engineers disappear that all of this disappears. As long as this is maintained by some skeleton crew, you will still continue to generate this much in social return. Anyway, I want to leave you there and hopefully that gives you a sense of what I see Khan Academy, what it is today, what I see it becoming and, I think, a sense of the real value that this can have for the world.