This flight of the Falcon 1 was the first flight test of the new Merlin 1C engine. The previous version of the engine was ablatively cooled, meaning that the rocket nozzle is lined with a material that slowly burns away, thus protecting the metal nozzle. The new version is regeneratively cooled, which means that the kerosene fuel circulates through channels in the nozzle to cool it before it gets injected into the engine. With regenerative cooling, when the fuel flow is cut off there is more residual fuel in the engine, so it takes longer for the thrust to drop off to zero. Evidently, SpaceX engineers did not take this into account by lengthening the time between engine cutoff and stage separation. The small amount of remaining thrust enabled the first stage to catch up with the second stage again instead of being out of the way when it ignited. You can see the whole thing in the video posted by SpaceX.
I mentioned last week that the various newspace bloggers and industry pundits have been engaged in an orgy of speculation and criticism. The new information from Musk didn't really quiet the crowd. It just shifted the focus to a discussion of whether this problem could have been foreseen and avoided. At some level that's no doubt true. Before the release of the video, some commentators had leapt immediately to the correct explanation, knowing nothing more than that there had been a "stage separation failure" and that this was the first flight with a regeneratively cooled engine. Much has been made, as well, about the fact that SpaceX has experienced three consecutive failures.
Why did they fail? There has a lot of loose criticism of SpaceX implying they are are a bunch of amateurs. That's not consistent with what I've heard about their organization and I think it is very unfair. I don't mean to say that every mistake they have made is unavoidable. But I also think there's another reason why one would expect them to have more failures than the established aerospace companies. Their whole strategy is to find ways to simplify and automate the process of designing, building, testing and launching rockets. They are being innovative and taking risks by seeing which parts can be cheaper, which processes you can do without, which jobs you can automate, etc. If you're really going to find the floor on costs you have to go a little too low and then selectively restore some extra checks and redundancies as needed to get acceptable reliability. New, privately held, self funded companies can afford to take these kinds of risks more than established organizations.
Entrepeneurship drives progress in unique ways, and we are fortunate that we still have an entrepeneurial culture in this country. Most fail, but some succeed, and they do it by exceeding what earlier suppliers were able to achieve. If Musk is wrong, and there are no economies to be found in the way rockets are currently designed, built, tested and launched then he will fail. He might fail anyway if he makes too many mistakes. But I think his basic premise is sound and I suspect he will, in the end, succeed.