The hard choice between HEAD and stable. Back to 4.6. (again)

Untill today I run only one site on a recent HEAD of drupal. And I am not at all happy with how things go there. This, I have to stress, is not because of certain features, or people, its because, I, (as a part of “we”) made some wrong decisions, that got us in a spiral of issues, everytime more issues, and so it got us stuck with a heavily delayed release. The problem is mostly that I have to meet deadlines. That I have to make plannings and quotes based on estimated amount of hours. And I have found that this is near to impossible when you want to be involved in development for HEAD. The “things” you might encounter are too uncertain. The severety of the open bugs is too uncertain, to start building a tightly budgetted project on top of.

One of my main concerns is the planning part. The one site I run on HEAD is now delayed about 4 months. Luckily the client did not really bother about that delay, but for me it feels a bit like I did not manage to get a good project off the ground. When one delivers software projects, or any project in fact (the same goes for a car mechanic, I am sure) you need certain security when it comes to planning, amount of involved hours and so on. Therefore I have to choose for a stable release. The problem that you see now, in Drupal, a very, very long slip in release date, is partly because of the ever growing bug queue. That growing bug queue, is there, because no one seems to find enough time to structurally fix bugs. And no-one finds that time, because all the accounted and payed jobs are being developed on a stable release. The most open source projects see this problem happening. So there are a few basic tricks you will see in all the bigger OSS projects:* Tight release cycles. Release quick, release often, release fast. Always.* Tight feature, bug and freeze schedules. Never, ever, introduce any big changes at the end of a cycle. Never! If that means lesser security, then most often you will see temporary solutions, so that the schedule can still be met. (KDE has an assigned person to manage this schedule, he or she can strip off features, or complete packages if needed).* Decide on fast following dot releases. Instead of ironing out all the issues, or instead of introducing big changes in the last minute, they often decide to go live with version x.0 today, and tomorrow already release x.0.1 and a week later x.0.2.* Make sure you have resources at hand. So never introduce big changes, if they cannot be followed up. If you trust too much on the idea that “people will most probably help” you are introducing uncertainty. We might get stuck with too many parts not working. Wich results in too many people reverting back to the last working version. Wich results in far less resources working on solving the problems.

So, after looking at KDE, Debian, Ubuntu and OpenOffice, I can only conclude that we, at Drupal have violated a few of the basic “never do’s”. And are thus now stuck with a big issue queue, and a release that moves further away every week, instead of getting closer. I fear that more people will have to make the choice I had to make a few times: revert back to 4.6 and finish another project without being able to contribute to a faster release of 4.7.

What we should have done, (its now too late, but this might be a lesson for the future) is freeze and relaesed before the Form API went in. Then release 4.8 after it is in and ironed out. And security is not a valid excuse. Because every day 4.7 is not yet released, the introduced security in 4.7 is not helping anyone either!I fear, that if we do not manage to break this circle, it might take months before we release. And even then, it will take months before enough contributed modules and themes are converted. And even then, I think people will rather wait a few months more for the next release. I fear, that if we do not find a way to move forward faster we might start loosing a lot of developers, along the route.

And obviously I have no clue how to solve this circle, otherwise I would be doing that now. All I know is that as of today, I am developing two more 4.6 projects. And that we are going to release our hosting plan on 4.6 too.