dw2

29 June 2008

The five laws of fragmentation

Filed under: fragmentation, leadership, Open Source, Symbian Foundation — David Wood @ 9:42 am

As discussion of the potential for the Symbian Foundation gradually heats up, the topic of potential fragmentation of codelines keeps being raised. To try to advance that discussion, I offer five laws of fragmentation:

1. Fragmentation can have very bad consequences

Fragmentation means there’s more than one active version of a software system, and that add-on or plug-in software which works fine on one of these versions fails to work well on other versions. The bad consequences are the extra delays this causes to development projects.

Symbian saw this with the divergence between our v7.0 and v7.0s releases. (The little ‘s’ was sometimes said to stand for “special”, sometimes for “strategic”, and sometimes for “Series 60”.) UIQ phones at the time were based on our v7.0 release. However, the earliest Series 60 devices (such as the Nokia 7650 “Calypso”) had involved considerable custom modifications to the lower levels of the previous Symbian OS release, v6.1, and these turned out to be incompatible with our v7.0. As a pragmatic measure, v7.0s was created, that had all of the new technology features introduced for v7.0, but which kept application-level compatibility with v6.1.

On the one hand, v7.0s was a stunning success: it powered the Nokia 6600 “Calimero” which was by far the largest selling Symbian OS phone to that time. On the other hand, the incompatibilities between v7.0 and v7.0s caused no end of difficulties to developers of add-on or plug-in software for the phones based on these two versions:

  • The incompatibilities weren’t just at the level of UI – UIQ vs. Series 60
  • There were also incompatibilities at many lower levels of the software plumbing – including substantial differences in implementation of the “TSY” system for telephony plug-ins
  • There were even differences in the development tools that had to be used.

As a result, integration projects for new phones based on each of these releases ran into many delays and difficulties.

Symbian OS v8 was therefore designed as the “unification release”, seeking as much compatibility as possible with both of the previous branches of codeline. It made things considerably better – but some incompatibilities still remained.

As another example, I could write about the distress caused to the Symbian partner ecosystem by the big change in APIs moving from v8 to v9 (changes due mainly to the new PlatSec system for platform security). More than one very senior manager inside our customer companies subsequently urged us in very blunt language, “Don’t f****** break compatibility like that ever again!”

Looking outside the Symbian world, I note the following similar (but more polite) observation in the recent Wall Street Journal article, “Google’s Mobile-Handset Plans Are Slowed“:

Others developers cite hassles of creating programs while Android is still being completed [that is, while it is undergoing change]. One is Louis Gump, vice president of mobile for Weather Channel Interactive, which has built an Android-based mobile weather application. Overall, he says, he has been impressed by the Google software, which has enabled his company to build features such as the ability to look up the weather in a particular neighborhood.

But he says Weather Channel has had to “rewrite a few things” so far, and Google’s most recent revision of Android “is going to require some significant work,” he says.

2. Open Source makes fragmentation easier

If rule 1 was obvious (even though some open source over-enthusiasts seem to be a bit blind to it), rule 2 should be even clearer. Access to the source code for a system (along with the ability to rebuild the system) makes it easier for people to change that software system, in order to smooth their own development purposes. If the platform doesn’t meet a particular requirement of a product that is being built from that platform, hey, you can roll up your sleeves and change the platform. So the trunk platform stays on v2.0 (say) while your branch effectively defines a new version v2.0s (say). That’s one of the beauties of open source. But it can also be the prelude to fragmentation and all the pain which will ensue.

The interesting question about open source is to figure out the circumstances in which fragmentation (also known as “forking”) occurs, and when it doesn’t.

3. Fragmentation can’t be avoided simply by picking the right contract

Various license contracts for open source software specify circumstances in which changes made by users of an open source platform need to be supplied back into the platform. Different contracts specify different conditions, and this can provoke lengthy discussions. However, for the moment, I want to sidestep these discussions and point out that contractual obligations, by themselves, cannot cure all fragmentation tendencies:

  • Even when users of a platform are obligated to return their changes to the platform, and do so, it’s no guarantee that the platform maintainers will adopt these changes
  • The platform maintainers may dislike the changes made by a particular user, and reject them
  • Although a set of changes may make good sense for one set of users, they may involve compromises or optimisations that would be unacceptable to other users of the platform
  • Reasons for divergence might include use of different hardware, running on different networks, the need to support specific add-on software, and so on.

4. The best guarantee against platform fragmentation is powerful platform leadership

Platform fragmentation has some similarities with broader examples of fragmentation. What makes some groups of people pull together for productive collaboration, whereas in other groups, people diverge following their own individual agendas? All societies need both cooperation and competition, but when does the balance tilt too far towards competition?

A portion of the answer is the culture of the society – as reflected in part in its legal framework. But another big portion of the answer is in the quality of the leadership shown in a society. Do people in the group believe that the leaders of the group can be relied on, to keep on “doing the right thing”? Or are the leaders seen as potentially misguided or incompetent?

Turning back to software, users of a platform will be likely to stick with the platform (rather than forking it in any significant way) if they have confidence that the people maintaining the trunk of the platform are:

  1. well-motivated, for the sake of the ecosystem as a whole
  2. competent at quickly and regularly making valuable new high quality releases that (again) meet the needs of the ecosystem as a whole.

Both the “character” (point 1) and the “competence” (point 2) are important here. As Stephen Covey (both father and son) have repeatedly emphasised, you can’t get good trust without having both good character and good competence.

5. The less mature the platform the more likely it will be to fragment, especially if there’s a diverse customer base

If a platform is undergoing significant change, users can reason that it’s unlikely to coalese any time soon into a viable new release, and they’ll be more inclined to carry on working with their own side version of the platform, rather than waiting for what could be a long time for the evolving trunk of the platform to meet their own particular needs.

This tendency is increased if there are diverse customers, who each have their own differing expectations and demands for the still-immature software platform.

In contrast, if the core of the platform is rock-solid, and changes are being carefully controlled to well-defined areas within the platform, customers will be more likely to want to align their changes with the platform, rather than working independently. Customers will reason that:

  • The platform is likely to issue a series of valuable updates, over the months and years ahead
  • If I diverge from the platform, it will probably be hard, later on, to merge the new platform release material into my own fork
  • That is, if I diverge from the platform, I may gain short-term benefit, but then I’ll likely miss out on all the good innovation that subsequent platform releases will contain
  • So I’d better work closely with the developers of the trunk of the platform, rather than allowing my team to diverge from it.

Footnote: Personally I see the Symbian Foundation codeline to be considerably more mature (tried and tested in numerous successful smartphones) than the codeline in any roughly similar mobile phone oriented Linux-based foundation. That’s why I expect that the Symbian Foundation codeline will fall under less fragmentation pressure. I also believe that Symbian’s well-established software development processes (such as careful roadmap management, compatibility management, system architecture review, modular design, overnight builds, peer reviews, and systematic and extensive regression testing) are set to transfer smoothly into this new and exciting world, maintaining our track record of predictable high-quality releases – further lessening the risks of fragmentation.

Advertisements

2 Comments »

  1. “…Personally I see the Symbian Foundation codeline to be considerably more mature … than the codeline in any roughly similar mobile phone oriented Linux-based foundation. That’s why I expect that the Symbian Foundation codeline will fall under less fragmentation pressure …”

    I agree with that Symbian has always shipped quality code. You can be proud of that, really. However, you will now have to integrate other parties’ work, too, like Nokia, UIQ, etc. that will eventually dilute the quality of your work, I’m afraid. And since you won’t own the code anymore (will you?) you can’t do much about it – it WILL happen. I still hope, though, that the new code will still be very robust, well-designed and less prone to be fragmented than your competitors’ code. You have responsibility on assuring that.

    Comment by Gábor Török — 30 June 2008 @ 1:28 pm

  2. “I agree with that Symbian has always shipped quality code. You can be proud of that, really. However, you will now have to integrate other parties’ work, too, like Nokia, UIQ, etc. that will eventually dilute the quality of your work, I’m afraid. And since you won’t own the code anymore (will you?) you can’t do much about it – it WILL happen”

    Good comment! But the intention is for the Symbian Foundation to continue to have high quality criteria: software won’t be admitted into the Foundation Codeline (no matter where it comes from) unless it meets various submission guidelines.

    That’s what good integration management is all about – you need to be a very tough gatekeeper, not letting in any software, regardless of the reputation or the seniority of the submitter, unless the rules are followed. To break these rules may give you short term peace, but it will give you long term misery.

    Comment by David Wood — 30 June 2008 @ 10:11 pm


RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Blog at WordPress.com.

%d bloggers like this: