25 September 2008

Revealing secrets

Filed under: APIs, compatibility, openness — David Wood @ 8:58 am

I’m increasingly looking forward to the unveiling of lots of secrets about Symbian software. The more I hear about specific ideas third party developers have for add-on and plug-in software for Symbian devices, the more I realise that their needs will be better served once the Symbian secrecy kimono is swept away.

Take one example, from my browsing of the Forum Nokia Developer Community Discussion Board. User “jas76” asks a question:

Hi, I need to mute downlink audio (i.e. from remote end) during call; and play music on music player. Remote voice is currently mixed with music and couldn’t find a way around it with any public API/SDK plugin. All discussion thread finaly result in to microphone mute. (mute mic is possible though via SDK plugin)

Is this possible from any private API that we could request? or there is no API that could achieve it? Any experince from Nokia partners in this area?

Less than 90 minutes later, there’s an answer on the Forum, from one of the Forum Nokia Experts, “symbianyucca”:

I’m rather sure there are not such an API available.

Needless to say, jas76 had been hoping for a different kind of answer. He followed up:

Hi Symbianyucca, Thanks for your inputs. So disappointing that such a simple functionality/requirement and Nokia made it not difficult but impossible. If this is because of security reason they could lock it under some capability. But making it impossible is really really disappointing.

Thanks anyways for your response. Anyone else tried to get API from Nokia for this purpose?

And the thread goes cold.

Is the answer on the Forum correct? Is the particular aspiration of the third party developer one that is reasonable, from a technical point of view? The point is, it’s really hard to tell, without gaining access to information that is currently not available to the majority of participants of this kind of forum.

By the way, before going further, I must say that I am impressed by the helpfulness, knowledge, friendliness, diligence, and upbeat humour shown on these forums by the Forum Nokia champions – who are generally providing answers on a voluntary (unpaid) basis. These individuals do a fine job, oiling the wheels of discussion, and making lots of useful suggestions, to nurture and expand a spirit of community self-help. However, there are often limits on the help that can be provided in this way. That’s because of current restrictions on distributing the source code of Symbian OS and S60. Without access to this source code, many discussions become stalled.

Take another example. Significant numbers of developers (including researchers at universities) are keen to explore the interesting possibilities of the information known to the phone about its location. The phone itself is usually in contact with several different cell towers. If applications could access this full set of information, it would allow them to determine the position of the phone more accurately than is possible by just using information from a single cell tower. However, I’m told that no APIs are available that provide this richer set of information. On the other hand, internal Nokia applications do seem to be able to benefit from this information. (Note: I understand that this information will be available to third parties in Symbian OS v9.5 – but that’s of little comfort to developers who are trying to develop location-aware applications on existing Symbian phones.)

That’s two examples. There are many more. I probably not have picked the best examples. I’m not sure about the details of each example, and I may got some of the details wrong. However, the general points seem clear:

  • Developers will in principle benefit considerably from being able to delve into currently restricted information on Symbian and S60 APIs, once the code is placed into open source;
  • This will avoid the situation where, at present, too many interesting discussion threads become stalled, and enthusiasm withers away before it has the chance to create an important innovation.

On the other hand, there are of course drawbacks to this kind of openness. There are often good reasons why internal APIs have not been publicised.

For example, as soon as they are published in official SDKs, APIs become subject to tight compatibility change control. This change control is vital, for the stability of add-on and plug-in applications that depend on these APIs. But it also restricts the freedom of internal Symbian and S60 developers to evolve and improve these APIs. Especially while APIs are at an early stage in their life, there can be compelling reasons for developers to completely change their design. So there will need to be a careful education campaign, that explains that internal APIs (and many other pieces of information which developers obtain by browsing the source code) can change without warning – and may even change between two apparently similar versions of the same phone (distinguished, perhaps, only by a minor build number)!

  • For this reason, I expect that discussion forums will collect and publicise information about the ways these internal APIs change between different phones.

Secondly, some part of the code of a phone will not be made public. Like it or not, the owners of some parts of the code may decide that the code would allow leakage of knowledge of commercially sensitive algorithms, data structures, or whatever. However, the good news is that, in the world of the Symbian Foundation, the extent of this kind of withheld code will be very much less than at present. Whereas the current bias often seems to be, “in case of any doubt, withhold”, the new bias will be, “release the code, unless there’s a very strong specific reason not to do so”.

So we can look forward to moving from a situation where developers are often constrained by lack of information, towards one in which the primary constraint will be their own ability. That’s a much better situation.

That leaves the question: how long will we need to wait? Given that it may take up to two years (from the initial SyFo announcement on 24th June) for the completion of the exercise to move Symbian platform code into open source, it may seem this is a far-off future vision. But many things will be happening before the end of that two year period. Parts of the source code will be released, in a staged manner, at various phases during 2009. We’ll be moving as fast as we can, subject to due care and attention – and subject to continuing to meet our very important “business as usual” commitments of interim deliveries of technologies and customer support.

15 September 2008

De-fragmenting the mobile operating system space

Filed under: fragmentation, openness, operators — David Wood @ 3:37 pm

One response to my previous posting, “Addressing the fragmentation of mobile operating systems“, gave me a lot to think about. The points made by this anonymous commenter deserve a considered response. Here goes:

>Don’t get me wrong, I would like to have just two operating systems in the mobile space. But I’m quite realistic and tired of hypocrites like Arun Sarin, that call for software homogeneity and then sell every OS available: please, put your money where your mouth is.

I recognise the disconnect that often seems to exist, between the strategy arms of various network operators, that say they are going to reduce the number of mobile operating systems they support, and the local operational arms, that appear to disregard that policy. However, I don’t think the disconnect is total. I am aware of several instances where an attractive new smartphone model was rejected by a network operator, on the grounds that the software platform of that smartphone wasn’t on their approved short list.

Admittedly, I suspect that if the smartphone model had been knock-out gorgeous, the strategic directive would have been trumped by local sales considerations. But seeing that the model was only “good” (and not “great”) the centrally approved policy held sway. So these policies do have some force (albeit not so strong a force as you and I might like).

>So, in your point of view, the losses caused by third parties delays and incurred by mobile manufacturers are so large, that they justify that manufacturers relinquish the short-term gains of incompatible operating systems and join their forces to reduce the number of operating systems and/or accept just one OS.

On reflection, it was unfair of me to pick on delays caused by third party software. Delays caused by malfunctions in second party software (written by the phone manufacturer) and in first party software (provided by the operating system vendor) can be equally bad. Unsuspected incompatibilities in the behaviour or functionality of lower-level software can result in lengthy debugging sessions to eventually understand why higher-level software is inexplicably failing to perform as expected on the latest build of the software platform.

My point remains that it can take a long time for a device creation software team to become truly expert in all the vagaries of the underlying software platform. I wish that weren’t the case; I wish that the underlying software platform was more transparent and predictable. Marketing representatives of all the major mobile software platforms might seek to convince people that their own particular platforms are uniquely stable and dependable. And large parts of these software platforms are, no doubt, both stable and dependable. However, because there are many millions of lines of code in the entire platform, and because there are lots of complex fast-changing inter-dependencies between different modules, the unfortunate reality is that the software platforms demand considerable attention from members of the device creation community.

For that reason, manufacturers of advanced mobile phones will put an undue penalty on themselves if they spread the attention of their development teams over several different mobile operating systems. It would be far better for them, over time, to focus on a smaller number of these complex operating systems.

>the natural solution to that problem… [is] to stop outsourcing to third parties those fundamental pieces of code and start producing internally themselves. And maybe, just maybe, that’s what Symbian Foundation is all about: tight control of every software layer

It’s true, the Symbian Foundation has the intent of providing a software platform with a greater degree of completeness that has previously been the case. The idea is that mobile phone manufacturers will receive a more self-contained package of software than in the past. They’ll have less need to source missing portions.

On the other hand, mobile phone manufacturers will still seek to differentiate their phones, by means of:

  • Replacing some of the components provided by the Symbian Foundation
  • Optimising other components, with an eye to the unique hardware and network needs of their own products
  • Adding in new low-level components, that support new services and user experience.

For this reason, I expect that the device creation marketplace will remain large and vibrant.

>Fragmentation manifests itself as a technical problem (incompatibilities between devices) but it’s not created by technical problems in itself, its real nature is political: the inability of all industry players to reach an agreement to des-fragment the mobile ecosystem and, in absence of that agreement, the power/control of one player to impose a standard

I understand the motivations of the different industry players who each participate in multiple different software foundations. We can call this “irresponsible promiscuity” and/or “a failure of industry alignment”, but it can also be seen as a pragmatic hedging of bets. For as long as it remains unclear which of several mobile operating systems will be long-term winners and which will be long-term losers, it’s understandable that industry players will loath to constrain themselves to just one or two mobile operating systems.

However, once it starts to become clear that a particular mobile OS is reaching new levels of utility, productivity, and innovation, industry players are likely to focus their minds sharply – and the political landscape will change. Even so, I don’t expect there will be just one winner. That would introduce its own problems. But I do expect that the number of mobile operating systems will significantly decrease.

>Short-term, no industry player profits from a des-fragmentation process, since it implies to lower the switching costs

Symbian has thought long and hard about the risks and issues in lowering switching costs. If we provide APIs that are more in line with those used by other platforms (for example, the APIs in our PIPS libraries), the risk is that software that runs on Symbian phones will more easily be ported to run on other devices.

At first sight, that looks like a negative outcome for Symbian. However, I think the actual outcome is positive. Partners and customers dislike being locked into a single operating system. If we make it easier for partners and customers to migrate their software assets to other platforms, it means, paradoxically, they will be happier to create software assets on our platform in the first place.

>Even Symbian, when Nokia S60 programs are not allowed to run on a S60 Samsung due to lack of certificates, is creating artificial fragmentation and switching costs: why waste resources with software compatibility when a certificate breaks it all?

I see this example as an implementation error rather than any deliberate policy. The subject of certificates is tricky, but I sincerely hope that we’ll diminish the pains they unintentionally cause. Essential fragmentation is hard enough, without “artificial fragmentation” making things worse.

>Anonymous said…

Final comment: initially I was unsure whether I wanted to reply at length to someone who didn’t give their name. I prefer open discussion to shadow discussion. However, I guess that there’s good reason for the commenter to seek anonymity here.

27 August 2008

Sympathy for the operators

Filed under: openness, operators — David Wood @ 2:51 pm

It’s not just Apple and the iPhone that are the subject of some extreme views (particularly in North America). Network operators also provoke some red-hot far-out responses. But whereas the iPhone tends to provoke unduly strong admiration, the operators tend to provoke unduly strong opprobrium.

For example, here’s some verbatim comments that came up in a private piece of research conducted in and around Silicon Valley earlier this year:

“Everyone in tech has rope burns around their necks from doing business with the carriers. They hung themselves trying to do carrier deals.”

“The operator is an adversary, not a partner.”

“The basic problem with mobile is that the operators are in the way”.

In London, the sentiment is less blatant, but it’s still present. During almost every Mobile Monday London event that I’ve attended, sooner or later some question from the audience takes a semi-joking, semi-serious pot shot at network operators, blaming them for one or other aspect of lack of openness. I find these comments uncomfortable – first, because I count many friends among employees of network operators, and second, because it seems to me that the issue is considerably more nuanced than this kind of easy scape-goating suggests.

It was for this reason that I deeply enjoyed discovering and reading the recent article “Open=Beta?” by former Qualcomm SVP Jeffrey Belk. The article started by recounting some of the the usual criticisms:

“The application community, and the Venture Community that finances them, are rightly tired of what can be perceived as a small global cadre of folks in the carrier community, as well as egregious application qualification processes, putting a chokehold on the deployment of innovation in the applications space. And the chokehold is stifling innovation and growth of the wireless data applications business”.

However, as Jeffrey goes on to say,

“But as usual, the truth is not so simple…”

“One REALLY cool company got a trial with a few operators. A small problem: Their application bricked (i.e. killed dead, dead, dead) the phones of some of the trial users. Other applications, usually written by folks that are accustomed to the massive memory and hard drives of PCs or Mac, are WAAAY too resource intensive (memory, processing power) for anything but the top tier of smart phones, and even with that tiny market, performance of the apps is often suspect. Another application (fixed now), kept a persistent data connection up between the phone and carrier network. This is a huge issue, as a lot of users still don’t have unlimited data, and if an app is sucking data without them knowing it, it could be costing them a fortune, let along putting an anchor on the data performance of the operator’s network…”

“For the operators, the simplistic reality is that the bottom line is the bottom line, and they CANNOT allow applications to either 1) raise their costs structures or 2) damage their brand/customer base, because when things go wrong with an application or phone, people blame the operator. Every piece of research I have seen (or conducted) over the past decade makes that point clear. And just why do operators need to protect their network? A few months ago, I saw a CEO of a major operator speak. In the Q&A, he mentioned as an answer to a question that it costs him a minimum of $8 per phone call to answer a support call. That’s not counting the pissed off customer factor and dilution of the brand that he’s spent billions of dollars or euros building…”

“At a recent developers conference, an operator was telling a story of a section of a city where service parameters all went to hell, cells shrinking, customers losing service. They tracked it back to an enterprise customer that had gotten permission to test a new piece of hardware, and that hardware was causing nasty network effects. Bad. Another example, not as brutal, is when several applications that, when I asked about some trial metrics, with persistence (or even no persistence but frequent network access) were impeding customers’ ability to make or receive voice calls. Very bad, again — something operators just ain’t gonna allow because when these things start to happen, customers are going to look down at their phone, look at the logo on the phone, and get pissed at their operator. And pissed off customers churn. And churn makes operators’ metrics look bad. And bad metrics make financial markets unhappy. And unhappy financial markets make operator executives lose their jobs. So it just won’t happen.”

In between the paragraphs I’ve quoted, there’s lots more interesting analysis. There’s no easy answer, Jeffrey suggests, except for some first-class development work by applications providers, who need to take the time to understand the special complications of mobile. Applications won’t be tolerated on the network, even with the label “Open”, if they are in reality only beta quality (or “pre-beta”), and risk significant network and support costs.

Is that the end of the story? Unfortunately, there is one more twist. Application developers often perceive that network operators have an additional motivation, lying behind their defensible motivation to preserve the quality of the network. That additional motivation is less defensible: it’s to block the kind of innovative services that could divert some of the network operator’s highly valued revenues to alternative services. Presumably that’s part of the reason why network operators appear to dislike open phones that support wireless VoIP.

That looks like another issue for which there’s no easy answer. It’s an issue that transcends mobile operating systems.

7 July 2008

Symbian signed and openness

Filed under: malware, openness, Symbian Foundation, Symbian Signed — David Wood @ 8:13 pm

The team at Telco2.0 have run some good conferences, and there’s much to applaud in their Manifesto. Recently, the Telco2.0 blog has run a couple of hit-and-miss pieces of analysis on the Symbian Foundation. There’s a lot of speculation in their pieces, and alas, their imagination has run a bit wild. The second of these pieces, in particular, is more “miss” than “hit”. Entitled “Symbian goes open – or does it?”, the piece goes most clearly off the rails when it starts speculating about Symbian Signed:

…the Symbian signing process doesn’t just apply to changes to Symbian itself — it applies to all applications developed for use on Symbian, at least ones that want to use a list of capabilities that can be summed up as “everything interesting or useful”. I can’t even sign code for my own personal use if it requires, say, SMS functionality. And this also affects work in other governance regimes. So if I write a Python program, which knows no such thing as code-signing and is entirely free, I can’t run it on an S60 device without submitting to Symbian’s scrutiny and gatekeeping. And you though Microsoft was an evil operating system monopolist…

This makes the Symbian signing process sound awful. But wait a minute. Isn’t there a popular book, “Mobile Python – rapid prototyping of applications on the mobile platform“, written by Jurgen Scheible and Ville Tuulos, that highlights on the contrary just how simple it is to get going with sophisticated Python applications on S60 devices? Yep. And what do we find as early as page 45 of the book? A two-line program that sends an SMS message:

import messaging
messaging.sms_send(“+14874323981″, u”Greetings from PyS60”)

I tried it. It took less than an hour to download and install the SIS files for the latest version of PyS60 from Sourceforge, and then to type in and run this program. (Of course, you change the phone number before testing the app.) Nowhere in the process is there any submitting of the newly written program “to Symbian’s scrutiny and gatekeeping”. The fanciful claims of the Telco2.0 piece are refuted in just two lines of Python.

So what’s really going on here? How is it that normally intelligent analysts and developers often commit schoolboy howlers when they start writing about Symbian Signed? (Unfortunately, the Telco2.0 writers are by no means unique in getting the Symbian Signed facts wrong.) And why, when people encounter glitches or frustrations in the implementation of Symbian Signed, are they often too ready to criticise the whole system, rather than being willing to ask what small thing they might do differently, to get things working again?

I suspect three broader factors are at work:

1. An over-casual approach to the threat of mobile malware

Symbian Signed is part of an overall system that significantly reduces the threat of mobile viruses and the like. Some developers or analysts sometimes give the impression that they think they stand immune from malware – that it’s only a problem that impacts lesser mortals, and that the whole anti-malware industry is a “cure that’s worse than the disease”. Occasionally I sympathise with this view, when I’m waiting for my desktop PC to become responsive, with its CPU cycles seemingly being consumed by excessive scanning and checking for malware. But then I remember the horrors that ensue if the defences are breached – and I remember that the disease is actually worse than the cure.

If we in the mobile industry take our eye off the security ball and allow malware to take root in mobile phones in ways similar to the sad circumstances of desktop PCs, it could produce a meltdown scenario in which end users decide in droves that the extra intelligence of smart mobile phones brings much more trouble than it’s worth. And smartphones would remain of only niche interest. For these reasons, at least the basic principles of Symbian Signed surely deserve support.

2. A distrust of the motivation of network operators or phone manufacturers

The second factor at work is a distrust of control points in the allocation of approvals for applications to have specific capabilities. People reason something like this:

  • OK, maybe some kind of testing or approvals process does makes sense
  • But I don’t trust Entity-X to do the approving – they have mixed motivations.

Entity-X could be a network operator, that may fear losing (for example) their own SMS revenues if alternative IM applications were widely installed on their phones. Or Entity-X could be a device manufacturer, like Apple, that might decide to withhold approval from third party iPhone applications that provide download music stores to compete with iTunes.

Yes, there’s a potential risk here. But there are two possible approaches to this risk:

  1. Decide that there’s no possible solution, and therefore the power of a system like Symbian Signed should be criticised and diminished
  2. Work to support more of the decision making happening in a fully transparent and independent way, outside of the influence of mixed motivations.

The second approach is what’s happening with the Symbian Foundation. The intent with the Symbian Foundation is to push into the public sphere, not only more and more of the source code of the Symbian Platform, but also as much of the decision-making as possible – including the rules and processes for approval for Symbian Signing.

Incidentally, the likely real-world alternative to a single, unified scheme for reviewing and signing applications is that there will be lots of separately run, conflicting, fragmented signing schemes. That would be a BAD outcome.

3. A belief that openness trumps security

This brings us to the final factor. I suspect that people reason as follows:

  • OK, I see the arguments for security, and (perhaps) for quality assurance of applications
  • But Symbian Signed puts an obstacle in the way of openness, and that’s a worse outcome
  • Openness is the paramount virtue, and needs to win.

As a great fan of openness, I find myself tempted by this argument from time to time. But it’s a misleading argument. Instead, freedom depends on a certain stability in the environment (including a police force and environmental inspectors). Likewise, openness depends on a basic stability and reliability in the network, in the underlying software, and in the way the ecosystem operates. Take away these environmental stability factors, and you’ll lose the ability to meaningfully create innovative new software.

The intention behind Symbian Signed to help maintain the confidence of the industry in the potential of smartphones – confidence that smartphones will deliver increasing benefits without requiring debilitating amounts of support or maintenance.

It’s true that the rules of Symbian Signed can take a bit of learning. But hey, lots of other vital pieces of social or technical infrastructure likewise take time to appreciate. In my mind, the effort is well worth it: I see Symbian Signed as part of the bedrock of meaningful openness, instead of some kind of obstacle.

« Newer Posts

Blog at WordPress.com.