10 February 2010

The mobile multitasking advantage

Filed under: Android, applications, architecture, iPhone, multitasking, Psion, universities — David Wood @ 11:48 am

How important is it for a mobile device to support background multitasking?

Specifically, how important is it that users can install, onto the device, applications which will continue to run well in background whilst the user is simultaneously using the device for another purpose?

Humans are multitasking creatures.  We get involved in many activities simultaneously: listening to music, browsing the web, holding conversations, taking notes, staying on the alert for interruptions… – so shouldn’t our mobile devices support this model of working?

One argument is that this feature is not important.  That’s because the Apple iPhone fails to offer it, and the sales of the iPhone don’t seem to have suffered as a result.  The applications built into the iPhone continue to operate in background, but downloaded apps don’t.  iPhone apps continue to sell well.  Conclusion: mobile multitasking has little importance in the real world.  Right?

But that’s a weak argument.  Customer sentiment can change.  If users start talking about use cases which the iPhone fails to support – and which other smartphones support well – then public perception of the fitness of the iPhone system software could suffer a significant downturn.  (“iPhone apps – they’re so 2009…”)

How about Android?  That offers background multitasking.  But does it do it well?

My former colleague Brendan Donegan has been putting an Android phone to serious use, and has noticed some problems in how it works.  He has reported his findings in a series of tweets:

I say, with all honesty that Android’s multitasking is a huge travesty. Doesn’t even deserve to be called that

Poor prioritisation of tasks. Exemplar use-case – Spotify [music playing app] + camera

Spotify will jitter and the photo will be taken out of sync with flash, giving a whited out image

Symbian of course handles the same use case flawlessly

Android really is just not up to doing more than one ‘intensive’ task at a time

Even the [built-in] Android music player skips when taking a photo

(Brendan has some positive remarks about his Android phone too, by the way.)

Mark Wilcox suggests a diagnosis:

sounds like the non-real-time, high interrupt latency on Linux is causing some problems in multimedia use cases

Personally, I find this discussion fascinating – on both an architecture level and a usability level.  I see a whole series of questions that need answers:

  1. Are these results applicable just to one Android phone, or are they intrinsic to the whole platform?
  2. Could these problems be fixed by fairly simple software modifications, or are they more deeply rooted?
  3. How do other mobile platforms handle similar use cases?  What about feature phone platforms?
  4. How important is the use case of playing music in background, while taking a photograph?  Are there other use cases that could come to be seen as more significant?

Perhaps this is a good topic for a university research project.  Any takers?

(Related to this, it would be interesting to know more about the background processing abilities of modern feature phones.  For example, it used to be the case that some feature phones would discard the contents of partially written text messages if there was an incoming voice call.  Has anyone looked into this recently?)

Regardless of the merits of these particular use cases, I am convinced that software responsiveness is important.  If the software system is tied up attending to task A when I want it to do task B, I’m frustrated.  I don’t think I’m alone in this feeling.

My 1990’s Psion PDA typically runs more than a dozen apps in parallel (several word processors, spreadsheeets, databases, plus an individual agenda, tube map app, calculator, and so on) and switches instantly between them.  That sets my baseline expectation.

Here’s another mobile use case that’s on my mind a lot these days.  It applies, not to a PDA or mobile phone, but to my laptop.  It’s not (I think) a device problem, but a wider system problem, involving network connectivity:

  • I frequently find myself in mobile situations where I’m browsing websites on my laptop (for example, on the train), and the pages take ages to load;
  • The signal indicator on the built-in wireless modem app says there’s a strong signal, but for some reason, wireless traffic is squeezed;
  • I sit watching empty tabs on my Firefox browser, waiting and waiting and waiting for content to appear;
  • In frustration, I’ll often open another tab, and try to visit the BBC website – to rule out the possibility that the server for the other web pages(s) has gone down – but that gives me another blank page;
  • Eventually, things recover, but in the meantime, I’ve been left twiddling my thumbs.

When I switch to a WiFi connection instead of a cellular connection, things are usually better – though I’ve had the same bitter experience with some WiFi hotspots too (for example, in some Starbucks coffee shops).

So what should the highest priority be for system architects to optimise?  Responsiveness comes high on my own wishlist.  I recognise that this will often require changes in several parts of the software system.

2 February 2010

Cutting edge computing science research for the real-world

Filed under: architecture, computer science, Google, universities — David Wood @ 11:54 pm

It is an amazing time in computer science.

This is a field that, while it is about 50 years old, has more opportunity today than it has ever had, by a large factor.

These were among the opening statements made by Google’s VP of Research & Special Initiatives, Alfred Z. Spector, in a colloquium a couple of weeks ago to the Computer Science & Engineering faculty of the University of Washington.  A video of the presentation is available from the University of Washington CSE website.

I previously mentioned this video at the tail end of a previous blogpost, “In praise of hybrid AI“.  The video is full of interesting comments about the direction of computer science.

As context, Spector mentioned “four application areas in flux today“:

  • publishing, education, healthcare, and government.

He also mentioned three “systems areas evolving“:

  • ubiquitous high performance networking, distributed computing, and new end-user devices.

This provided a prelude to “three truly big results brewing“:

  1. Totally transparent processing
  2. Ideal distributed computing
  3. Hybrid, not Artificial Intelligence

It’s worth highlighting some points about each of these “big results”.  In all cases, Google seek to follow a quantitative approach, looking at large sets of data, and checking results as systems are incrementally changed.  As Spector said, “more data is better…”

1. Totally transparent processing

Spector spelt out a vision encompassing each of four dimensions: processing should be “effectively transparent”:

  • Across all types of end-user access devices,
  • across all human languages (both formal and informal),
  • across all the modes of  information (eg text, images, audio, video, sensor data, maps, timelines),
  • and across every body of knowledge (both online and offline).

In this vision:

  • There should be “no dependence or occlusions because something has got in the way” or is in the wrong format;
  • There should be “fluidity across all these forms”.

Some subsets of this grand vision include solving “voice to text”, “image recognition”, “find similar images”, and “language translation”.  Spector claimed that progress was being made across many of these sub-problems.

2. Ideal distributed computing

Spector pointed out that

Distributed computing is 30 years old, but, not very deeply understood until recently;

There was a limitation of understanding of (truly) large-scale, open integrated distributed systems.

Particular aspects of distributed systems that had not been deeply understood included:

  • Requirements for systems in which the application needs (and APIs) are not known in advance;
  • Systems with 10^6 or even 10^7 processes, with consequent enormous complexity.

Spector claimed that – as in the case of transparent processing – “there has been lots of incremental progress done with distributed systems, picking away at problem areas”.

Improvements that can be expected for huge distributed systems of computers, arising from computer science research, include:

  • Online system optimisation;
  • Data checking – verifying consistency and validating data/config files;
  • Dynamic repair – eg find the closest feasible solution after an incident (computer broke down);
  • Better efficiency in energy usage of these systems;
  • Improvements in managing security and privacy.

3. Hybrid, not Artificial Intelligence

Hybrid intelligence is like an extension of distributed computing: people become part of the system that works out the answers.

Spector said that Google’s approach was:

To see if some problem can be solved by people and computers working together.

As a familiar example, Search doesn’t try to offer the user only the one best result.  It provides a set of results, and relies on the user picking answers from the list generated by the computer.

Hybrid intelligence can be contrasted with AI (artificial intelligence):

  • AI aims at creating computers as capable as people, often in very broad problem domains.  While progress has been made, this has turned out to be very challenging;
  • Instead, it has proven more useful for computers to extend the capability of people, not in isolation, and to focus on more specific problem areas.

Computer systems can learn from feedback from users, with powerful virtuous circles.  Spector said that aggregation of user responses has proven extremely valuable in learning, such as:

  • feedback in ranking of results, or in prioritising spelling correction options;
  • semi-supervised image content analysis / speech recognition / etc.

(This idea has evolved over time, and was previously known as “The combination hypothesis”: computers would become smarter if different methods of learning can be combined.  See for example the 2003 article “IBM aims to get smart about AI” from a time when Spector worked at IBM.  It’s good to see this idea bearing more and more fruit.)

Selected questions and answers

A couple of the questions raised by the audience at the end of the lecture were particularly interesting.

One questioner asked if Google’s guidelines for research projects specified any “no-go areas” that should be avoided.  Spector answered:

No one wants a creepy computer.  So the rule is … “don’t be creepy”.

(Which is an unusual twist on “don’t be evil”!)

Spelling this out in more detail:

  • Google aim to apply extremely neutral algorithms to ranking and crawling;
  • They want systems that are very responsive to users’ needs, without being in any way creepy;
  • Views on “what is creepy” may change over time (and may be different in different parts of the world).

A second audience member asked if there are risks to pursuing a quantitative, evolutionary approach to computer science problems.  Spector answered:

  • Yes, the research might get stuck in a local maximum;
  • So you can’t do everything “strictly by the numbers”.  But having the numbers available is a great guide.

Ongoing research

As I viewed this video, part of my brain told me that perhaps I should return to an academic life, in the midst of a computer science faculty somewhere in the world.

I share Spector’s conclusion:

It’s a time of unprecedented diversity and fertility in computer science – and amazing challenges abound;

The results from computer science should continue to make the world a better place.

Spector pointed out that key research challenges are published on the Google Research Blog.  Examples he listed included:

  • increasingly fluid partnership between people and computation;
  • fundamental changes in the methods of science;
  • rethinking the database;
  • CS+X, for all X (how Computer Science, CS, can assist and even transform other fields of study, X);
  • computing with ultra-low power (eg just ambient light as a power source).

Stop press: Google’s Focused Research Awards

Coincidentally, I see that Google have today made a new announcement about their support for research in specific areas of computer science, at a small number of universities worldwide.  The four areas of research are:

  • Machine Learning
  • Use of mobile phones as data collection devices for public health and environment monitoring
  • Energy efficiency in computing
  • Privacy.

It looks like “energy efficiency in computing” is receiving the largest amount of funding.  I think that’s probably the right choice!

22 October 2008

Winners of Symbian student essay contest

Filed under: Essay contest, Smartphone Show, universities — David Wood @ 6:25 am

At the Smartphone Show yesterday, Symbian announced the results of our first Student Essay Contest, and called for entrants to a new contest – with an entry submission deadline of 31st January 2009.

The theme for the 2008 contest was “The next wave of smartphone innovation“. The prize winners are as follows (listed in alphabetical order of surname):

  • Benoît Delville, Ecole Centrale de Lille, France: The hardware tech of smartphones. Benoît’s essay examines four factors which threaten to prevent the fuller adoption of smartphones.
  • Alexander Erifiu, University of Applied Sciences, Hagenberg, Austria: New interaction concepts in mobile games. Alexander’s essay describes a project the author carried out with some colleagues to increase the suitability of smartphones for certain types of games.
  • Andreas Jakl, Johannes Kepler University, Linz, Austria: Optical translator: word spotting and tracking on smartphones. Andreas’s essay considers some developments that will enable advanced new applications that take advantage of the high quality camera technology that is currently widely available on smartphones.
  • Florian Lettner, University of Applied Sciences, Hagenberg, Austria: Smartphones in home automation. Florian’s essay investigates the possible use of smartphones in a number of practical situations, including several in the home.
  • Pankaj Nathani, Bhavnagar University, Gujarat, India: Improved development and delivery methodologies. Pankaj’s essay focuses on the fact that developers can face many challenges in developing and delivering novel or evolved services on smartphones.
  • Milen Nikolov, The College at Brockport, State University of New York, Brockport, USA: Exploiting social and mobile ad hoc networking to achieve ubiquitous connectivity. Milen’s essay examines a particular example of what is known as a ‘Mobile Ad hoc Network’ (MANET) involving smartphones.
  • Aleksandra Reiss, Petrozavodsk State University, Russia: The next waves of smartphone innovation. Aleksandra’s essay is targeted at discovering what new functionality can be added to smartphones in the near future.
  • Sudeep Sundaram, University of Bristol, UK: Situation aware maintenance mate. Sudeep’s essay reviews possible uses of a smartphone in coordination with a head mounted display, where for example, a user could see the positioning of electrical wires in a wall and carry out diagnostics.
  • Iftekhar Ul Karim, BRAC University, Dhaka, Bangladesh: Opportunities with smartphone technologies for the base of the pyramid. Iftekhar’s essay challenges readers to consider novel uses of smartphones for users in the so-called ‘base of the pyramid’ – the four billion poorest people on the planet.
  • Alejandro Vicente-Grabovetsky, University of Cambridge, UK: The smartphone of the future: A powerhouse or a mere terminal? Alejandro’s essay explores the potential for the smartphone to act as a ‘social computer’ as opposed to merely copying features from the ‘personal computer’.

My congratulations to the prizewinners! There are thought-provoking elements in all of the winning essays. For extracts and summaries, see developer.symbian.com/essays.

The contest received many other essays that also contained interesting and valuable observations. My recommendations to entrants of future contests is that that essays are more likely to be awarded prizes if they:

  • Concentrate on making a small number of points well, rather than on trying to cover a large number of different points;
  • Address specific issues, rather than describing abstract theories;
  • Have a clear structure and a logical flow of argument;
  • Back up their claims by providing evidence (for example, references).

My goal for the 2008 contest were threefold:

  1. To encourage university students to carry out research on topics of interest to Symbian, its wider community and the mobile industry;
  2. To find out where the most interesting research was being carried out;
  3. To stimulate interest in Symbian’s emerging University Research Relations programme.

Following the success of our 2008 contest, we’re repeating it in 2009. The deadline for submission for the next Symbian Student Essay Contest is 31st January 2009. The overall theme for this new contest is “Architectures to enable breakthroughs for mobile converged devices.” Students are encouraged to address one or more of the following topics in their essays:

  1. Software development that takes best advantage of multiple processor cores
  2. Allocation of responsibilities between managed code and native code
  3. Delivering maximum power from the hardware and the networks to applications
  4. Security and privacy concerns in mobile device architectures
  5. Taming the complexity of mobile system architecture: the role of open source
  6. Enabling devices, applications and services that appeal to huge new groups of users
  7. The role of system architecture in significantly improving consumer experience.

Winners of the 2009 contest will receive £1,000 with runners up earning special commendations. For the rules of this contest, see www.symbian.com/universities.

10 July 2008

Inspiring the rising stars in universities

Filed under: collaboration, Essay contest, universities — David Wood @ 11:28 pm

One of the goals I set myself for 2008 involves influencing university research departments around the world to become more active in the areas of smartphones and Symbian OS.

With that goal in my mind, I decided to accept an invite to the “Wireless 2.0” conference organised by Silicon South West, here in Bristol, where I’ve travelled for the event. I decided to attend because of the mix of both industry and university attendees.

The event hosted a “Rising Star Awards Dinner” this evening, where six university students studying electrical engineering (or a related degree) received special awards – a plaque and a handy amount of spending money. There was one winner from each of the six universities in the area covered by Silicon South West: Bath, Bournemouth, Bristol, Exeter, Plymouth, and West of England. It was heart warming to hear the personal testimonies of the winners (and their university tutors).

But links between commercial research departments and university research departments aren’t always so rosy. Universities and industry have many overlapping interests, but also some conflicting cultures. I see Symbian as having had mixed success, historically, in relations with universities:

  • On the clearly positive side, we’ve run good graduate recruitment and induction programs, every year since 1993 (that was in the Psion days, pre-Symbian); these have gone from strength to strength.
  • On the increasingly positive side, 58 universities have enrolled into the Symbian Academy program, in which Symbian supports university lecturers to deliver academic courses on Symbian OS software development.
  • On the “could do better” side, there are still only a small number of truly productive ongoing research collaborations between Symbian and individual universities, in which findings from university research projects regularly feed into Symbian’s roadmap (and vice versa).

It turns out that it’s not just Symbian that feels somewhat uncomfortable about the limited benefits realised from attempted collaboration with universities. Other commercial companies have noted similar concerns. And this has even become a field of academic study in its own right, known as (amongst other names) UIC, meaning University-Industry Collaboration. My friend Joel West of San Jose State University recently attended a two-conference on UIC at University of California, Irvine, and wrote up his observations. There’s lots to ponder there. For example, Joel described three pieces of advice on successful UIC negotiations, as given in a presentation by UIDP executive director Anthony Boccanfuso:

  1. A successful UI collaboration should support the mission of each partner. Any effort in conflict with the mission of either partner will fail. (Joel’s translation: all deals must be win-win)
  2. Institutional practices and national resources should focus on fostering appropriate long term partnerships between universities and industry. (It’s more than just the money)
  3. Universities and industry should focus on the benefits to each party that will result from collaborations by streamlining negotiations to ensure timely conduct of the research and the development of the research findings. (There is a finite window for commercialization)

With Symbian research projects, one additional hiccup has been the difficulties in allowing universities access to Symbian OS source code. Time and again we’ve been discussing an attractive-sounding joint research project with a university, when we’ve realised that the project would need more visibility of Symbian source code than was possible under the existing licensing rules. And that’s constrained the kinds of projects we can consider. (This realisation was just one of many that led to an increasing desire inside the Symbian ecosystem to find ways to liberalise access to our source code – and thus helped to set the scene for the mega-decision to embrace open source principles.)

However, not all research requires close access to source code. With that thought in mind, Symbian Research decided a few weeks back to launch the Symbian Student Essay Contest. This involves students writing an essay of no more than eight pages on the general topic “The next wave of smartphone innovation – issues and opportunities with smartphone technologies“. Up to ten students will receive a prize of UKP 1000. (See here for the contest rules.)

This prize contest has some common principles with the Silicon South West “Rising Star Awards”:

  • We’re seeking to encourage and reward individual students who show particular insight into this ever-more important set of ideas
  • We’re also seeking to inspire individual universities to give a higher priority to this domain of study.

High quality essays from a university will indicate to Symbian that there is good smartphone expertise in that university. That’s something we’re particularly interested to find out, since Symbian Research needs to decide which universities worldwide should receive higher priority attention for future collaborative research projects. That’s a tough decision to make.

Footnote: At tonight’s dinner, Prof Joe McGeehan of the University of Bristol mentioned that wise heads had been advising him, ever since 1973, that “there’s no future in research in wireless communications”. Thankfully, he persistently ignored these skeptics, and the field has indeed grown and grown. There’s now an impressive list of local south-west companies that have world-beating wireless technologies. I’m looking forward to hearing, tomorrow, what they have to say. The future of smartphones is, of course, a big part in “wireless 2.0”, but there’s lots more going on at the same time.

Blog at WordPress.com.