1. Can search predict the future?

    Posted December 23, 2009 in research, strategy  |  2 Comments so far

    Today, we often  search for information about upcoming major events in our lives – both good and bad – before we experience them. When facing financial difficulty or unemployment, many of us will go online at the earliest opportunity to look for help and guidance. And when we’re considering major financial decisions such as buying a house, search engines are usually consulted before estate agents are called.

    Traditional economic reports, on the other hand, look at events that have taken place. Unemployment figures tell us how many people are claiming benefits rather than how many people have been put at risk of redundancy. Average house prices are based on completed transactions, not how many people are currently looking to buy. So while we can be fairly confident of these reports, they don’t provide us with particularly current insights.

    This trade-off between confidence and currency was, in the past, largely academic as analysing current data was almost impossible. But in the age of the real-time web, this might be about to change: maybe patterns in search behaviour can give us a glimpse of future patterns in the economy.

    We first became interested in this topic back in spring 2009, so we analysed search patterns for two sets of keywords as the UK economy went into recession. We looked for relationships between these search patterns and related economic indicators, and listed some tentative predictions based on what we observed.

    House prices

    In April 2009, we looked at volumes for 23 keywords that homebuyers might use, including buying a home, cheap mortgage and mortgage providers. UK search volumes for these keywords were then compared to house prices.

    House prices charted against search volumes for 23 related keywords, from January 2004 to April 2009. Sources: Nationwide, Google Insights

    Searches typically decline as autumn ends before rebounding in January. But in 2008, the January rebound was lacklustre and the decline came in spring – much earlier than usual. This was in line with house prices, which peaked in late 2007 and dropped severely from spring 2008.

    In the first few months of 2009, however, search volumes enjoyed a far stronger January rebound than in the previous year – so we hypothesised that house prices would bottom out or even start to rise again in the middle of 2009. Let’s look at how accurate that hypothesis turned out to be.

    House price data to the present

    House prices charted against search volumes for 23 related keywords, from January 2004 to December 2009 Sources: Nationwide, Google Insights

    Sure enough, the search volume resurgence was accompanied by house price growth throughout 2009. But you’ll notice that search volumes soon tapered off, with a particularly steep fall after August. Our revised hypothesis, then, is that house prices will initially plateau and then drop again. We’ll revisit the statistics in spring 2010 to see how things turn out.

    Financial difficulties

    The second set of keywords we analysed was related to impending financial difficulties such as joblessness, debt and insolvency. They included signing on, mortgage arrears and debt problems, and were compared to the UK jobless rate.

    Financial problems searches versus unemployment

    UK unemployment rate charted against search volumes for 24 related keywords, from January 2004 to April 2009 Sources: Office for National Statistics, Google Insights

    These search volumes dip at the end of each year before rising in January – and the rise in early 2008 was more pronounced than in previous years. The jobless rate started climbing three months later, suggesting that in this case search patterns might anticipate economic statistics. We observed that search volumes had dropped significantly in the first few months of 2009, so our hypothesis was that the jobless rate would stabilise but not drop between April and July. The chart below shows what actually happened.

    Financial difficulties searches versus unemployment, until now

    UK unemployment rate charted against search volumes for 24 related keywords, from January 2004 to April 2009 Sources: Office for National Statistics, Google Insights

    The unemployment rate has indeed stabilised, wavering between 7.7% and 7.8% since early June, suggesting that our original hypothesis was valid. And search volumes have kept on dropping throughout 2009. If search trends do anticipate economic reports in this case, we should see the unemployment rate drop steadily between now and spring 2010. Again, we’ll revisit these figures in April to see if this happens.

    Conclusion

    Our hypotheses from April 2009 were largely borne out as the year progressed: the drop in house prices was reversed and unemployment rates stabilised. So maybe there is some truth to the notion that search patterns can shed some light on forthcoming economic change.

    But these hypotheses were in tune with the economic mood of the time. Many commentators were talking about green shoots and a V-shaped recession – there was a feeling that recovery was just around the corner. Today, we remain in what has become the longest-running recession in recorded history and there is considerable uncertainty about what 2010 will bring.

    Our new hypotheses are less likely to be tainted by current economic consensus, precisely because no real consensus seems to exist right now. For this reason, the idea of search predicting the future will be seriously tested as the year unfolds. Don’t forget to come back in April 2010 to see the results for yourself.


  2. An open assault on the walled garden

    Posted December 21, 2009 in strategy  |  No Comments so far

    Mobile telcos charge us for the texts, minutes and megabytes we use. They buy our loyalty by heavily subsidising our increasingly expensive phones. And they’re terrified of becoming like the people who supply our electricity or gas. They’re terrified that one day they’ll be nothing but interchangeable providers of a commodity, irrelevant logos printed on tedious, humdrum bills.

    This is why their marketing focuses so much on music, culture and lifestyle. It’s why O2 customers get priority tickets to concerts at the arenas bearing their name. It’s why Orange customers get half-price cinema tickets on Wednesdays. And it’s why T-Mobile runs that insufferable campaign about Josh and his ever-growing band.

    T-Mobile advert screenshot

    Join the b(r)and: T-Mobile want to be associated with music and lifestyle

    Customers are being encouraged to associate the brands of mobile operators with a certain type of lifestyle experience instead of just voice and data. This experience extends from the marketing to exclusive content services and even the interfaces and feature sets of the handsets themselves.

    In this sense, mobile telcos are offering their customers a walled garden, in which the mobile internet is presented as part of a convenient package branded Orange, AT&T, T-Mobile or O2. If your internet memory goes back as far as the mid-1990s this might sound slightly familiar. But in the next ten years this walled garden is due to come under direct assault.

    Charlie Stross has posted an excellent, thought-provoking piece looking at how the next ten years might pan out for the mobile industry – and making it sound in some ways like a technology rehash of the Great Game, with Apple and Google as the chief protagonists.

    As Stross sees it, Apple and Google both want to destroy the walled garden built by telcos but for different reasons and in different ways. As a premium marque, Apple wants to work with telcos while preventing their brands from adulterating the Apple experience:

    Apple don’t want to destroy the telcos; they just want to use them as a conduit to sell their user experience… [they] want to maintain the high quality Apple-centric user experience and sell stuff to their users through the walled garden of the App Store and the iTunes music/video store

    Google, on the other hand, wants people to view more of its ads. To make this happen, Google wants to fundamentally reshape the mobile industry:

    I think Google are pursuing a grand strategic vision of destroying the cellco’s entire business model… turning 3G data service into a commodity… getting consumers to buy unlocked SIM-free handsets [like the Nexus One]… and ultimately do the Google thing to all your voice messages [through Google Voice] as well as your email and web access.

    These distinct strategies both threaten the mobile telcos, who stand to lose any emotional connection they have with their customers either way. But this doesn’t mean that Apple and Google are going to be bedfellows:

    Apple’s iPhone has been good for Google: iPhone users do far more web surfing — and Google ad-eyeballing — than regular phone users. But Apple want to maintain…  the walled garden of the App Store and iTunes… [and] Google can’t slap their ads all over those media. So it’s going to end in handbags at dawn … eventually.

    The piece (here’s the link again by the way) has me thinking that the coming decade in mobile networks will be much like the previous decade was in land-line internet service provision.

    If Charlie Stross is right, the idea of the telco as provider of an experience will not last the decade, meaning that flash mobsOrange Rock Corps and Josh Ward will become nothing but a dim and distant memory. And customers will hopefully have greater choice over how they use mobile networks, which would be nothing but a good thing in my opinion.


  3. Are mobile apps here to stay?

    Posted December 17, 2009 in mobile  |  3 Comments so far

    A few weeks ago a guest speaker came to our office to talk about mobile apps. His company produced a lot of them, for pretty big brands. He knew his stuff: the team here was both impressed and engaged.

    But an exchange during the following Q&A session stuck in my mind later. One of our directors asked a question: is the mobile app destined to be a transitory phenomenon, something that will fade away as mobile browsers become capable of delivering the same functionality?

    The speaker was adamant that this was not the case and that mobile apps were here to stay. He felt that Google’s increasing preference for mobile browser apps over native apps was misguided and that Google were wrong on this one. Mobile browsers were so far from rivalling the functionality of native apps that it wasn’t even worth thinking about.

    I was tempted to counter this point by bringing up the iPhone’s support for HTML 5 and starting a detailed discussion about in-browser capabilities. But this wasn’t the main subject of the talk and I’m in no way an expert on HTML 5, so I decided to keep my mouth shut instead.

    In the weeks since the talk, however, I’ve often found myself turning this question over and over again in my head. And the more I think about it, the more I feel that mobile apps are basically doomed – or at least I hope they are.

    Don’t get me wrong – they play an important role. It’s good that so many people today see phones as devices for more than just calling or texting, and the iPhone and its suite of native apps is largely to thank for this. But in the longer run, the publication and distribution model they are based on has to go.

    The idea of tying software to a single hardware platform is anachronistic, uncompetitive and limits user choice. This is bad enough when you’re dealing with computers, but it’s even worse when the devices are as personal as mobile phones. People should be free to choose a different phone without needing to buy new versions of the software tools that have become integral to their lives.

    Aside from user choice, there’s a more practical reason why the native app model is unsustainable. Developers won’t want to keep maintaining multiple codebases for the apps they produce, especially when there’s the option of building an equally functional in-browser app which any standards-based client can run. And although Apple might hope to render this point irrelevant by establishing monopolistic domination of the smartphone market, relieving developers of the need to consider other platforms, current research indicates that they won’t succeed.

    The smartphone OS market will be more fragmented in 2012 than in 2009

    The smartphone OS market will be more fragmented in 2012 than in 2009

    A more fragmented smartphone OS market will increasingly compel developers to support separate codebases for Windows Mobile, RIM, Android, Symbian and the iPhone. But as mobile browsers become capable of delivering similar interactivity, serious developers will become inclined to start using the browser as the platform instead. This will be a good thing for users and the industry alike.

    If I’m correct and native apps do fade away over time, we may look back on the era of pointless mobile apps as just one among many strange blips in the history of technology. But despite some early rumblings from notable developers, native mobile apps will be with us for some time yet – and, in the medium term at least, they still have an important role to play in encouraging mainstream adoption of the mobile internet.

    Edit: This article was later reposted on Android and Me and attracted numerous comments. Click here to see the conversation on Android and Me

    Edit 2: Stephen Fulljames shared a couple of links related to this post. PhoneGap is a toolkit for developing mobile apps in HTML & JavaScript. And this post from front-end consultant Peter-Paul Koch provides some background to his work with Vodafone on mobile browser compatibility and W3C widgets.


  4. How to post your Last.fm loved tracks to Twitter

    Posted December 8, 2009 in How-to  |  5 Comments so far

    I remember when Twitter was still quite new. Back then, a lot of people were still trying to think of uses for it and one thing that was fairly common was to plug it into your Last.fm account.

    In retrospect I can see why that was seen as a good idea. Twitter was supposed to be about broadcasting minor ephemeral details, and the music you were currently listening to definitely fell into that category. But there was a downside. People listen to a lot of music and, with a Twitter post for each track played, that added up to a lot of useless information on Twitter. Thankfully, the practise of scrobbling directly to Twitter soon faded out.

    Today there are some more useful and less irritating ways of posting information from Last.fm (or, indeed, its open source alternative Libre.fm to your Twitter account. One of them, Tweekly.fm, produces an automated weekly tweet of your top three artists. Another one, which I’m going to explain here, involves posting tracks that you “love” on Last.fm to your Twitter account.

    Here’s how it works:

    1. If you don’t have a Last.f account, create one here
    2. Get the URL of your “Loved tracks” RSS feed. This is easy: just change “USERNAME” in the URL below for your Last.fm username.

      http://ws.audioscrobbler.com/2.0/user/USERNAME/lovedtracks.rss

    3. Test the URL by opening it in a browser. You should see something that looks a bit like this:
      Last.fm RSS feed browser output
    4. If it works, go to Twitterfeed.com and create an account if necessary
    5. Once logged in to Twitterfeed, click on the “Create new feed” button to the top-right of the screen
    6. In “Step 1: Send Feed To”, select Twitter. Click on the large “Authenticate Twitter” button and enter your Twitter account details. You’ll then be directed back to Twitterfeed.com
    7. In “Step 2: Name feed & source URL”, enter a name for the feed – this can be anything you like. In the “RSS Feed URL” field, paste the URL of your RSS feed
      Twitterfeed screenshot 1
    8. Click on the “test feed” button to make sure the feed is valid
    9. Click “Advanced settings”. A bunch of new options will appear underneath. Here’s a screenshot with the things you need to check circled in red:
      Twitterfeed's advanced settings

    10. In “Post content”, select “Title Only”. This will ensure that the posts to your Twitter account only contain the artist, title and shortened URL to the track you loved
    11. Make sure “Post link” is checked and a URL shortening service is selected
    12. You might also want to enter some text in the “Post Prefix” or “Post Suffix” fields, otherwise your tweets might be slightly baffling
    13. You’re done – just click “Create feed” and that’s it set up.

    Now whenever you “love” a track on Last.fm, your Twitter account will post a link to it. This makes Last.fm’s “love” feature a bit more useful when it comes to recommending music to other people – especially people who don’t use Last.fm. And as long as you don’t love everything you listen to you won’t be clogging up your Twitter feed.