the loose wire podcast

Just because it can be found doesn’t mean it’s worth indexing

We’re living in a digitally ass backwards age, where everything you do of commercial monetary value is retained, amazingly easy to recover and commercially mined at mind-boggling speed, but everything else — photos, an SMS, an email you sent to someone last week, the deed to your house — is lost in the muddle of that drive an inch or two from your finger tips.

Mike Bird’s tweet above is one illustration of this paradox. But perhaps the best one is this: I have a vast database of stuff I’ve collected over the years from the internet. But although I use the best tools out there to index that stuff, it’s still much easier to search for those documents on the internet via Google. Easier as in faster and more thorough (more documents, newer versions). It wasn’t always so, however. For one thing, the internet wasn’t always faster than your apps+your hard drive.

For another, Google used to have an excellent tool called Google Desktop which searched your computer for you while also searching the net. This was during the wonder years of 2004 through to about 2008 when software was Good, in both senses of the word, and the idea of indexing your hard drive and making things easy to find seemed like a worthy cause; some of you may recall Enfish and some other efforts in this field, which I suspect Google Desktop killed off before it took its own life, probably having completed its mission.) For a heavenly few years the two were united: what you had in your computer and what was online were one and the same.

Enfish: out of its hands

But Mammon won. Because the simple calculation was: there is no commercial value on your hard drive because privacy prevents Google from mining that data or learning intent from your searches so why would we waste money on that. Once Google had won the search engine wars in the 2000s (see my post on that here) which was what the Google Desktop war was about, Microsoft being their main competitor at the time), there was simply no commercial imperative in continuing to support Google Desktop. Besides, by then we had been persuaded to move most of our email onto the web, so why would we want to be searching our hard drives anyway? There really is no market for what I, foolishly, once believed was the future of the browser. To me then, and now, I still can’t quite figure out why would you NOT search the web and search your own computer at the same time?)

So in short: search is only valuable to the provider of that search if it is to look for things of commercial value, and things we have not yet purchased, and things we might intend to purchase, and have the means of purchasing. It’s not hard, when you look at it that way, to see how things have skewed, in the past 20 years from a paradigm of: “Your computer! It will be where you will create and save all your memories, your documents, your house plans, your photos, all in one place! Forever!” To: “Your devices! You can consume anything you like! Whenever you like! Instantly! Forget about the future! Live for now!”

OK, that’s obvious for the ‘order now from the gig economy’ world. But it started long ago. It started with iTunes, and continued with things like Kindle. Those digital books you ‘bought’? You don’t own them. You bought a license. You die, that book reverts to Amazon. You might be able to save those items somewhere, but you can’t really do anything to them as if they were yours, you can’t search them outside the Kindle app. They are imprisoned. 

And yes, it’s true that your photos are saved from your camera relatively seamlessly to other devices, except it’s less than obvious how, evidenced by all the hours you spend trying to find a photo from that birthday party ten years ago that should have been saved to a computer but doesn’t seem to have been because that computer was upgraded a few years ago and doesn’t seem to have been moved across, or the WhatsApp message that was on an old phone or an email that was on an old account which you ditched when they started charging outrageous fees for attachments. Or automatic online and time machine backups which all go smoothly for years until you actually need to restore something in which case the one file you need is missing. And did you know that hard drives actually only last five years tops? Assuming you’ve been using hard drives to back up your treasured memories for 15 years (20 in mine) that means you should have replaced those hard drives at least twice.

In short, even if you have been doing everything right, you’ve been doing it wrong. 

Our digital world is built to be commercially optimised for speed, not for ensuring our memories, our records, our heirlooms, belong to us, and are safe. This is not to say companies like Apple don’t try to sell us the whole shebang, make us feel that by selling us computer+services+software that we’re still in that 20-year-old paradigm of ‘your own computer where only good things happen’ but in fact it’s better to think of it as as a big shopping cart. Few of the things you do with it you actually create or produce, and while Apple does offer some tools to help you move stuff from one old device to another, the emphasis is not on safeguarding your ‘digital heritage’ as much as persuading you to keep subscribing to their digital services — to expand your plot in their walled garden.

Now you could argue, I suppose, that Facebook has taken a different approach, building a business on digital longevity, lovingly creating a website that thrives on us tending to our shared memories. And perhaps there’s some truth in that. But let’s explore that for a moment. Can I easily search my memories for something? Sort of. I suppose if I organised things correctly I could find a birthday party album. But the search bar on the timeline doesn’t seem to work, and the best way to navigate seems to be by year. That is a tedious option. Overall, the Past in Facebook world seems to be something for the company’s algorithms to dish up occasionally, rather than a place for us to explore. Like Google, Facebook presumably has less interest in it because it’s not reflective of our present or upcoming needs and desires.

It’s time for us to apply some pressure for the commercial world to stop thinking like this. We have given so much of our lives and data to these vast harvesting machines it’s time to wrest some of it back. And not just to demand the data, but to demand it in the format that we left it in. For example: You can now download your data from Facebook and Google but this is all — intentionally, in my view — in such a mess that it would take a year to put it some order. It is absurd, in my opinion, that the tools to curate our footprint online are impressive (in Facebook, for example, I can create an album called ‘Mum’ to remember my mother, where the photos are dated, geotagged and name-tagged, where they are displayed as thumb-nails and where I can choose who can view them and where those people can comment. This is all assumed as based features.

Facebook cares about me and my memories, so long as I don’t’ try to find them, or download them some place else

Now if I want to download that album to my computer, how do I do that? It took me one minute (not too bad, considering) to figure out that (arguably) probably my best option was to download all my photos. This of course may take some time. So while that was going on I decided to right click and download the photos individually. Now to be fair this is not as hard as it could be (in Safari, right click on the image, download as image to the default folder). The problem is that all that richness and effort is lost: The filenames are now meaningless — 30535610150348063416882213385n.jpg rather than 1959 Mum with baby Aldrich.jpg— the metadata is gone, the geotagging, the people tags, the comments, the albums. All that curating work you did on Facebook is for naught. It’s a stark reminder that the tagline about Facebook caring about you and your memories is a flagon of balderdash. (The download process is no better, though the albums are at least in separate folders.)

So that’s the first step. I will talk about future steps in future columns. Perhaps this Covid-19 year (and some of next, no doubt) will not be such a waste. We will realise the devices in front of us are not just oversized credit card swiping machines or pocket-sized cinemas, but actual treasure chests for organizing and storing our heritage, our heirlooms, things we can create and pass on. But first we need to reclaim them. This needn’t be a zero sum game. The Walled Garden machines can help us do this, and expand their business models to include better, more resilient home networks that include resilient storage, tagging and search (more of this in a future column.)

For now, start thinking: all this digital gardening you’re doing? Who is it for? And will it leave it a trace for anyone but those hordes of digital marketers? 

The change is this: last year Apple introduced something called Sign in With Apple, inevitably abbreviated to SiWA. You’ve probably seen it in action: it sits alongside other so-called social logins from Google, Facebook and others, allowing users to sign up for services and apps with their account at one of those services, meaning they don’t have to enter lots more information, and remember a new password.

Google and Facebook have been in this game since around 2011, and Apple is a little late to the party. They only introduced the option last September, and said that developers only needed to add the option to apps on their platform if the only existing choice was another social login.

So before we get in the weeds here, let’s just take a couple of steps back. What is the benefit to companies like Google, Facebook, Amazon, LinkedIn, Twitter and Apple to offer this log-in service? Surely it’s extra work for them to wire another company’s plumbing, so to speak?

Well, no. As you might have guessed, there’s a significant advantage for the likes of Facebook and Google — they get to track you to yet another corner of the web, while the host service saves money on setting up, managing and securing passwords etc. Yes, you might say, that is true of those companies that make money out of your data, but Apple don’t do that, right? Why would they be interested in doing this?

Here is where it’s a big murkier. Apple’s argument, as I understand it, is essentially this: We want to help keep our users’ data secure, and so if we offer this service — where users don’t need to set up a new password, they just use their existing Apple account — we help limit their exposure to crooks and bad guys. As if to prove that point, they offer an extra layer onto their SiWA: you can make yourself partly anonymous to the service you’re signing up for via a layer of throwaway email addresses, all of which Apple handles and which can’t be tracked back to you.

So no question: if you’re an Apple user and you want to use one of these services, then Apple is the way to go. So far, so good. Until March, that is. On March 4 Apple quietly removed a word — “exclusively” — from its terms for developers to post their apps on the App Store. Developers can’t get their apps onto your iPhone or iPad unless they get approval of the App Store police, and so this word turns out to be quite an important one.

comparison of text, pre- and post-change

Previously the sentence read: Apps must include the Apple SiWA option if they “exclusively use a third-party or social login service”. When that word disappeared, it meant that now if any app offered any third-party or social login service, they must offer the Apple version as well. Otherwise their app would not be approved by, or an existing app slung from, the App Store.

So what in practice does this mean? In effect, Apple is saying that either apps remove all third-party sign-in options, or they include Apple’s as well. This might seem like no big deal, but some developers aren’t happy. One, a company called Pushbullet which (ironically) makes it easier to connect devices together, has pulled its iOS app in part because of the rule which they said would create a lot of extra work (See below). Another developer I spoke to said that after getting their app thrown out of the App Store until it complies they have decided to remove all third-party options and set up their own system, costing them $20,000 and likely some subscribers in the process. “It’s an example of Apple imposing their will as a condition for approving a minor change,” he says.

Some app developers have already baulked at the idea of social logins, realising they lose visibility into and access to their customers. While Facebook, say, will share some of the user’s profile details with the service when they use that sign-on option, many say that most people use for their social login the service they use the least: oftentimes the Facebook profile lacks a full name, or an email address. Others have found that they now have one more layer, rather than one fewer, when it comes to security. Indian top-up service Freecharge said this week it had recently removed social logins for Google and Facebook had been removed “for security reasons”. A spokesperson told me that “As part of our regular reviews, we decided to disable social login and streamline the login process with an OTP as it provides an easier user experience,” without elaborating.

And this is the thing. These social logins are popular among users, but probably for all the wrong reasons. We like them because they’re faster, less hassle and because we don’t have to remember our password. But that’s just it: we feel like we’re giving away less information, whereas we aren’t. We may use our least-used account for the sign-in, but that’s still data being passed onto two services — the app developer and the big technopoly. Some services still go ahead and create a separate account, so you’ve ended up worse off than better off.

There are other reasons to think this isn’t a good deal either. Security is one: All these big players have been hacked at one point or another, so just because we have put all our sign-on eggs in one big basket doesn’t mean the eggs can’t break. And if someone does hack your Facebook account, say, you’ve just made it a lot easier for them to access all the other services you’ve provided your login details to.

Mohammad Ghasemisharif of the University of Illinois, who co-wrote a paper exploring the security risks of single sign-on (the umbrella term which includes social logins)1, told me: “I do think there is a trade-off here. On one hand, big tech companies have enough resources and talents to allocate towards securing their products. However, they are still prone to mistakes like any other company, with a caveat that their mistakes could have a much larger impact.”

And is it a good thing, asks Troy Hunt, who runs the excellent haveibeenpwned, and has been critical of social logins in the past: “I worry about the conditioning of people to enter their social credentials into a context different to the social site itself,” he told me. At the same time, he sees advantages: “I love that it reduces the footprint of the total number of credentials repositories out there. I also like that it’s a lot harder to screw up from an implementation perspective.”

And Apple? Well Apple is right when it says it’s more in the privacy game rather than the data game. Friends I’ve spoken to say that having Apple offer social login means they don’t have to give their details to some random site, and yet still receive emails. They’d rather entrust their data to Apple than to someone mining it for data.

But my feeling is this: that shouldn’t really be what gives you pause. The real reason they (and Google, and Facebook, and Amazon) are doing this is because it adds another brick to the wall and a lock to do the door to keep you in their walled garden. For them, as I’ve said before, more important now is not selling you more hardware — that’s assured, even if you’re not maybe upgrading as much as you did before — but to get you locked into more services that are exclusive to Apple. An Apple SiWA fits the bill perfectly.

Some options are more visible than others

What happens, for example, if you swap your iPhone for and Android phone? How are you going to use your Apple SiWA now? (The answer is, you can, but it’s a kludge, and Apple has no incentive to make it any easier.)

Of course, this is also why Facebook, Google, LinkedIn (Microsoft), Twitter and Amazon are in this game. It’s a game of overlapping territories, a grand canvas upon which the great technopolistic game is played. No-one player dominates the whole map, but Apple has belatedly realised that this is a theatre they need to be playing in. Being a technopoly is always about controlling the gates, because as gatekeeper you can decide what goes through and what doesn’t, and what price has to be paid. Being a gatekeeper you can change the rules, as Apple did with no fanfare or explanation on March 4. They can shut things down.

The idea is always to make it easier to move around within the walled garden, so long as you’re not doing anything where you either go out of the walled garden, or your money does.

Does this all really matter? In itself, no. Though what I’m trying to show is how these small things end up becoming big things, but in a way that mostly flies under the radar. I couldn’t find anything in the Judiciary Committee documents about these social logins, nor in the recent UK’s Competition and Markets Authority’s report about the creeping battle to control the sign-up and authentication process. But it seems an obvious new battleground, and now that Apple is in it, expect it to get hotter, and sneakier, as the garden walls grow taller.

It’s hard to read some of the documents released by the House Judiciary Committee without some rising bile.

What is clear — among other things — is that the internet as a commercial entity has spent more of its life dominated by companies determined on building walled gardens than it has a genuinely free space.

First a bit of history The internet as a commercial environment only came into being through services like AOL and CompuServe in the 1980s, which allowed users to dial in to a quasi-internet service via modem.

CompuServe was launched in 1977, though it wasn’t until the mid to late 1980s that it became a broader service. By 1989 it had half a million subscribers in the U.S., and had services in a dozen countries. (I signed up for it in Hong Kong in 1991, by which time it had 620,000 subscribers and annual revenues above $200 million.) 

CompuServe was a walled garden, curating the content you could view and extracting a heavy toll via its subscription fees. It did have access to the internet, but it was limited. I could also send and receive emails from my CompuServe account to those with ‘real’ internet addresses, and vice versa, but that was pretty much the extent of it in those years. By the mid 1990s it was facing competition from Internet Service Providers (ISPs) — who offered a suite of apps to give you access to the web itself, and a dial-in number — and AOL. By 1997 CompuServe was owned by its adversary, although it still had a user base of more than 2 million.
(Source: History of CompuServe Interactive Services, Inc. – FundingUniverse)

Indeed, by 1995 there was really no point in having a CompuServe account: just sign up with a local ISP and you had the web to yourself. Suddenly we were away from the walled gardens and we were free to wander the web. There was only one catch: it wasn’t easy to find stuff — something CompuServe excelled in via its directories and spaces — until search engines came along.

I don’t know about you but I still think of that decade, from about 1998 through to 2008, as the halcyon days of user choice, when it came to search engines. We had Altavista, Yahoo!, Ask Jeeves, InfoSeek, Lycos, Inktomi’s HotBot, AlltheWeb, and of course Google. But look more closely, as I did with data I cobbled together from various sources, and you’ll see that Google had stolen the show from as early as 2003:

Search engine market share. Research by loose wire, from various sources 

By the end of the decade Google was the pre-eminent search engine with a market share of more than 90%. Yes, Google was a great search engine, but as we know now, this loss of choice carried with it a cost: our world became Google’s world, and Google’s world became ours. It was a walled garden much like CompuServe’s. Only this time it was being directed to Google’s services, or to Google’s growing data trove by being watched by Google’s cookies, essentially indirectly paying Google a tax for its dominance.

But they weren’t the only walled gardeners in town. Once again, we think of the arrival of Apple’s iPhone, iPad and App Store as this liberation from the yoke of Microsoft, boring software and hardware, and a lack of anything interesting to do with your phone. And that is true. What we tend to forget is that Steve Jobs had no interest in delivering an experience beyond the walls of his own garden. It’s quite explicit in the documents, and it’s a sober reminder to me that Apple, from the start of its ecosystem, had no interest in helping the user unless it could cream off the experience.

Here he is in Nov 2010 telling Amazon to use Apple’s payment mechanism or get the hell out:

And here he is in Feb 2011 making clear he wants only one bookstore in the App Store:

There’s nothing unclear about this. There’s no weasel words about protecting the user, or ensuring their experience is as good as possible. It’s old fashioned rent-seeking and wall-building. And of course it’s obvious to most of us. I suppose what is surprising to me is how far back it went, almost to the start.

And Facebook and Amazon are no different. All four appear insatiable, according to the documents, to use their market dominance, for their customers’ benefit, but the opposite: to make sure the rent all other players pay them is maximised, and where competition and choice arise, to snuff it out quickly. There’s nothing in the documents that I’ve read so far (and admittedly, I’m only some ways in) to suggest that any of these companies had any regard for their users in any of the decisions being made and negotiations being conducted. Of course, there’s no need for them to do so; there’s no oversight committee or PR angle that needs to be put front and center. When the TV arc lights are out and they think no one is listening, it’s all about building those walls ever higher.

So, it turns out, as a historian, that the internet has been under the influence or control of walled gardens longer than it’s been free: arguably the only time when we had a significant degree of freedom was from about 1995 to 2002. Certainly by the end of the 2000s we were doomed. I find it intriguing to note that this period was maybe when the web was its most creative, coming up with many of the tools and principles that these technopolies took as their own. But I’ll save that for another time.

The BBC World Service Business Daily version of my piece on cameras. (The Business Daily podcast is here. Script is here.)

Loose Wireless 120516

To listen to Business Daily on the radio, tune into BBC World Service at the following times, or click here.

Australasia: Mon-Fri 0141*, 0741

East Asia: Mon-Fri 0041, 1441

Thanks to the BBC for allowing me to reproduce it as a podcast.

The BBC World Service Business Daily version of my piece on the Google Dilemma (The Business Daily podcast is here.)

Loose Wireless 120117

To listen to Business Daily on the radio, tune into BBC World Service at the following times, or click here.

Australasia: Mon-Fri 0141*, 0741

East Asia: Mon-Fri 0041, 1441

Thanks to the BBC for allowing me to reproduce it as a podcast.

The BBC World Service Business Daily version of my piece on the rise of the smartphone (The Business Daily podcast is here.) 

Loose Wireless 120111

To listen to Business Daily on the radio, tune into BBC World Service at the following times, or click here.

Australasia: Mon-Fri 0141*, 0741

East Asia: Mon-Fri 0041, 1441

Thanks to the BBC for allowing me to reproduce it as a podcast.

The BBC World Service Business Daily version of my piece on The Web of Things to Come  (The Business Daily podcast is here.)

Loose Wireless 122111

To listen to Business Daily on the radio, tune into BBC World Service at the following times, or click here.

Australasia: Mon-Fri 0141*, 0741

East Asia: Mon-Fri 0041, 1441

Thanks to the BBC for allowing me to reproduce it as a podcast.

The BBC World Service Business Daily version of my piece on how video doesn’t always tell the truth. (The Business Daily podcast is here.) The piece it’s drawn from is here. 

Loose Wireless 120711

To listen to Business Daily on the radio, tune into BBC World Service at the following times, or click here.

Australasia: Mon-Fri 0141*, 0741

East Asia: Mon-Fri 0041, 1441

Thanks to the BBC for allowing me to reproduce it as a podcast.

The BBC World Service Business Daily version of my piece on my predictions for next year  (The Business Daily podcast is here.)

Loose Wireless 110913

To listen to Business Daily on the radio, tune into BBC World Service at the following times, or click here.

Australasia: Mon-Fri 0141*, 0741

East Asia: Mon-Fri 0041, 1441

Thanks to the BBC for allowing me to reproduce it as a podcast.

Here I discuss on my regular slot with Adelaine Ng of Radio Australia patent wars, Amazon’s pricing policy and anything else we can think of. (This is from last Wednesday, so probably of historical interest only, unless you like hearing me say ‘er’ a lot.) 

Loose Wireless 111101

Thanks to the ABC for allowing me to reproduce it as a podcast.