Copyright, content industries and "that's not fair"

In the sense of "and I'm going to tell my mummy government". Robert Heinlein penned the following quotation, as part of his first ever published story:

"There has grown up in the minds of certain groups in this country the notion that because a man or corporation has made a profit out of the public for a number of years, the government and the courts are charged with the duty of guaranteeing such profit in the future, even in the face of changing circumstances and contrary to public interest. This strange doctrine is not supported by statute or common law. Neither individuals nor corporations have any right to come into court and ask that the clock of history be stopped, or turned back."

Robert A Heinlein, Life-Line, 1939

The copyright reform battle is enshrined in this thinking. On the one side we have people reasonably pointing out that the move to digital from analogue completely changes how content can and is being reproduced, reducing some formerly expensive processes to zero cost; on the other we have the rights holders shouting "no fair" and demanding the law is made tighter, rights extended longer, and that everyone else must be made to expend their resources and all unreasonable efforts to protect their lucrative business model.

But their business model, historically lucrative and rather young, is now outdated. Setting a book for a PDF can be carried out using cost-free software in hours and reproduction is free; people are creating popular music in their bedrooms and distributing it at no cost, and with the rise of Aframe (and presumably soon a Samwers clone ;) ) film production costs are following the huge drop in distribution and reproduction that digital and the Internet had already brought.

A start-up founder should...

A start-up founder should be able to man the support lines, develop strategy, make a sale, prepare financial statements, design a marketing plan, write website content, balance accounts, build a desk, set targets, comfort their staff, take orders, give orders, cooperate, act alone, solve equations, analyse a new problem, pitch to investors, programme a computer, cook up a business plan, grow efficiently, fail gallantly. Specialisation is for big corporates.

(with apologies to the ghost of Robert A Heinlein)

BT and the insanity of over-large companies

Here's a thing. Rural broadband. It's bad. It's bad partly because it's a long way along copper lines to the houses from the exchange (although I know people next door to their rural exchange who get suspiciously low speeds). This major element of the problem could be quickly and easily resolved by moving to fibre to the cabinet (FTTC). The cabinet is typically only a few hundred yards from the houses it serves, and fibre optic connections can carry signals over great distances with minimal loss of performance.

But BT, who rural residents are dependent on for infrastructure, don't see rural exchanges as economic to upgrade. The same BT, but in a different department, are tearing their hair out over the cost of maintaining copper infrastructure for these rural exchanges, as copper theft becomes increasingly common and rural locations are relatively low risk targets. Replacing a stolen length of copper serving an entire village is expensive in so many ways; the cable is expensive, the civil engineering that can be required is expensive, the telephony engineers to reconnect all the connections is expensive, the customer support dealing with complaints for the week or so the village is disconnected is expensive; and then there's the societal cost where elderly residents are cut off from the outside world for that long, potentially putting lives at risk.

Is it really so expensive to move rural exchanges to FTTC? That much more expensive than the costs of the copper cable thefts? More expensive than lives? Is it really uneconomic? No, probably not. But the upgrade department budget will be so very, very distant from the repair department budget; there's almost certainly no one with oversight of the two cost centres at a detail level necessary to see the opportunity. So rural residents get hit from both sides, having the lowest level of service while paying the same amount for their connection, while BT pay out for patching symptoms of a problem they could just rid themselves of.

Big companies don't have to be run in this short-sighted, disconnected way. But so many are...

What we can learn from monkeys (part 2)

If, the saying goes, you take enough monkeys and typewriters then given enough time it is statistically probable that they will reproduce the entire works of Shakespeare. Not a word or a phrase at a time, but each play as a coherent whole. The chance of the letter "n" being pressed, out of the 46 keys on our typewriter, is 1/46, the same for "o", the same for "w", for " "[1], for "i","s"," ","t","h","e"," ","w","i","n","t","e","r" and so on until we get the complete and excellent opening line of Richard III:

"Now is the winter of our discontent made glorious summer by this sun of York"

The probability of this occurring with just one monkey is 1/46*1/46*1/46 and so on for the number of characters (76). To be more specific, the probability of creating just this first line is

1 in a number far too big to even write out in normal numbers (46^76)

Hmmm. Ok, to bring it down to numbers worth writing out, let's just look at the first two words "Now is", a total of 6 characters, which the chances of one monkey randomly tapping out is

1 in 9,474,296,896 (making the odds of winning the UK lottery jackpot of 1 in 13,000,000 seem positively likely!)

But with enough monkeys and enough time it becomes more and more likely that not only will one them tap out those opening 6 characters, but also the opening 76 characters, and even the all of the large number of characters that is Richard III, all the rest of Shakespeare's surviving plays and even the lost ones (although I'm not sure how we would know the lost ones had been correctly typed out...). By combining an infinite number of monkeys and typewriters, it would not be statistically significant that the monkeys produced the complete works; if you started with nothing and in 6 weeks someone delivers a typewritten manuscript the fact that infinite monkeys were involved means that statistically it would be not unlikely i.e. not *mathematically* improbable that the manuscript has been produced by pure random chance by a bunch of monkeys with typewriters.

Arthur looked up. "Ford!" he said, "there's an infinite number of monkeys outside who want to talk to us about this script for Hamlet they've worked out."[2]

So here our ever helpful monkeys are helping teach us something about statistics; that they are a dangerous source of truths. While not mathematically improbable, this truth is heavily dependent on a few highly unlikely things, like having an infinite living space in which to house infinite numbers of monkeys with their typewriters.

This becomes even more of an issue when numbers turned into statistics to be used by politicians and news outlets and those others with an agenda, who all too often mistake correlation for causation, using statistics to demonstrate why some new policy or other is needed or why a current one should be changed, when (1) not understanding that statistical significance is mostly about having, or assuming, the right amount of monkeys and that (2) the fact that two measurements correlate does not mean one caused the other. News outlets in particular also have a tendency to reproduce statistics as the agenda-pusher would have them reproduced "the murder rate in the country has gone up 10%" (was 10 in year 1, 11 in year 2); the same numbers could just as easily, and probably less misleadingly, have been reported with a more qualitative statement such as "the murder rate in the country was stable".

It is this misuse and misunderstanding that help give rise to the saying "Lies, damned lies and statistics"[3]; but everyone is at it. Internal "news outlets" (news inlets?) are equally prone to misleading messages of the types "90% of users rate the IT support service as 4/5 or higher", which is more precisely reported as "of the people who got around to filling out the satisfaction survey when their helpdesk ticket was closed, 90% rated the service as 4/5 or higher". A statement that would be further informed by the information that the survey defaults to 5 and you have to change it to anything lower; then take into account such truisms as people are more likely to complain than praise. Eventually a qualitative statement would work out to be more useful "the IT support service is making very few people angry", which if new software is being rolled out is good news indeed!

Qualitative statements are seen to carry less weight than ones laden with numbers, which in organisations is probably the fault of the CFO; this seems ironic, given the quantity of assumptions and informed guesswork that is the basis of corporate accountants...

In the end remember this: statistics just provide information in a numerical form. What it all actually means is a matter of interpretation. In other words don't be mislead into believing the numbers are not just another qualitative measure...

[1] Although surely the space bar is so much bigger it would be more likely to be pressed? Damn these complications and assumptions...
[2] Douglas Adams, The Hitchhikers Guide to the Galaxy, with a little help from the Improbability Drive
[3] Said by someone, some time in some form: http://www.york.ac.uk/depts/maths/histstat/lies.htm

Installing Linux on Dell E5420

As is customary, to help others following along the path, here's some instructions for installing Linux (Kubuntu 11.04 but I'd guess any modern distro would be a similar experience) on a 2011 Dell E5420. In case the particular specification has an impact on the steps needed I've appended it to the bottom of this post.

These steps assume you have a prepared bootable install medium. I use a USB key, but CD/DVD would be similar.

Step 1 (optional): Ensure laptop is connected to mains and network (best to use wired).

Step 2: Press F12 during BIOS spash screen to reach boot menu, and select your boot media.

Step 3: When install option comes up, select it. Accept all defaults and complete user information as requested/desired.

Step 4: When instructed, reboot laptop. If using USB and an SSD this is in less than 5 minutes.

Step 5: Everything works. Enjoy :)

Sadly this traditional post is basically pointless, because it was that easy. All but one of my nieces could have done it (the youngest is only a month old, give her a chance...). In less time than it takes to install a Windows service pack you can take a blank laptop and transform it into complete working environment with all the core applications (office suite, etc.) already installed. No technical expertise required.

PS Rather nice, solidly built, nice to use laptop, in case you're interested. 3 mouse buttons on the Trackstick too. Battery lasted 7 hours out of the box, including all the install work :)

Spec (yes, no Microsoft Windows was supplied nor paid for):

Latitude E5420
Processor : One Intel Core i3-2310M (2.10GHz, 3MB cache, Dual Core)
Display : 14.0in UltraSharp HD+ (1600X900) AntiGlare LED-backlit with Premium Panel Guarantee
Palmrest : Dual Point Touchpad and Trackstick
Camera : Integrated Full HD Camera with Microphone
Memory : 2GB (1x2GB) 1333MHz DDR3 Dual Channel
Hard Drive : 128GB Mobility Solid State Drive
Optical Drive : DVD +/-RW Drive
Battery : Primary 9-cell 87W/HR, 3 Year Warranty
Wireless : Dell 375 Bluetooth Card
Wireless Card : EMEA Intel Centrino Advanced-N 6205 (802.11 a/b/g/n) Half Mini Card
Keyboard : Internal UK/Irish Qwerty Backlit Dual Pointing Keyboard (antimicrobial)
OS Media : Not Included

When not finding something is an error

Or to be exact is a #VALUE! - a passing tip for those using spreadsheet functions that return an error code instead of a usable outcome.

I was using the FIND function to look for a character in a text string. Specifically I was trying to find "/" so that I could rip domain parts from a huge list of URLs. My problem came when some of the URLs were just domains and contained no trailing "/" - at this point my formula

=FIND("/",A1)

returned a #VALUE! code instead of a 0, where FIND's outcome is normally a numeric reference indicating the location of string. A bug, I thought, so let's get it fixed. As I was using LibreOffice Calc and their release cycle is more or less monthly, I figured I could live with the issue for now and get the bug fixed for next month.

A conscientious software user always, I duly posted the bug and a day later I get an email indicating someone was processing it. I open the email and... the bug has been closed, marked NOTABUG. The associated comment said "This is not a bug. It is the expected behavior in ODF 1.2". As the newly completed OpenDocument Format 1.2 had some of the world's top experts developing the formula specification, this seemed, well, odd.

For those who, like me, couldn't understand why 0 isn't a logical outcome for "not found" in the FIND function, I asked someone I know who has some expertise in this sort of thing (Rob Weir of IBM, a spreadsheet developer since 1990ish and the chair of the ODF Technical Committee). Turns out it's that damned legacy thing again...

Back in the early days of spreadsheet development, 20 or so years ago, it was believed that a user wouldn't understand 0 as the outcome of a test that finds nothing, potentially causing subtle errors to cascade through the spreadsheet. So spreadsheet developers adopted the practice of outputting an error as the null result for this type of function. As it is now a well-established practice it was decided ODF should stick to it in the formula specification.

As my FIND function was nested in a larger formula, a error output just caused the whole formula to fail, breaking my automated process. Luckily it turns out there is a workaround using another function called ISERR (ain't it great knowing really clever people?). So if like me you need FIND to return an integer or otherwise testable answer rather than an error, wrap it in a ISERR:

 =IF(ISERR(FIND("my string",<the cell to search>)),0,FIND("my string",<the cell to search))

Sorted. Or rather found...

The public good of software

Or public goods, to be more precise. Every economics student can promptly skip three paragraphs.

The subject of economics likes to classify and simplify things; some of these classifications and simplifications are useful, some not. Several in classical economics are downright dangerous, but I digress. One classification that more or less all economists would agree on is the concept of the public good. A public good is one that is collectively paid for because it makes no sense to pay for it any other way; the majority of services provided by the public sector are, or should be, public goods per se.

A classical example of a public good is streetlights. Streetlights have been around since 800AD or so; the first streetlights as we recognise them today were gas powered and switched on in Pall Mall in 1807; the first streets to be lit by electric light were in 1881 (Cleveland, Ohio and Newcastle-upon-Tyne appear to share this honour). All these streetlights share one distinctive feature; it is effectively impossible to have streetlights and charge individually for benefiting from them.

Early streetlights were either installed for public benefit or as a marketing ploy; modern day streetlights are more or less universally funded from local taxation. No one has ever managed to charge directly the user of the streetlight; creating and managing a use-based charging model would be several orders of magnitude more expensive than the provision of the light itself. Light being tricky stuff would provide much of the complication. How much light is enough before you get charged? So how close to the light can you get for free? Do you get charged different amounts depending on the how dark it would be if the light was turned off? How foggy it is? Do people who get the light spilling into their houses get charged? Anyhow, it's far too complex a question and to my knowledge it has never even been attempted. Residents in the town/region pay, everyone who passes by at night benefits.

Recently I argued against open source software being a cartel market. I closed that post by pointing out that software could not actually be seen as a normal economic good; can software be best seen as a public good? Interesting question...

The formal definition of a public good is that it is non-rivalrous and non-excludable. A good is seen to be excludable if you can easily and cheaply prevent those who have not paid for the good from consuming it, while a rivalrous good is one where your consumption of it prevents another from having it.

So here I am with my computer, using LibreOffice to write this post. If software is a normal good then my using it should be preventing you using it. Which clearly it does not as even if you don't yet have it you could just follow the download link to get it, I could give you a copy of the one I'm using and, on a thin client setup, we could all be using the same installed copy, so software (all software) is non-rivalrous; one tick in the box. Interestingly the common example of a non-rivalrous good is broadcast media - yet software is utterly non-rivalrous while broadcast media has a number of restrictions (no simultaneous use of wavelengths, requirement of near line of sight to transmitter, limited range). An alternative definition of a non-rivalrous good is one that could be provided at a very low additional (marginal) cost for each additional customer. Software clearly satisfies this definition and then some as the marginal cost for each additional customer/user is effectively zero.

So is software excludable? Well, companies whose business model is the production of software for sale have historically gone to great lengths to prevent people from using software without paying for it; and yet even with significant public monies being spent in helping prevent this illicit use, people continue to make use of all sorts of software they are not supposed to, entirely free. Arguably even the reasonably effective methods that have been used to prevent software being used illicitly, like dongles, are so expensive relative to the cost of the copy of the software that they would automatically fail the excludability test.

So software is non-rivalrous and non-excludable. A pure public good by normal economic definition. Yet it is not how many people currently view it, and much software is sold at high prices to individuals as if it was a normal good. Actually much software is sold prices so high relative to the cost of the copy of the software that it's priced at what economics would view as monopoly prices.

Why isn't software provided as a public good? Why isn't it something we collectively provide? Surely this would be to the greatest benefit of all, economically speaking, and be far more in keeping with the nature of software. Oddly, it is not uncommon for companies who produce software for sale to spend more money on selling, marketing and protecting the sales of the software than they do on making the software better. Economically speaking this makes no sense - it is the attempt to create artificial exclusions that costs so much money, rather than improvements to the product.

So software could be, should be a public good. Come to think of it, wouldn't it be even better, something more than a public good, if software was not only provided collectively but in a way that made it easy for others to improve? Like if the underlying programming code was available and people could fix stuff and add new features and then make the resulting improved, added value version available to everyone else allowing the virtuous cycle to start again. Wouldn't that be a truly amazing thing? Ever more added value for all, the economics are just brilliant; just need a name for the approach and then surely this "available code" software will spread like wildfire...