Thursday, January 17, 2008

Statistics: Getting Your Sums Wrong

As one jester said, 85% of statistics are made up on the spot. That was certainly the case for the numerous crimes against the taxpayer in which I was a minor accomplice. The second Department of Social Security project I worked on in 1982-1986 was allegedly the second biggest IT project in history, second only to the computer projects which supported the Apollo moon shots. The original estimate was £700 million (over a billion dollars even then). Needless to say, it climbed and climbed to over £2 billion by the early 1990s, so God alone knows what it finally cost.

One particularly telling episode was the frenzied week when the decision was taken to reduce the number of mainframe computer centres from seven to three in a desperate effort to make the cost/benefit figures add up. I was frantically multiplying the original estimates by 3/7 or 7/3 in one of the crudest recalculations you can imagine. But any questions of value for taxpayers' money and justifying the investment evaporated as time went on. When one of the senior bullshitters was summoned before a Parliamentary committee on public expenditure, he unblushingly explained the original £700 million estimate as "an academic exercise". The original "seven centres" network design was influenced by the 7 regional controllers in DSS who wanted an IT centre in each region; any region without a centre was perceived as second-class. It is a measure of the almost limitless contempt in which the public administrators of this country hold the taxpayer that they are able to indulge in such expensive power games.

Another (mercifully minor) project was justified on the basis on a 10% saving of existing clerical costs. This was a figure I had plucked from thin air, with not a shred of solid evidence to support it. It looked modest enough to be credible and large enough to make the discounted cash flow calculations deliver a worthwhile return after 7 years. The whole project never reached the 7 year mark, being abandoned after a trial at one site. Plainly no one in government has learnt anything from these fiascos as hundreds of millions continues to be poured down the toilet on aborted or inadequate civil service IT projects year after year. The most monstrous outrage at the moment is the National Health Service computer strategy which will cost anything from $20 billion to $60 billion, depending on which estimate you believe - or more likely don't believe.

The old jibe about the truthfulness of statistics (Lies, damned lies and statistics) has an extra edge in counting Christians. The recent revelation that Catholics had overtaken Anglicans in British church attendance loses some of its lustre when you think about the quoted figures. Both were less than 900,000. Out of an alleged 4 million Catholics and 40 million Anglicans, the figures ranged from bad to abysmal. Far from any cause for Catholic celebration, they underlined the long term decline in both Churches' influence. As I said in my previous post, the fact that Catholic church attendance is relatively healthy is largely due to the East European invasion we have enjoyed since 2004.

The Polish invasion has revealed some of the glorious confusion in head counting any large group. According to one figure I have seen, there are nearly one million new Polish arrivals in the UK out of a total population of 60 million, i.e. less than 2%. Yet if you take the figures town by town, it looks rather different, nearer 5 to 6%. The moderate sized town of Crewe in north-west England had a population of 45,000 until 3,000 Poles arrived. Most of them were recruited by a local employment agency whose owner confessed that he could not believe what he had done to his home town. The ocean port of Southampton, with 300,000 people, has allegedly 15,000 to 20,000 Poles. Reading, with around 250,000 people, has 8,000 to 14,000 Poles. These figures look a trifle imprecise, especially when you realise that both estimates appeared on the same page of the "Reading Chronicle" and were offered by two people with allegedly intimate knowledge of the local Polish community. All you know for certain is that three Polish shops have popped up in a half mile stretch of the Oxford Road.

From the Polish viewpoint one advantage of Reading is the long-established Polish church of the Sacred Heart and the Dom Polski social club. They are not ideally placed for the new arrivals, being in the east/centre part of Reading while the immigrants are clustered heavily along Oxford Road is on the west side. Up to 2004 Sacred Heart had two masses each weekend and a declining congregation as the post-WW2 migrant generation who did not return to a Communist Poland slowly died out. Now it has 4 masses each weekend, all packed to the doors. Yet curiously its church attendance is not included in the diocesan directory figures which contribute to the calculations of the national total of Catholics which were quoted above. If you add in the numerous Polish churches, and other national minority Catholics around the country which are similarly excluded, the true figure is probably considerably higher than 900,000.

Dom Polski tends to cater for the older generation of Poles (plus any number of English/Irish customers), so the newcomers have launched "Ecce Homo" (Pontius Pilate's words on presenting Christ to the crowd) as a social/educational society to attract young Poles, educate Poles about England and the English about Poles.

Oxford Road has seen wave after wave of immigrants; Jewish, Irish, West Indian, Pakistani, Indian and now East European. Some linger in the crowded, but cheap side streets off Oxford Road, most move on to more prosperous suburbs. A multi cultural collection of buildings bears testimony to this complex mix; the Irish Centre is 200 yards north of Oxford Road, the synagogue is 100 yards off Oxford Road and the first purpose-built mosque in town is being built right on Oxford Road. The steel skeleton of its minaret dominates the nearby Methodist church and the restaurants of several nationalities which line the road.

The church attendance figures, flawed as they are, provide the main basis for decisions on diocesan administration. Some of the truly bizarre proposals in the plans for reorganising our Portsmouth diocese seem to have arisen from "management by spreadsheet" where distant bureaucrats carved up and regrouped parishes based purely on their attendance figures. Sacred Heart, bursting at the seams, was not even mentioned in these plans. But it is not only church attendance figures that entice Catholic leaders into highly suspect initiatives. A current fad backed by allegedly startling figures is a pretty good bet for launching clerical careers.

Nearly 20 years ago I attended a talk by a priest who was advertised as the diocesan expert on AIDS, which was the main threat to the human race at that time. His main qualification for the job seemed to be a willingness to swallow and regurgitate every terrifying statistic promoted by the AIDS industry. Among his prophesies was the assertion that in 5 to 10 years we would see people with Karposi's sarcoma on their faces on the buses and in the shops on a daily basis. Needless to say, I have never seen a single person anywhere with Karposi's sarcoma - except in Hollywood gay propaganda such as "Long Term Companion" or "Philadelphia". I have also never seen a single diocesan expert on amoebic dysentery or malaria or TB, all much bigger health problems than AIDS......but then I have never seen a Hollywood star wearing a shit coloured ribbon to advertise his burning compassion for amoebic dysentery sufferers. These people have the unforgivable bad taste to have an unchic fatal disease which doesn't afflict celebrities. And I would not be surprised to find that our diocesan AIDS expert has now moved on to global warming activism armed with a fresh set of terrifying statistics. AIDS is so five minutes ago, especially as barely 0.1% of the UK population suffer from it, despite widespread chaotic promiscuity.

That talk on AIDS, incidentally, was unforgettable for a ferocious intervention by the retired doctor sitting in the row in front of me. He was probably irritated by a non-medical guy lecturing on his turf, but his venom was fuelled mainly by the utterly secular and rationalist attitude to the disease displayed by the priest.

"What the church should be teaching, Father, is that sodomy is mortally sinful. That fornication is mortally sinful. That if a virgin man marries a virgin woman, they will not get AIDS...."

The priest was visibly startled by the force of this call to morality and was belatedly and unconvincingly forced to concede the importance of the Church's teaching on sexual behaviour. It was plain that for him the only real mortal sin was being judgemental i.e. truthful. It is difficult to decide what was most shocking about the old doctor's comments: his total lack of respect for the clergy, his complete lack of knowledge of the precepts of Political Correctness (unforgivable in a man of his education) or his use of clear unambiguous English.

Education is a major area of contentious figures. The "league tables" of secondary schools provide statistics by the sackful on the exam results of every school in the country. Like the church attendance figures above, they are mostly a herald of dismal underperformance, especially given the ever increasing sums poured into state schooling. The exams for 16 year olds (GCSE or General Certificate of Secondary Education) must be among the most derided qualifications ever promoted by any major education system. The basic "satisfactory" standard is that a pupil gets a grade C in 5 subjects, including English and Maths. But according to some credible reports, you need get only 32% or 36% to get a grade C. In other words a bad failure in any other exam system is enough to get you a Grade C, with still lower marks getting you a pass at D or E grades (i.e. abysmal failure). Thus the league tables for the best local schools look like Albanian election results in the Stalinist glory days, with over 99% of the electors voting with joy for Enva Hoxa. Reading, Kendrick and Abbey Schools regularly get 97 to 100% of their pupils with at least 5 GCSEs at grade C. Of course, this Stalinist 100% includes their dumbest pupils with the basic 5 "C"s and their best with 11 or even more Grade A*s. So the statistics conceal as much as they reveal. The fact that you have an A* grade as well as the basic A is grim testimony to grade inflation.

As you descend the tables, like descending the circles of Hell, you get schools with 50%, 40%, even only 20% getting 5 C"s. You can only imagine the moral of staff in such places. The John Madejski Academy was recently launched in the grim South Reading housing estate of Whitley, replacing an earlier massively failing school (Thamesmead, formerly Ashmead). Despite an investment of $50 million on gleaming new buildings, IT facilities, etc, it is plainly still one of those schools where the "middle classes would rather eat their own livers" than let any child of theirs step inside. The JMA hit the national press recently as one of the 5 worst schools in the country. It must be mortifying for local multi-millionaire and philanthropist John Madejski who also bankrolls the local soccer team, various museums and art galleries. But, as a very successful businessman, he probably reflects that you win some, you lose some. The very worst in town (Phoenix College, previously Reading Alternative School) is not even on the tables for obvious reasons. It is always a really bad sign when a school or business changes its name and even worse when it is changed twice. No wonder that a parishioner at Christ the King church, across the road from the JMA, works a second job cleaning toilets at midnight to send his daughter to a private school.

A more scarey figure for education is the percentage of pupils who carry knives. Plainly the league tables don't carry this more useful piece of information. My cynical take is that the percentage of knife carriers is in inverse proportion to the GCSE results. So Reading, Kendrick and Abbey have zero slashers while JMA and a few others have 20%+. I have no statistics or metal detectors to back this hunch, but I would sooner not use my vital organs as a knife detector. So I and most other adults ignore the more abominable misbehaviour by underclass youths, on the buses and elsewhere.

The problem with statistics is that they are a basis for decision making at all levels of private industry, local and national government. One of my acid-tongued, but extremely well informed friends at church tells me of one policy document circulating in the civil service in London which basically recommends writing off the lowest 30- 50% of the British population - those below social classes ABC1. Decades of ever increasing investment in education have produced consistently disappointing results and it is not clear that further increases would create any better returns. Needless to say, I want to get my hands on this paper to see what sort of recommendations it is proposing, but obviously no one in authority would ever admit to entertaining such lines of thought for a second or sponsoring such Darwinian thinking. Given some of the utterly appalling proposals coming from the "elite", such as near-compulsory organ donation, it will be interesting to see how long it takes such ruthless thinking to surface into the public domain.