Comments from cphc members on my
Open Letter to my MP
about government IT procurements
Aaron Sloman
Last updated: 10 Oct 2006
A significant new development. A group of senior Computer Science researchers, the signatories to open letters calling for an independent inquiry into the NPfIT Programme, have set up a web site here.

From Tony Tue Aug 29 12:31:16 2006
Content-Transfer-Encoding: quoted-printable
Subject: RE: Isoft fiasco
Date: Tue, 29 Aug 2006 12:31:10 +0100

To: "Aaron Sloman" 

Dear Aaron,

I enjoyed reading your analysis of large-scale government procurement.
But I don't think Lynne Jones will: it is far too long.  Politicians
think in soundbites these days.

Think what you actually want her to do.  Is it to ask for a commission
of enquiry into government high-tech procurement?  Then say so; and
perhaps write a 'terms of reference' for the enquiry.

Yours,

Tony.


From George Coulouris Tue Aug 29 14:09:38 2006
Subject: Re: Isoft fiasco
Date: Tue, 29 Aug 2006 14:08:55 +0100
To: Aaron Sloman 

Aaron,

> Some of those developments might have happened faster if the near
> monolithic power of IBM in the commercial world had not prevented
> deployment of much better alternatives to IBM PCs (e.g. products
> from Sun, Apple, Apollo and other smaller companies using non-Intel
> based cpus, and non-Microsoft software). It's not easy to tell
> whether there would have been the same drop in costs because of
> growth of numbers and the existence of clone makers. (Sun
> Microsystems from the start allowed others to build hardware to run
> their operating systems, unlike Apple, for instance.)

It might be worth mentioning the existence of UK-based alternatives
(Acorn, Whitechapel Computers - there's quite a bit on the net about
the former, but apparently nothing on the latter, though its
designers are still extant), none of which received any significant
government recognition or support, unlike their US competitors
(through defence contracts).

> Some of the history of the ten years from 1900 can be found here.
Should be 1990.

Best wishes,

George

George Coulouris
Visiting Professor in Residence
Computer Laboratory
Cambridge University

[I replied mentioning Bleasdale too.]



From C.Tully Tue Aug 29 15:37:21 2006
Date: Tue, 29 Aug 2006 15:36:20 +0100
From: Colin Tully
Subject: RE: Isoft fiasco
To: Aaron Sloman 

Dear Aaron

Super read.  I only hope your MP finds the time to read it!

You say:  "If there is a national problem that seems to require the
development of a large and complex system, then governments must find a
way to grow our understanding of the problem as we grow the solution,
and thereby grow our understanding of all the detailed sub-problems, in
small steps with a lot of parallel exploration of options."

Does this seem to be a modern restatement of Karl Popper's position in
"The open society and its enemies", about which Bryan Magee wrote as
follows?

"Because he regards living as first and foremost a process of
problem-solving he wants societies which are conducive to
problem-solving. And because problem-solving calls for the bold
propounding of trial solutions which are then subjected
to criticism and error elimination, he wants forms of society which
permit of the untrammelled assertion of different proposals, followed by
criticism, followed by the genuine possibility of change in the light of
criticism. Regardless of any moral considerations ... he believes that a
society organized on such lines will be more effective at solving its
problems, and therefore more successful in achieving the aims of its
members, than if it were organized on other lines."

I'm aware of the mixed and lively reactions that Popper evokes, but I
hope I'm not just being mischievous.

Yours, Colin.

=====


From Aaron Sloman Tue Aug 29 17:40:24 BST 2006
Date: Tue Aug 29 17:40:24 BST 2006
Subject: Re: RE: Isoft fiasco

Responding to Colin Tully

Thanks Colin

> Super read.  I only hope your MP finds the time to read it!

I know it is rather long (one of my faults), and more than one person
has said so, but I wanted to provide examples to liven it up and also
make it fairly easy to read, which ruled out some types of compression.

Lynne Jones has a PhD in biochemistry and is capable of reading and
writing about quite technical topics.

I agree with her on most things -- but not all.

> You say:  "If there is a national problem that seems to require the
> development of a large and complex system, then governments must find a
> way to grow our understanding of the problem as we grow the solution,
> and thereby grow our understanding of all the detailed sub-problems, in
> small steps with a lot of parallel exploration of options."
>
> Does this seem to be a modern restatement of Karl Popper's position in
> "The open society and its enemies", about which Bryan Magee wrote as
> follows?
>
> "Because he regards living as first and foremost a process of
> problem-solving he wants societies which are conducive to
> problem-solving. And because problem-solving calls for the bold
> propounding of trial solutions which are then subjected
> to criticism and error elimination, he wants forms of society which
> permit of the untrammelled assertion of different proposals, followed by
> criticism, followed by the genuine possibility of change in the light of
> criticism. Regardless of any moral considerations ... he believes that a
> society organized on such lines will be more effective at solving its
> problems, and therefore more successful in achieving the aims of its
> members, than if it were organized on other lines."
>
> I'm aware of the mixed and lively reactions that Popper evokes, but I
> hope I'm not just being mischievous.

Not at all. I was not consciously echoing Popper. I am a great admirer
of much of his work (though not the over-simplified versions often
attributed to him, and I think he got some things wrong, including the
emphasis on the discovery of laws as opposed to the discovery of
possibilities -- a key feature of computer science). I had completely
forgotten the bit you quote, though I have often recommended Magee's
little book on Popper (in the Modern Master's series) as an excellent
introduction to Popper's ideas, and to philosophy of science.

Perhaps I'll add a footnote.

Thanks also to the others who have sent me comments.

Jon wrote:
> perhaps we could all send copies or versions to our MPs

Since I have made it public it can be used in any way anyone
desires, though sending a link may be better than sending
a copy, since it may evolve.

One thing I am not yet clear on is what action can be taken
and at what level. One suggestion I've had is to propose
a commission of enquiry into government high-tech procurement.
I am not sure I know enough to do that well.

Aaron
http://www.cs.bham.ac.uk/~axs/

From jon.crowcroft Tue Aug 29 16:06:07 2006
To: Aaron Sloman 
Subject: Re: Isoft fiasco
Date: Tue, 29 Aug 2006 15:24:13 +0100
From: Jon Crowcroft

you're leter is spot on (if a bit long for a non techie, but ought to be
ok if you MP Is clueful)

perhaps we could all send copies or versions to our MPs

i assume you've read the missive
by ross anderson et al

i think is shocking they fell into this trap!
they had several (even government) projects that set good examples...

cheers
j.


From Aaron Sloman Tue Aug 29 18:17:42 BST 2006
To: Jon Crowcroft
Subject: Re: Isoft fiasco

Thanks Jon,

> you're leter is spot on (if a bit long for a non techie, but ought to be
> ok if you MP Is clueful)
>
> perhaps we could all send copies or versions to our MPs

See previous comment.

> i assume you've read the missive
> by ross anderson et al

I've heard of it, but haven't found it yet. It doesn't seem to be
accessible via his web page. I'll try asking google another way. Or if
you have a link you could send me I'll be grateful.

I guess you have now seen his response to me.

> i think is shocking they fell into this trap!
> they had several (even government) projects that set good examples...

I fear this is a government with too many stupid fanatics with a little
dangerous learning near the top.

Aaron



From owner-cphc-membersTue Aug 29 17:27:26 2006
Date:         Tue, 29 Aug 2006 16:37:21 +0100
From:         Ross Anderson
Subject: Re: Isoft fiasco

Aaron

> Comments and criticisms welcome

I've wondered for many years why so many government IT projects fail,
and I'm not sure complexity arguments suffice. They explain some of
the failures, but not why the public sector is so manifestly worse
than the private sector.

I believe you have to consider factors from organisational theory and
public-choice economics. Civil service departments behave the way they
do because of the incentives facing civil servants and ministers.
There is a vast research literature on this; entry points include

   http://en.wikipedia.org/wiki/Public_choice_theory

   http://en.wikipedia.org/wiki/New_Institutional_Economics

   http://en.wikipedia.org/wiki/James_M._Buchanan

This body of literature was mined diligently by the scriptwriters of
'Yes, Minister'. Briefly: people optimise their own utility as well as
that of their employer. Ministers go for soundbites, while civil
servants try to expand their bureaucratic empires. Getting projects
approved often involves log-rolling - putting in features to appeal to
colleagues or political constituencies, rather than for strictly
functional reasons (think of all the things the ID card was going to
do at various times during its journey through Parliament). Dodging
liability, and dumping risk, are deeply entrenched. Ministers often
optimise their career chances by acting as advocates for their
department rather than for the public that elected them. This is all
well-explored by many shelf-feet of books on politics and economics.

There's other stuff on the systems side. Why is it, for example, that
everything from Windows to mobile phones ends up with too many
features, at the cost of both usability and dependability?  Well,
features get added until the marginal cost equals the marginal
benefit. The benefit of each feature accrues to a small vocal group of
customers; the cost is a slightly increased probability of a blue
screen of death or whatever, spread across the whole user base. The
result is a 'tragedy of the commons' in which every programmable
device at equilibrium has so many features that the median user is
just about to throw it at the wall out of frustration. Dependability
is very far from the social optimum: the time spent each year by
people waiting for Windows to reboot may exceed Microsoft's market
cap. For more, see for example:

    http://www.cl.cam.ac.uk/~rja14/econsec.html

How do we combine the insights from political economy and the analysis
of system dependability? This would make a nice PhD thesis for someone
who didn't mind becoming seriously unpopular in Whitehall! There are
various starting points. First, politics in a democracy is about
reconciling conflicting interests, so it often seems wise to keep the
specification of a system vague. Second, as ministers crave publicity,
they need frequent announcements - so they keep changing the spec,
just like the tax code.  Third, as ministers rotate every few years,
promising a system that will solve all current problems, in five
year's time, is a risk-free way of 'doing something'.

It would be nice to have the time to develop this further - there are
no doubt dozens of other factors. But I'm sure the core of it is this:
the way ministers work is exactly at the other end of the spectrum
from how a good project manager works.

If you're building a system to a fixed-price contract, you race to
close down options as fast as you can, to remove uncertainty, to get
the client to make the hard decisions about design tradeoffs and sell
them to his colleagues, to agree not to tinker with anything until you
deliver the software, and to take ownership of the system as you build
it. All of these are at odds with the incentives facing an ambitious
minister.

Now, the dependability literature teaches us that large software
project failures are mostly due to overambitious, vague and / or
changing specifications, coupled with poor communications and an
inability to acknowlegde the signs of failure early enough to take
corrective action. Sound familiar?

This is not anybody's fault. It's just how things are, like gravity or
fog. Government departments can't develop systems well because of the
sort of organisations they are. But that's OK. We don't want to live
in Stalin's Russia or even in Singapore. We've got the democracy we
have because, although it's awful, everything else that people have
tried has been worse.

Ross



From Aaron Sloman Tue Aug 29 22:52:20 BST 2006
To: cphc-members
Subject: Re: Isoft fiasco

Responding to: Ross Anderson

Thanks Ross,

(various bits of your message snipped):

> I've wondered for many years why so many government IT projects fail,
> and I'm not sure complexity arguments suffice.

I am sure you are right that there are different factors. Similar points
to yours about how governments and their employees work were made by
some of the commercial/industrial people on the local linux users group
mailing list, on which I invited comment.

I suspect, however, that even if all the factors you mentioned were
somehow fixed there would still mostly be disastrous failures in
decade-long multi billion £/$ IT projects, whether paid for by public or
by private funds -- for the reasons that I gave, namely reasons
concerned with the inherent complexity and unpredictability of the
problem-solving/design/development processes, and the impossibility of
producing in advance sets of *requirements* that will be seen to be good
ones at the end of the project -- partly because of the new insights
that will have been gained from the project, whether successful or not,
and because of other changes in technology, user knowledge and
expectancies, etc.

> They explain some of
> the failures, but not why the public sector is so manifestly worse
> than the private sector.

One factor may be the relative infrequency of such large long term
expensive IT projects in the private sector?

> I believe you have to consider factors from organisational theory and
> public-choice economics. Civil service departments behave the way they
> do because of the incentives facing civil servants and ministers.
> There is a vast research literature on this; entry points include
 ...

Thanks for all the pointers.

> Dodging
> liability, and dumping risk, are deeply entrenched.

Yes: I see that in decisions taken at university level by administrators
and central computing system managers, who take decisions where the
needs of teaching and research are given lower priority than avoiding
some remote legal risk, for which they might be blamed. ('Covering
backs'.)

> Ministers often
> optimise their career chances by acting as advocates for their
> department rather than for the public that elected them. This is all
> well-explored by many shelf-feet of books on politics and economics.

What I've seen in recent years suggests that an additional factor can
operate: blinkered fanaticism of ministers (and prime ministers) who see
themselves as the saviours of the nation (or something bigger), and
think they really do know the truth whereas the voters, judges,
commentators, house of lords, etc. have all got it wrong, but will
eventually see the light... etc.


> There's other stuff on the systems side. Why is it, for example, that
> everything from Windows to mobile phones ends up with too many
> features, at the cost of both usability and dependability?  Well,
> features get added until the marginal cost equals the marginal
> benefit. The benefit of each feature accrues to a small vocal group of
> customers; the cost is a slightly increased probability of a blue
> screen of death or whatever, spread across the whole user base. The
> result is a 'tragedy of the commons' in which every programmable
> device at equilibrium has so many features that the median user is
> just about to throw it at the wall out of frustration. Dependability
> is very far from the social optimum: the time spent each year by
> people waiting for Windows to reboot may exceed Microsoft's market
> cap.

I would not disagree with your general point, and I don't often defend
Microsoft, but having seen and cursed windows 98 and after that windows
2000 over the years, running on my wife's machine (because there's a
map-making program she can't do without which requires windows), I
noticed a huge change for the better when she got a new machine running
XP. Of course there may be some disaster in the offing as she has only
had it for a few months.

(I still wouldn't use Windows myself as it's far too rigidly constrained
for me, and I prefer other ways of dealing with newly detected
deficiences, risks, etc. than MS update.)

> For more, see for example:
>
>     http://www.cl.cam.ac.uk/~rja14/econsec.html
>
> How do we combine the insights from political economy and the analysis
> of system dependability?

If my argument is right it cannot be done in *any* 10 year monolithic
project whose goals and even approximate costs and timetable are fixed
in advance.

>...
> It would be nice to have the time to develop this further - there are
> no doubt dozens of other factors. But I'm sure the core of it is this:
> the way ministers work is exactly at the other end of the spectrum
> from how a good project manager works.

But no project manager, however good, could overcome the problems I
listed, especially if some of them can only be solved by switching
development teams and companies (and project managers?) in unpredictable
ways -- to take advantage of some newly developed expertise.

> If you're building a system to a fixed-price contract, you race to
> close down options as fast as you can, to remove uncertainty, to get
> the client to make the hard decisions about design tradeoffs and sell
> them to his colleagues, to agree not to tinker with anything until you
> deliver the software, and to take ownership of the system as you build
> it. All of these are at odds with the incentives facing an ambitious
> minister.
>
> Now, the dependability literature teaches us that large software
> project failures are mostly due to overambitious, vague and / or
> changing specifications, coupled with poor communications and an
> inability to acknowlegde the signs of failure early enough to take
> corrective action. Sound familiar?

If my arguments are right, among large, long term IT projects, only
trivial, non-innovative ones can possibly succeed, and even most of
those will fail simply because they produce something as out of date as
their original requirements specification.

> This is not anybody's fault. It's just how things are, like gravity or
> fog. Government departments can't develop systems well because of the
> sort of organisations they are. But that's OK. We don't want to live
> in Stalin's Russia or even in Singapore. We've got the democracy we
> have because, although it's awful, everything else that people have
> tried has been worse.

I don't know of any government that has tried the approach I sketched
(which Colin pointed out is very Popperian), which treats every large
long term project as inevitably being not just a design and
implementation project, with pre-specified requirements, but also a
major national, or possibly international, *learning* process, in the
course of which requirements, knowledge, technologies, possibilities,
opportunities and constraints will all change in unpredictable ways,
alongside whatever designs and implementations are produced, and will
change partly *because* of that production process.

However, you may be right that the institutional and cultural pressures
in governments and the civil service will prevent the benefits of such
an alternative approach being achieved.

Since the recommended approach allows for a substantial subset of
projects to be axed relatively early, while the knowledge gained remains
publicly available, it may be that (a) the total amount of wasted money
will be much smaller (b) and all those failures may nevertheless
indirectly contribute to other successes, in the way that the internet
grew out of learning processes and many failures.

Aaron
http://www.cs.bham.ac.uk/~axs/



From Aaron Sloman Tue Aug 29 23:02:50 BST 2006
To: cphc-members
Subject: Re: Isoft fiasco [PS to Response to Ross]

A thought has just struck me.

Perhaps we could reverse the old idea of requiring an
*industrial* uncle for academic research projects, and require an
*academic* nephew/niece for every government-funded project
placed with a private company?

The main task of the academic would be to analyse and report
publicly on what was learnt through that expenditure, in addition
to contributing in other ways, if appropriate.

He she would have to be paid *entirely* from public funds and
debarred from owning shares in or receiving any financial
benefits from the company for several years!

Of course, there are messy details that could undermine this.

Something similar was suggested by someone on the SB linux users
list commenting on my letter.

He also wrote

>  Private Eye No.1164 17th August page 6 has some serious comment
> on this and other wastes of taxpayers money.

I have not seen that.

Aaron
http://www.cs.bham.ac.uk/~axs/


From owner-cphc-members Wed Aug 30 17:14:19 2006
Date:         Tue, 29 Aug 2006 18:21:34 +0100
From:         "Michaelson, Greg"
Subject: Re: Isoft fiasco
Comments: To: Ross Anderson


Of course we don't know about failures of similar sized projects in
theprivate sector because they're not subject to the same degree of
publicscrutiny. So we don't know whether it's the project size or the
publicsectorness that is significant. And given the shared Tory &
Labourobsession with outsourcing we don't know whether in-house groups
woulddo any better or worse than private consultants. I suspect that
theprofit motive is a major factor in out-source failure as it
forcescontract holders to minimise their own costs by cutting corners in
orderto keep their shareholder sweet.

Best wishes

Greg

________________________________



From owner-cphc-members Tue Aug 29 21:23:20 2006
Date:         Tue, 29 Aug 2006 20:34:57 +0100
From:         Yorick Wilks
Subject: Re: Isoft fiasco
Comments: To: "Michaelson, Greg"

Im sure Greg has put his finger on it: the failure of companies driven
by a profit motive with shareholders (Google, Microsoft etc ) compared
the successes of all those good old nationalised corporations without
either. Are we the old two left still mourning the loss of Post Office
Telephones and the 6 month wait for a home line?
Yorick Wilks

On 29 Aug 2006, at 18:21, Michaelson, Greg wrote:

> Of course we don't know about failures of similar sized projects in
> the private sector because they're not subject to the same degree of
> public scrutiny. So we don't know whether it's the project size or the
> public sectorness that is significant. And given the shared Tory &
> Labour obsession with outsourcing we don't know whether in-house
> groups would do any better or worse than private consultants. I
> suspect that the profit motive is a major factor in out-sourcefailure
> as it forces contract holders to minimise their own costs by cutting
> corners in order to keep their shareholder sweet.
>  
> Best wishes
>  
> Greg
>


From G.Michaelson  Wed Aug 30 09:25:17 2006
Subject: RE: Isoft fiasco
Date: Wed, 30 Aug 2006 09:23:29 +0100
From: "Michaelson, Greg "
To: "Aaron Sloman" 

Aaron

My email was exactly as quoted by Yorick.

Best wishes

Greg

-----Original Message-----
From: Aaron Sloman [mailto:A.Sloman@cs.bham.ac.uk]
Sent: 29 August 2006 21:55
To: Michaelson, Greg
Subject: Re: Isoft fiasco

Hi Greg,

I have seen your message as quoted by Yorick, but for some reason did
not get the original.

Is the bit quoted by Yorick complete?

> Of course we don't know about failures of similar sized projects in
> the private sector because they're not subject to the same degree of
> public scrutiny. So we don't know whether it's the project size or the
> public sectorness that is significant. And given the shared Tory &
> Labour obsession with outsourcing we don't know whether in-house
> groups would do any better or worse than private consultants. I
> suspect that the profit motive is a major factorin out-sourcefailure
> as it forces contract holders to minimise their own costs by cutting
> corners in order to keep their shareholder sweet.
>  
> Best wishes
>  
> Greg

If not could you re-send me the whole message please?

It's possible that it's temporarily jammed somewhere on Janet and
will find its own way here later, but I thought I should make
sure.

Incidentally, there are some known commercial failures, though not of
the same size. One example was the GEC-63 computer in the 1980s.

Thanks.

Aaron

From owner-cphc-members Wed Aug 30 06:21:20 2006
Date:         Wed, 30 Aug 2006 04:22:00 +0100
From:         Jon Crowcroft
Subject: Re: Isoft fiasco
Comments: To: "Michaelson, Greg"


it is worth remembering that two major software houses with track record
in very large scale systems integration did NOT bid on this project (IBM
Is the obvious one) - the contract was placed in such a way as to put
them off (from what I;ve learned talking to people there)  - microsoft
didnt bid even though some aspects of the GP and ward systems would have
been something they could have done plausbly (though the image
archive/exchange system might have been beyond them since data bases
aint there thing)

unfortunately, from talking to medics on the frontline in testing these
systems, I have to say that some of the failures start to smell of
something worse than simple large systems project failures through
incompetence, and look more like bordering on actual dodgy behaviour
(for example, legacy databases on immunisation were discontinued before
accepteance tests for replacements had been passed, despite complaints -
amidst finger-pointing amongst the various sub contractors - I doubt
anyone will get done for criminal incompetenece (leading, possibly to
unnecessary deaths) but they ought to be...

I think a more constructive thing for us to do would be to point (as
some of Aaoron's letter does) at ways not only to do this right, but to
repair the programme of work that is, at least, partly getting there  -
federation was used in the Welsh system, as well as some smart social
engineering using incentives for local services to use their national
databases (as disaster recovery backup for example) - I never understood
why the UK system was divided into such large pieces (and why it wasnt
grown out of existing trusts' bottom up attempts to build IT for their
own management)

there are other pieces that may never work (e.g. proper privacy with the
way the cradle to grave paitent record is envisaged) but the components
like the x ray and other image stuff, and the prescription system ought
to be seperable tractable projects even at the scale pictured (actually,
i dont see why they are a whole lot harder than the dvla and passport
systems, which are (after some teething problems) ok, let alone the
inland revenue which is slowly getting there with online tax returns etc
etc of course the merger between inland revenue and customs and excise
has yet to play out, and should provide some "amusement"...

I'm not quite sure why databases with 60M objects keyed on one value,
with a few kbytes entry per person should be considerd dificult in these
days where laptops ship with 100GByte drives, but there you go...

yes, private projects fail too - lots. and they dont have to be IT
based, just complex....anything (as ross so rightly says) with a lot of
humans stakes held is subject to all sorts of gremlins and pookahs...

 cheers

   jon

From rjp Wed Aug 30 09:22:38 2006
Date: Wed, 30 Aug 2006 09:22:22 +0100
From: Rob Pooley
Subject: Re: Isoft fiasco [PS to Response to Ross]

Absolutely, Aaron. I have touted the idea for several years that the
best way to support software engineering research, which always has a
problem with being able to observe and experiment on realistic systems,
would be exactly the requirement for all large public sector projects to
award funding to academics for this purpose. It is, in effect, how some
research gets grafted on to EU projects.

Rob

From owner-cphc-members Wed Aug 30 10:18:26 2006
Date:         Wed, 30 Aug 2006 10:16:46 +0100
From:         Steve Linton
Subject: Re: Isoft fiasco

Many excellent points from Ross. I especially like:

>  But that's OK. We don't want to live
> in Stalin's Russia or even in Singapore. We've got the democracy we
> have because, although it's awful, everything else that people have
> tried has been worse.
>
Interestingly, from what I have picked up from a superficial reading the NHS
IT stuff has, by government standards, been put together relatively well.
There is a publically visible non-political long term project manager, whose
personal utility is fairly directly aligned with the success of the project.
He has let the primary contracts to a couple of large, well-financed
world-class integrators (IBM, Accenture and CSC are in there, I forget the
details) who get to make most of the medium-level requirements decisions,
carry most of the financial risk and get to structure the contracts with the
actual software providers. More or less equivalent software is (or was) being
developed in parallel by two companies (iSoft and someone else) for these
contractors, so if one developer goes belly-up they can just buy the other
package, and it's STILL in danger of falling apart (essentially because both
software developers may fail).

    Steve

--
Steve Linton    School of Computer Science  &
      Centre for Interdisciplinary Research in Computational Algebra
         University of St Andrews    Tel   +44 (1334) 463269
http://www.dcs.st-and.ac.uk/~sal     Fax   +44 (1334) 463278


From owner-cphc-members Wed Aug 30 12:12:06 2006
Date:         Wed, 30 Aug 2006 11:00:19 +0100
From:         Ian Marshall
Subject: Re: Isoft fiasco

Aaron

I am firmly convinced that almost none of the issues are unique to IT

However, there is one significant issue that does appear to be software
specific - it is the widespread belief of people who do not develop
software, that software can be changed at no significant cost.  One easy
way forward is to remind politicians and managers that any creation
requiring human effort, including programming, will require additional
human effort (on a scale at least comparable to what was required to
create it in the first place) to make significant changes.  We do not
expect plasterers to work for free if we change the design for the
mouldings after they have made them, and nobody should expect software
builders to be any different.

Poor estimation is the characteristic of all artisan based activity (think
builders, plasterers etc) and is more associated with poor understanding
of hidden risks than with size or complexity. Greater use in software
projects of estimation and risk techniques widely used in the construction
industry would doubtless be useful.

Your recommendation to build understanding by use of small scale solutions
that get incrementally improved and integrated is good.  There still needs
to be an occasional large scale refactoring (analagous with IPv6?), but
this should only happen in the light of operational experience with a more
cobbled together solution, and with the involvement of the end users.
IMHO too much of what we build (as a society) is designed by "experts" who
are divorced from the reality of the environment their creations will
inhabit.  The result is products that do not take account of place
(geography), culture, human occupants/cohabitants/users, etc when
designing solutions.  As this "brutalism" is a generic problem (affecting
much more than software) we will not fix it by offering solutions to IT
issues alone.  I personally think planners and beureaucrats have promoted
brutalism because it is easier than the (usually) messy business of
consulting the people who will have to live with the end reults.  As
people are starting to realise easier does not necessarily mean cheaper.
One approach (an effective use of academic expertise) to enlightening the
decision makers further might be an authoritative quantitative study on
whole life costs comparing products created with a user centric approach
to products created with a brutalist approach.  Actually this seems so
obvious it must exist already (but possibly not for software).  Does
anyone have pointers to studies of this kind, that could be precised for
governmental audiences?

Regards

Ian


From owner-cphc-members Wed Aug 30 13:58:56 2006
Date:         Wed, 30 Aug 2006 13:57:45 +0100
From:         "Bernard Cohen"
Subject: Re: Isoft fiasco

Responding to Ian Marshall's message.

Nobody in this discussion group seems to have noticed that the issues being
raised are precisely those associated with 'systems of systems', particularly
in the miltary and aerospace fields.
I realise that drawing a comparison between military and healthcare demands may
be anathema to some, but I suggest that texts such as D. C. Alberts' 'Power to
the Edge' describe dilemmas faced by both military and healthcare enterprises,
and by their suppliers.
In both, there is a need to design both products and organisations so that the
latter can 'orchestrate' the disparate services provided by the former, in
response not only to fantasies elaborated by those at the 'centre', or 'top'
(politicians, staff officers heathcare administrators, etc.), but to
unanticipatable demand situations encountered by embodied individuals at the
'edge' (soldiers and clinicians as well as non-combatants and patients).
Systems of systems do not admit analysis using classical systems theoretic (nor,
therefore, computer science) models (see, e.g., Doug Norman of Mitre Corp). They
need a framework in which the pragmatics of individuals with different desires
and the semantics of systems with different capabilities may be expressed,
composed and critically analysed so as to reveal the 'holes' that they will
have to deal with if their enterprise is to be sustainable.
See www.brl.com for a thorough discussion of these issues and proposals for
their resolution.

Prof Bernard Cohen, Dept of Comp Sc, City Univ, Northampton Sq.
London EC1V 0HB   tel: ++44-20-7040-8448 fax: ++44-20-7040-8587
WWW: http://www.soi.city.ac.uk/~bernie
"Patterns lively of the things rehearsed"


From George Coulouris Wed Aug 30 14:06:36 2006
Subject: Re: Isoft fiasco
Date: Tue, 29 Aug 2006 19:59:37 +0100

Aaron,

I ought to have said in my previous message that I agree with almost
every word of your analysis and wish I had thought of it myself, it's
been too easy to just sneer at the many government project failures
without thinking the problem through as you have now done.

Here are a few further thoughts:

Some very large scale software developments in industry have
succeeded (banks, airlines, ...).  The ratio of successes:failures is
much higher for large industrial projects. These successes are part
of the reason for the politician's misplaced confidence in the
software industry. The reason for them ought to be explained.
[I wrote that before seeing Ross's comment along similar lines.]

Your reference to the history of the internet is excellent, but it
almost makes it sound like an isolated case, whereas we know that
without ARPA + industry -funded open research and development, other
key developments such as timesharing, interactive graphics,
distributed systems and personal computing would have been stunted or
drastically delayed.

Of course there were plenty of ARPA-funded projects that never saw
the light of day. And such examples don't address a key difference
between the US and UK. In the US the electorate has enough faith in
technology to allow large amounts of taxpayers money to be spent on
advancing it without clear applications.

My intuition is that you're right about the need for incremental and
open development, but more thinking needs to go into the process by
which it can be structured to satisfy guardians of the public purse.
The term 'deliverables' comes to mind, but we all know how wrong-
minded that is. Working, useful systems at each stage in the process,
even if they don't address a very ambitious set of requirements seems
more like it. But then how would big leaps ever be taken? Maybe only
in research projects.

Best wishes,

George

George Coulouris
Visiting Professor in Residence
Computer Laboratory
Cambridge University



From owner-cphc-members Wed Aug 30 15:32:55 2006
Date:         Wed, 30 Aug 2006 14:36:59 +0100
From:         Ross Anderson
Subject: Re: Isoft fiasco

Aaron

I don't think it would ever occur to a private-sector company to do
something like NPfIT. I have been involved in private death-march
projects too, the worst being a 200-person, 3-year project to move a
bank's core systems from ICL to IBM mainframes which ended up taking
400 people 5 years and helped cost the bank its independence. That's
about as big, and as bad, as it gets in business. In man-year terms XP
was slightly bigger, and Vista bigger still, but the user lock-in
gives Bill some assurance of commercial success, even if the product
ships a year late. Also, when Microsoft ships a new product it's the
result of an evolutionary process which gives them a huge amount of
specification and masses of test cases from day 1.

NPfIT is one or two orders of magnitude bigger; the only comparably
sized private-sector engineering project of which I'm aware was when
Boeing gambled their company on the development of the 747. Given that
they were already making the 707, 727 and 737, that was a much more
tractable problem. They knew how to make fuselages, wings, engines,
hydraulics and so so; they had test pilots and airworthiness
certification procedures. It was evolution not revolution.

NPfIT has many other problems. It's not driven by the users but by
civil servants, who fear doctors and envy the systems they have built
for themselves - all of which are to be ripped and replaced. The
complexity of these systems was not understood, and the users' support
(or even acquiescence) has not been won. NPfIT is predicated on the
idea (and driven by the desire) that all medical records should be
gathered in from the multitude of GP PCs, hospital servers and filing
cabinets where they currently lie, and made accessible to the centre
for a multitude of secondary purposes (Sir H says 'research' but he
means audit, cost control and various kinds of surveillance). This is
rightly resisted by doctors because of patient confidentiality and
professional autonomy.

I did a survey for the National Audit Office which compared the health
IT spend in the UK with that in the USA and a number of European
countries:

  http://www.cl.cam.ac.uk/~rja14/Papers/nao-report-final.doc

which showed that the UK is a real outlier. Other developed countries
are building federated systems: the centre limits itself to developing
and promulgating interoperability standards, which then get built into
local systems as they're upgraded or replaced. Over time, things like
electronic prescribing start to work over more and more of the country
- without huge disruption, expenditure and risk. And given that the UK
is 5% of GWP (and heading towards 1% as India and China develop) there
is no choice but to use the market leading systems. The UK can't
deevelop the best systems for cardiologists, diabetologists, and all
the other specialities. We will end up using international standard
software just as the NHS already uses Microsoft Word. We will thus get
health services that use standard, federated systems, like it or not.
NPfIT will thus have the same sort of effect as the attempt by BT and
Racal ten years ago to get doctors to use X.400 rather than SMTP: it
held up the adoption of email in hospitals by five years. For more in
this vein see

  http://www.theregister.co.uk/2006/06/13/letters_1306/

  http://politics.guardian.co.uk/publicservices/story/0,,1860168,00.html

In short, the NHS is building the wrong system for the wrong reasons
and it deserves to fail.

Ross

From owner-cphc-members UK Wed Aug 30 17:40:19 2006
Date:         Wed, 30 Aug 2006 10:13:56 +0100
From:         "Michaelson, Greg"
Subject: Re: Isoft fiasco

In a similar vein, we still don't know what effect if any the Millenium
Bug had in the private sector, again because of secrecy.

It's curious that we don't demand the same degree of transparency from
the private sector as from the public sector, even though they are now
easily the largest provider of services and beneficiaries of public
expenditure. In particular, no one seems to point fingers at the makers
of these truly awful one-size-fits-all systems. Instead they just get
given more time and money, and the public sector victims of PFI get the
blame. For example, why are the endemic inadequacies of University
finance and student record systems always the fault of poor old
Computer/Corporate Information Services rather than of the software
suppliers?

Like Yorick, I miss decent nationalised industries. Yes they were
wasteful, but that's probably the price of mass provision at a uniform
standard for all. I'm just back from a long weekend on the Atlantic
fringes of the South West Highlands: terrible mobile phone coverage but
lots of red telephone boxes. One of the reasons nationalised industries
gave better service was because they did things in house. My first
employers, the old Scottish Gas, had fleets of engineers in vans who
seemed happy to stay until a problem was fixed. Now you get a time slot
and an arbitrary contractor of highly variable competence, and when you
complain to the energy provider they disclaim responsibility. Again,
many Universities, having privatised their cleaning, have gone back to
in-house provision because it's cheaper, more flexible and gives better
service.

Rant, wheeze...

Greg

PS privitisation = turning into a hedge fund


From Jim Hunter Mon Sep 04 13:38:05 2006
Date: Mon, 04 Sep 2006 13:27:24 +0100
To: Aaron Sloman 
Subject: Re: Isoft fiasco

Aaron

Good on you! - interesting to see the others' comments.

This may be anecdotal, but I've been told that the management of IT
change in the NHS here in Scotland (which of courses is devolved) is
much more incremental (and hence more successful). This was from an
IT manager in NHS Grampian.

Jim


Prof. Jim Hunter
Department of Computing Science
University of Aberdeen
King's College
ABERDEEN
AB24 3UE
UNITED KINGDOM
http://www.csd.abdn.ac.uk/~jhunter/

TO BE EXTENDED

Maintained by Aaron Sloman
School of Computer Science
The University of Birmingham