In this article I’m going to offer some reflections on one of the most prominent failures of data governance of our time; namely, the Post Office / Fujitsu Horizon scandal, and what it can tell us about the importance, for any organisation which handles data, of ensuring it meets the necessary ethical standards alongside its legal and regulatory obligations.
What is the scandal about?
The story is currently so prominent in the media on a daily basis, that I probably don’t need to repeat the details of it here. In short, though, the situation is as follows.
Between 1999 and 2015, a fault with Fujitsu’s Horizon computer software, which was used by the Post Office, resulted in more than 900 sub-postmasters being subjected to false prosecutions for misreported cash losses at the branches for which they were responsible.
The impact of these false prosecutions were profound, as in many cases, because of the contractual stipulation that sub-postmasters assume responsibility for unexplained cash losses, the individuals affected were driven to destitution, illness and in several cases, suicide. In 2019 the UK High Court ruled that the Horizon software was at fault, and a public inquiry was ordered by the government to take place in the following year. The government has allocated £1bn for compensation of the victims, and as of January 2024, £153 million has so far been paid out to successful claimants.
However, despite the discrepancies arising because of faults in Fujitsu’s software, the Post Office and the government itself have been implicated in the scandal, as the reporting by sub-postmasters of faults were not acted on and there are indications that the government was aware of the problem but did not intervene to correct the situation.
The argument about who should be held to account for the damage done continues in an unedifying way between the Post Office and the government at the time of writing.
What is its significance for data ethics?
Where data governance is effective, in terms of meeting both the necessary ethical and legal standards, by definition it successfully anticipates, mitigates, and defuses risks that might follow from the failure of the relevant governance processes. This means that the risk anticipation exercise that constitutes successfully avoiding those harms from occurring can seem somewhat dry, technical, theoretical, hard to quantify and so on. To that extent, good governance could appear somewhat uninteresting: after all, everything going fine is neither exciting nor newsworthy.
To illustrate the importance of this point, we can look at the analogy that the philosopher Mary Midgely famously drew between philosophy and plumbing: both are complex technical systems that underlie different aspects of everyday life, but whose existence we only really register when they develop a fault. As well as the direct analogical relevance, it is also secondarily significant because, of course, ethics is a branch of philosophy, and as such, when done and applied well, ethical thinking will help to ensure that faults – and more specifically, harms to people – are prevented from occurring.
It’s because the successful avoidance of failure or harm through good governance can, on the surface, be uninteresting, that when there is a major failure which causes harm, it’s worth commenting on it and extracting the lessons that can and should be learnt from it. And that’s why, at this juncture, it’s helpful to think about the Post Office / Fujitsu scandal: if ever there were an example that features strongly in the public consciousness which underlines the importance of data ethics, this is it. For reasons that I hope will become clear, it’s worth digging into the ethical ramifications of the scandal to gain some understanding of how and why similar failures should be prevented in future.
Retrospective and anticipatory justice
Presently, it appears that, at last, justice is being done, albeit belatedly. Reparations are being made through the courts, in view of the recognition and acknowledgement by the Post Office, Fujitsu, and the British government that an egregious miscarriage of justice has taken place. Of course, such redress through the courts is both legally and morally required: this is a fortunate instance where ethical obligations are being enforced in law.
However, there is a sense in which the reparations, while welcome and necessary, cannot make up sufficiently for the harm that has been done, not least because, as it turns out, these harms could have been prevented from occurring if the governance procedures had been adequate. When we focus on this, we are reminded of the ethical importance of the anticipatory component of ensuring justice, as well as the retrospective component which has started to address some of the harms caused by the failure of governance.
So, in turn, when we think about why the prevention of harm through effective risk management is so important, we are also reminded that such anticipatory measures are not only legally required, but morally required as well. There must be a moral obligation to attempt to prevent harm as well as to provide redress if it occurs, since if there were not, and we afforded harm no moral weight, there would be nothing to ground the moral case for redress.
To emphasise the seriousness of this point, it is worth noting the unequivocal terms in which even Fujitsu’s CEO has accepted organisational responsibility and articulated the need for redress, acknowledging that the company has a moral obligation, which is to say, as well as a legal responsibility, to compensate the individuals harmed, and the families of those who have died.
When seen in these terms, therefore, we can make a clear case for the moral, as well as the legal, requirement for organisations handling data to ensure that they engage in good governance. This in turn makes explicit what might otherwise be implicit or obscured by regulatory demands backed up by legal force; namely the necessity of ethics in data governance.
Ethics and organisational accountability
The importance of this can be highlighted in other ways, too. For instance, it was, as I mentioned earlier, a stipulation of their contract with the Post Office that sub-postmasters must pay for any shortfall detected in discrepancies between actual and recorded takings. This feature of the contract has been contentious, because in many cases, it was these discrepancies which drove the scale of the failure in terms of bankruptcies, harms to mental health, and deaths.
Of course, the ethical permissibility of such a contractual stipulation is contestable and can be argued in either direction. What is a transparent moral wrong, however, is the continued deployment of the software system without attending to the reported faults, and the inadequate governance infrastructure associated with it that allowed the errors to proliferate, rather than to be detected and resolved. As it turned out, the Horizon software was faulty, and the Post Office organisationally too diffuse, lacking in accountability and oversight. Had these systemic failures not been embedded in the way that they were, much of the harm could have been prevented from occurring.
So, to return to the earlier analogy, in the same way that, for example, ensuring that your boiler is serviced each year can prevent numerous serious plumbing problems from occurring, so appropriately responsible governance would have gone a long way to preventing the harm that occurred here.
To develop this point and strengthen the case for incorporating engagement with the ethical dimensions of data governance into your organisation’s processes, I direct you towards an argument that I made in a previous article; namely, that data ethics is business ethics, or perhaps more specifically, is at the very least a part of business ethics so integral to it that it cannot be excluded or treated in isolation from the operation of the whole, if the whole is to function in an ethically optimal way.
The reason for raising this point here is because the failure that led to the harms in the Post Office / Horizon case was institutional in nature. This is to say, they did not occur only because of one errant malfunction whose impact remained restricted to the discrete area of the system in which it arose. Rather, it was the culture of the organisations involved and the (lack of) communication between them which undermined effective oversight, detection, mitigation and harm avoidance, such that the impact of the malfunction spread and proliferated.
As with many aspects of the scandal that have come to light, the claim here is not just conjecture. It is clear that the Post Office, at the organisational level, failed to or address the highly unusual occurrence of so many instances of the same anomaly, namely cash discrepancies, arising repeatedly in such quick succession. Similarly, despite being aware of glitches in the Horizon software after its rollout, Fujitsu admit to having, literally, no answer, to why such glitches were not addressed.
Again, the harms that these institutional, organisational failures of data governance were largely avoidable. And since avoidable harm is a moral wrong, it obviates the moral obligation of organisations handling data to engage specifically in ensuring good ethical practice in this regard. Moreover, and to emphasise the pragmatic importance of good organisational practice in data ethics, as well as the foundational moral importance, the reputational damage to Fujitsu is likely to be significant. As such, in the interest of protecting the bottom line of one’s organisation, as well as protecting the individuals of whose data it is a custodian, it is worthwhile for any organisation to ensure that demonstrates trustworthiness, such that people are prepared to pay for its services.
What lessons can we draw from this?
So, what can we distil from this analysis, in terms of drawing conclusions about ethical data governance and how to ensure its realisation? I make three suggestions.
First, if there is a moral imperative that miscarriages of justice are overturned and compensated retrospectively when they come to light because of the harm they cause, then there is also a corresponding moral imperative to do whatever can be done to prevent them from occurring or worsening in the first place. We see a similar argument to this operating in the justification for the moral obligation to implement public health measures to prevent harms to health, rather than only to seek to ameliorate those harms after they have occurred.
Second, in any organisation that handles people’s data, the responsibility for ethical practice in data handling is not restricted to a single individual. Although there might be an individual who has overall managerial responsibility for implementing the relevant systems and training that underpin successful data ethics practice, because it is not a part that can be successfully separated from the whole, so accountability has to be spread across the organisation and must be a collective effort.
And, third and finally, to avoid buck-passing in an environment of distributed responsibility, in a way that occurred in the Post Office scandal and led to the harms that arose, the culture of an organisation needs to promote the observance of the behaviour required by the second point. This, in turn, and to reiterate the point made earlier, once again underlines the requirement for data ethics understanding across the organisation, and not only considered to exist in its own silo with risks that it is assumed will not spread to infect the rest of the system.
I hope it’s clear from the preceding analysis why we at IGS consider it so important that any organisation which handles data ensures that it meets the highest ethical standards. If you need assistance or support in making sure that your organisation can and does do this, we are available to help with all aspects of ethical practice in data governance.
