Justice versus Conscience

The real scandal is that the court automatically assumed that the digital data presented is correct and trustworthy. That is the main topic of our times: Are our digital systems trustworthy enough to count as evidence in court? Perhaps Computerphile should invite Ross Anderson again for some basic security engineering lessons. 😉

It’s a disgraceful story. Distributed transaction processing with two-phase commit was sorted and implemented in the mid-1980s. If systems today are failing the ACID test it’s due to incompetence by the system designers.

You got the “isolation” in ACID wrong. It doesn’t mean different parts of the system are isolated from each other, it means concurrent transactions don’t interfere with one another.

These kinds of bugs are remarkably easy to get. I had to fix a bug in a client’s webshop that would sporadically get way more stock in thewebshop than they actually had. Turns out there was some sort of denial of service-ish thing where robots where trying to reserve as many items as they could. But ofcourse if you never actually purchase then the system will remove your shoppingcart and return the items to stock. Turns out that if people where reserving items while it was getting returned to stock the deleting of the cart would fail and a little later the same cart would be deleted again and presto; more stock than you started with. That disappeared when I implemented transactions. And yes I notified the author of the shop and basically told him to go and learn how databases work.

This was an absolute scandal and ruined people’s lives. The power the post office had under an ancient law to privately interview the accused in police stations supposedly under caution and hide other cases from them was egregious and compounded the problem. My heart goes out to the victims who have suffered from these baseless accusations. I recommend all SW engineers and system designers listen to the BBC and guardian podcasts on the subject to appreciate how important it is to get these things right and show true due diligence in their profession. Don’t be afraid to call out issues you might see in any systems you work on. Anthony (semi-retired SW engineer).

This whole story warrants a lot more conversation in the software industry. Going forward it should be required reading alongside the Therac-25 scandal. It also demonstrates the issues with blindly trusting computer programs without any compassion for real people. The many stories of those who were prosecuted, fined, jailed, some of them dying before their verdicts were overturned, are heartbreaking.

I am pleasantly surprised by how many times I enjoy watching videos of the British postal system. Thanks to Tom Scott for introducing me.

This was huge, people lost their livelihoods and were accused of theft and fraud and every case wrongly won by the post office against the sub postmasters was a gross miscarriage of justice. There was a woman local to me who’s even own family thought she was taking money and this is a perfect example of when you should know your users. There was also some evidence of the post office and Fujitsu the supplier knowing that there were issues and remained silent while people went to jail and in one case ended their own life. (Allegedly)

This is an example of why all software that touches public funding should be free, and, in the case of internal government systems, should be released to the public. Being able to audit Horizon would’ve given people a fair defense in court. (See the FSFE’s “Public Money, Public Code” initiative.)

I’ve had a problem similar to this before, a meal payment transaction system would charge you without checking to see if the order had actually been successfully received. Bad code does show up often in the world.

Closed source, poorly audited, faceless unaccountable/uncontactable software devs. Execution by Software rather than Executable Software.

I see everyone in this comments section is commenting on the (shameful) scandal itself, but I wanted to take a moment to say that this is also a great video explainer! Clearly explained so that all the main issues are understandable to even the slowest of us… great job, both Professor Murdoch and Computerphile!

The behaviour of the Post Office was atrocious. Has anyone been held accountable?

Good video, definitely want to see more stuff about databases, even if it is historic information i.e. development of ACID. Steven seems like a good presenter so look forward to more from him.

I’m surprised this sort of thing doesn’t happen more often. Once you replaced the mainframe guys who were very, very careful about ACID compliance with a bunch of Java guys in the 90s and 00s who had never heard of ACID correctness and then tried to get it to work across multiple, disparate systems with no global transaction rollback was sure to break somewhere.

A perfect example of how we can all expect to be treated as “infallible” machines and algorithms control more and more of our lives.

The explanation of isolation wasn’t exactly correct. It is more about concurrent transactions, and the need for serializability (i.e. order of operations doesn’t change final outcome) regardlessly of whether they are on the same system or not.

A lot of managers, executives, and lawyers working for fujitsu and the post office should be spending a looooooooong time in prison for conspiracy to pervert the course of justice over this

Let me guess: This system cost hundreds of thousands or maybe millions of pounds to be developed?

Please tell me a massive scandal isn’t going to be the exact same problem used to introduce concurrency problems to freshman CS students.

A great explanation of the Horizon ‘logical error’. My heart goes out to those that suffered as a result.

As a DBA, the gross incompetence — from bottom to top, tools to management — disgusts me.

thanks for this, this is exactly what i wanted to learn about – what bugs causes what issues. poor people who went to jail and lost their jobs

Some people in very high positions knew about the mistake that put inocents in jail. Life sentences for those psychopats would be a reasonable start to making them atone for it.

I followed this via Private Eye and as a computer science graduate I was always very suspicious of the guilt attributed to the people prosecuted. It just didn’t make sense that people were brazenly robbing the Post Office of money in the manner that they were supposedly doing so. Also it should have been fairly straightforward to look at the personal accounts of the defendants and seen either more cash being deposited into their accounts or them spending less from their accounts due to more cash in hand. Of course the argument could have been made that they gave the money to others but this would be unlikely. One of the biggest problems was that the prosecution hid from the defence the fact that there were many other postmasters and sub-postmasters who were being prosecuted for the same issue, so the defence could not then see the commonality in it being the computer system that was at fault. Thank you for this video because it shows just how complex transactional systems and are how difficult they are to maintain in sync – it reminds me of the Two Generals’ problem which was also excellently covered on Computerphile I think.

Nobody mentioning the video quality of Steven Murdoch? Looks so good!

This scandal destroyed people’s lives!

This is a wonderfully calm explanation of the technical failures behind an absolutely outrageous scandal.

Keeping this system running this long was criminal!

I worked for a company where the accounting software had a variance of about $2.49 between the net assets and the total equity. They must have had a weakness at some point that allowed an unbalanced transaction. To get that fixed we would have had to give the file back to the software provider. We just lived with it as it wasn’t material.

As a SWE I have to say we need more regulations in SWE field. Until we treat software like a nuclear reactor, we are going somewhere very very wrong!

Absolutely mindboggling I can absolutely believe that people got locked up for this, though Lots of arguments can be made for why software should be prohibited from prod haha

Computerphile is so great. Bringing light to these kind of things.

One of the most shocking scandals I have heard of. I hope those wrongly accused get the justice they deserve.

It seems like this was a kind of perfect storm of novel technology. In 1999, the internet was definitely established as a viable commerce platform, but hasn’t reached the level of maturity where you could expect software like Horizon to be implementing basic standards of design competence. It’s up to humans to eagerly question the technology when inconsistencies arise. And never assume malice what can be attributed to incompetence.

So the Postal Service bought untested software from the lowest bidder, and assumed it would work perfectly? Then it never seemed strange that shortages appeared that weren’t there before the new software was installed? This is so incredibly incompetent that it sounds like something that would happen in the US, and that should be embarrassing. I’m amazed people were convicted on the evidence from a new and untested buggy system. If I was a person affected, after I was done suing the Postal Service, I would sue the F@C% out of Fujitsu because they obviously covered up their pathetic system problems while peoples lives were being ruined because of it.

Famous IT-saying: “Avoid duplication of volatile information”.

The designers and maintainers of the system itself should be facing trial and be held accountable for this. I say that as a software developer and researcher. You make software that deals with serious problems, you better be damn sure you give it serious consideration. The story is remarkably similar every time the government contracts out a software project, low skill SWEs, ridiculous prices, long lasting contracts – perfect kickbacks.

they hung out loads of postmasters out to dry over this

Does anyone know what happened to the Horizon system? Is it still in use (presumably patched) or has it been replaced?

Yes computers always count correctly, but people program computers and people make mistakes all the time. The fact that someone would come out and say this software is like a tank is just stupid. I hope those who have covered this up will themselves go to jail

“Computers don’t make mistakes”. True, but programmers do.

If these mistakes were a random wouldn’t they have been equal number of cases where the postmaster was in surplus? If so what did the post office do about that? If that was not the case it implies the existence of some corrective mechanism that was built to only work in One Direction.

Lesson learned: If there is a way to do something wrong, government will do it the wrong way almost every time

developers! listen and learn very carefully ^^ up to date, even banks screw up transactional logic and its not that hard tbh (if you pay attention from the start)

I read an essay about this. It made me wince loudly. My God, when I think about AI and how we’ve convinced so many people that algorithms and other AI technologies are seemingly infallible, I get shivers. We may see similar cases in the future on that front.

Could you please enable Auto-Generated English Closed-Captions? Even if it’s not perfect it’s really useful for non-native English speakers. Thanks

I learned these things in my third semester of computer science! How can such a large corporation make such awful software?

Were innocent people sent to jail? This is terrifying. I hope justice prevailed in the end and the innocent were exonerated to the fullest and properly compensated — though I have no idea how Britain’s tort laws compare. But given history, I have my doubts. It’s not too far a stretch to equate this disaster to the modern reliance, and unmerited trust of AI and machine learning systems. How long before some AI flags some innocent as a ‘potential malefactor’ that must be pre-denied, pre-ostracized, pre-cancelled, or otherwise denied their right to exist, just because some AI said so?

Steven Murdoch great guy he did a great talks at ccc about chip and pin.

Agile approach can sometimes foster these situations because it discourages solving “future problems”, and it happens more often than anyone will admit. Experienced engineers can mitigate such problems but they sometimes have to fight an uphill battle to do the right thing.

I work in the UK public sector in the IT industry, this is an episode I’m gonna have to miss, it will just make me too angry

As a former bookkeeper this sounds like a technology failure compounded by a bookkeeping failure. All of these shortages should have been easily catchable with daily or even weekly checks.

So big question is: Who bought this shitty system? I’m not even specialist in these systems, but I saw so many wrong things (synchronizing logs? What? Why? It’s point for logs to be untouchable so you can fall back on it if something goes wrong). If you think about it, banks do pretty much similar stuff, arguably on even bigger scale, these systems should be worked out since forever… Feels like someone was trying to save some money, or someone contracted a new company to give a friend nice cozy job to me. And people paid for it…

So this is what happens when you accidentally program an accounting database to act out the plot from Superman III or Office Space.

Remember everyone, lives were actually lost thanks to this scandal. And you can blame not only Horizon but also senior post office managers who allowed this to happen. The company needs to be sued beyond imagination and people need to go to jail. Incompetence has led to irrepairable damage and even loss of life

I worked in IT in the 1990s, not for the PO or Fujitsu. It was an open secret in the industry that the system was an utter shambles. The BIG question is that The Board of the PO rejected the distemper in September as having major issues that they could not implement it. However, 4 weeks… 4 WEEKS later they approved it and signed it off… WHY? I smell corruption and this may explain the 2013 cover-up.

One of the issues with writing robust code is the difficulty in simulating hardware or network failures. Bugs might only show in very specific circumstances, and recreating these circumstances during QA or UAT is a rare skill. That’s besides for the Heisenberg Uncertainty Principle as applied to computers: Putting software through a debugger often changes the software (or the way it runs) just enough to hide potential failures, making fixing bugs much more difficult.

Prof Stephen might be the best dressed and we’ll framed guests on computerphile.

Selling stamps? Not the crazy stamp collecting machine? Seriously though, it sounds like it was way too common to be a real thing and should make the post office consider it might be their system’s fault. I’m surprised I hadn’t heard about this, this level of stuff usually comes to our news as well. Sounds like absolutely terrible and usual government level software and system purchase.

I wonder if the software was old enough to have been written on an ICT 1900?

I would love to know, when Horizon was put out to tender (I assume it was a bespoke system written for the Post Office) how many bids were there and why was the winning bid chosen? Who was the person\consultants who created the system specification, did these person/s have an established track record and did they hang around for long in the project? What was the timeline on the project and did it rollout in phases? Did the project have test criteria that covered all possible use cases? How many changes did Post Office Management make to the project mid-stream? Were there changes of Project Management and were there disagreements between IT and Post Office non-IT management? I suspect somewhere in all of this some non-technical person got some great hooraa for a totally botched product but it also wouldn’t surprise me if the system supplier took existing system code that wasn’t suitable and used this as a basis to fast track the creation of something that would appear to tick all the boxes just for the sake of winning the contract.

The post office management should be sent to jail for this!!!

Now remember how many critical systems run on things like COBOL and MUMPS/Cache, and that many of the original developers are dead at this point…

There was also a legal assumption built in to legislation that computers did not error.

Wow. So the basic architecture of the Horizon System was not up to the task.

Such a simple bit of software. And they screwed it up. In this day and age?

To err is human, but to really foul things up requires a computer. How true those words are.

Breaking all 4 parts of ACID is quite a feat.

731 Absolutely shocking, disgraceful, the arrogance and hubris of the management. Someone needs to go to prison.

I always wondered why British gangsters claimed to rob post offices. I always thought that was petty and juvenile.

Wait wait wait wait… how is concurrent logging a hard problem? Aren’t multiplayer video games with 64 people essentially a big concurrent log, updated 60 times a second? That doesn’t sound like a hard problem.

So did real money go missing? My bank balance is a complex number: there is a real part and an imaginary part.

Accountancy made interesting! It’s a miracle!

The black background makes it look like the speaker is explaining the British post office in the void of intergalactic space

Can you do a video on the Microsoft Print Spooler vulnerability “PrintNightmare” ?

This is Skynet, version 0.0.1. When people trust computers more than they should, don’t understand how they operate, and end up harming other people as a result.

We had a similar problem with the library’s computer system at my elementary school, a book would be marked late or not returned even when we personally handed it to the librarian.

No system is better than its programmers who built it. Seems this was a shitty programming from the beginning and this is just horrible if people got blamed left and right. Poor organization just spiral things. This doesnt suprise me…but poor employees. Management should be in jail for this crap. They were incompetent for sure and probably overpaid. It goes hand in hand.

On the logs, you can encrypt the logs. That restricts access. Secondly you can sign each log entry, with a crypto check sum to prevent modification. Then you have the question of adding and removing log entries. Here you need to generate a crypto sequence of numbers, where the generation from the previous to the next isn’t easy, if you don’t have the key. Then you can check for deletions and insertions into the log

So they went 4 out of 4? wow what was going on in there

This is why cryptocurrency is based on a distributed ledger. You can’t just jam the entire world into one ACID database.

So, as a layman, I guess this is what blockchain is supposed to fix?

Couldn’t listen to him for long, but he muddled through okay.

Hal 9000 :- “It can only be attributable to human error”  Horizon:- “Hold my beer…”

“What’s PostgreSQL? Best practices? It’s like you’re talking another language!”

The fact that a Person went to jail because the government does not update an ancient computer system is unacceptable. Shame on the law people involved on this case.

In a nutshell, this is pretty basic and lame. Stock reconciliation is a problem for all systems – everywhere, due to the transient nature of goods and basic stock management. This video assumes, in defence of ALL PO counter staff, that post offices staff do not ‘pocket’ money. This is simply not true and there is evidence for that. Post office balance sheets are often wrong, the processes needed updated because this is either mismanagement or theft, right? Horizon tried to do that, by making the counters ‘accountable’, but as there are tens of thousands of ‘counters’ processing millions of daily transactions and its very easy for post office staff to steal money, make routine mistakes and of course investigations are very labour intensive to investigate. Horizon did that investigation with very smart technology. People just don’t like getting caught with their ‘fingers in the till’, so no matter how much wittering on this laddie does, he’s missing lots of details on how the Horizon System actually works. Of course, everyone is entitled to their opinion these days, but that doesn’t make it true, or right and on this occasion he is very wrong about so many things.

Horizon need to not only be sued but people need to go to jail. Negligence and incompetence caused lives to be impacted, some permanently so …….

so was everyone accused a victim of bugs in the software then ?

Double entry book keeping, not rocket surgery….

Nice try but computer science doesn’t explain the ongoing cover-up.

classic case of companies skimping on QA

Would blockchain contract technology solve alot of the issues in this video?

Top video answer just what i was looking for

So the moral of the story is don’t run a franchise for the government.

What!? Damn. The UK post office speked into civil service hard lol

It seems to me that some kind of blockchain could solve most of these problems.

Computerphile we need subtitle, please at least turn on the auto cc subtitle.

Effects of crapy software!

I think the videos of this channel can be more presented. Im interested in the subject of this video, but i cant understand eveerything of it because it was presented in a boring way (just talking). Consider maybe putting more images and demonstrating how the things gone.

So how much of the ACID test did you fail? Horizon Systems:”All of them(?)” “¯\_(ツ)_/¯“

abso outra horizon still in use,maybe the fatcats can now stop licking the cream

What was the podcast? It’s not in the description.

anyone else read the title as “after” + (microsoft) “office”? something to do with after using MS office?

So what exactly is wrong with the Horizon systems? Horizon Systems:”Yes”

why is lemonad stand sell stamP??

People served time in Jail for this. The coders should be in jail now!

Oh look, a video where the audio and video quality isn’t complete Zoom garbage!

What the hell is a lem needs stall? Ive never heard of that before.

COBOL ?

In the UK people can be jailed based on no evidence except a computer printout. Think about that !

These guys ever heard of the Block-chain????

Let me save you 16min: Accounting software was bad because it didn’t keep track of things properly. That’s all this video is, with a few examples of what the things it didn’t keep track of properly were. I’m finding Computerphile videos more and more shallow recently, and increasingly devoid of anything approaching technical explanations.

love from Nepal 🇳🇵

START TRANSACTION;

Seventh.

29th.

the seventh first?

Uke

Can the BBC be trusted to report anything correctly these days?