The story of Visa charging a number of customers $23,148,855,308,184,500 has been all over the news the last couple of days. Slashdot commenter rickb928 provides a plausible explanation for the error.
I work in this industry. The only novelty here is that the error got into production, and was not caught and corrected before it went that far.
Submitters send files to processors which are supposed to be formatted according to specifications.
Note I wrote ‘supposed to be’.
Some submitters do, from time to time, change their code, and sometimes they get it wrong. For instance padding a field with spaces instead of zeros. Woopsie…!
Seems that’s what happened here. Sounds like a hex or dec field got padded with hex 20, and boom.
This is annoying, especially when the processor gets to help correct the overwhelming number of errors, and then tries to explain that it wasn’t their fault. Plenty of blame to go around with this one.
And then explains why they don’t both validate/sanitize input, and test for at least some reasonable maximum value in the transaction amount. A max amount of $10,000,000 would have fixed this. That and an obvious lapse in testing. This is what keeps my bosses awake sometimes, fearing they will end up on the front page of the fishwrap looking stupid ’cause their overworked minions screwed something up, or didn’t check, or didn’t test very well. I love one of the guys we have testing. He’s insufferable, and he catches genuine show-stoppers on a regular basis. They can’t pay him what he’s been worth, literally $millions, just in avoiding downtime and re-working code that went too far down the wrong path.
Believe me, this is in some ways preferable to getting files with one byte wrong that doesn’t show up for a month, or sending the wrong data format (hex instead of packed binary or EBCDIC, for instance) and crashing the process completely. Please, I know data should never IPL a system. Tell it to the architects, please. As if they don’t know now, after the one crash…
If you knew what I know, you’d chuckle and share this story with some of your buddies in development and certification.
And pray a little.
At least it didn’t overbill the cardholders by $.08/transaction. That would suck. This is easy by comparison. Just fix the report data. Piece of cake. Evening’s worth of coding and slam it out in off-peak time. Hahahahaha!
That’s quite a missed test case!