« I have a new Book! | Main | Hiatus ... »

How I got here in the end, part nine: the little start-up that could

The Red Queen's Race: "you've got to run just as fast as you can to stand still," as the Red Queen told Alice. That's what it's like all the time when you're working inside a startup venture that's actually going somewhere, or so it seems to me.

In late 1997, I signed on as a contractor for Gavin and Dave's new venture. Little did I know just where it was going ...

Picture this: you're sharing a tiny office — about the size of two cubicles — with one other guy. (He smokes; you don't. Workplace smoking hasn't been banned yet. Live with it.) You know the company has got about three to four months' revenue in the bank. There's a guy down south, trying to sell skeptical punters on the idea of this string-and-duct-tape demo you're trying to jury-rig.

The demo is a large glob of Perl that talks to an msql database (MySQL had yet to show up) and a web server. Customers — retailers with a NatWest Streamline merchant account who've talked the bank into letting them experiment with taking credit cards over the web — throw transactions at your script, which calls up the bank over an ISDN connection, gives them a credit card number and some miscellaneous other gunk (Merchant ID, Terminal ID, card expiration date/issue number, debit amount, phase of the moon) and expects to hear an authorization code or a bugger-off message. Results of these transactions go into the database, whereupon an offline process, coaxed into life manually about once a day, generates a batch of transactions which can be uploaded to the bank for processing. The demo is balky, very much hard-wired to deal with the eccentricities of the NatWest system, and periodically chokes and falls over (in no small part because it was thrown together in two months flat by someone who doesn't really understand what they're doing, yet).

DataCash went live in the run-up to Christmas 1997, handling transactions for about half a dozen merchant storefronts. The public didn't exactly stampede to stress-test my software; in the first week we had about four or five test transactions submitted by the merchants ... and one bleeding-edge maniac who was prepared to trust the keys to their bank account to my software.

Nevertheless, it was a qualified success. "But we need something a bit more robust," said Dave, and I agreed; I could already see that this was too inflexible and lumpen to do what we needed. We needed to be able to talk to multiple banks. And to process transactions for multiple merchants, with multiple terminal IDs per merchant, simultaneously — turns out that the banking systems didn't like concurrent transactions on the same terminal ID, and because we were providing a web service we had to be able to handle concurrency. Somewhere in the mix we had to be able to provide refunds, and to cancel transactions that hadn't been processed (not the same thing at all).

Piece of cake. Not.

1998 was a painful learning experience. It turns out that while the British banks all adhered to the APACS protocols for exchanging financial transactions, the protocols in question were prone to, ah, flexible interpretation — and each and every one of the banks interpreted them differently, seemingly in order to lock out their competitor's suppliers' EPOS terminals.

Gavin (understandably, with a decade of added perspective) had begun over-selling DataCash from the get-go. He was not only the CEO, he was the salesman and main investor. And if a customer said "can it do X?" His automatic response was to say, "sure!" and then pick up the phone to Dave to say "I sold it to this guy, he wants it to do X — make it so." Even if feature X contradicted feature Y which we'd implemented the week before. I don't blame Gavin for doing this — we needed the money — but I wish he'd kept in touch with the real capabilities of our software a bit more closely. On the other hand, this is the sort of thing you get when your CEO is a smooth-tongued arts graduate who doesn't get the technical stuff but knows how to sell.

Another huge problem in the early days was that the banks hadn't got a clue about this internet thing. NatWest, by far the most forward-looking bank in early 1998, had a strict policy: they'd treat internet transactions as cardholder-not-present, or telephone credit card transactions; you only got permission to do that stuff if you had a two year solid gold track record. This led to some amusing incidents. (Allegedly, a NatWest call centre got a phone call one morning. "Hi, this is Fred. We want to start selling over the internet. Can we have a merchant account?" To which the call centre operator checked Fred's records and said "sorry, but you only began taking credit cards last year — we can't authorize that." Whereupon the crisis escalated to emergency boardroom level by lunchtime — because the Fred in question was the head of IT at Marks and Spencer, the biggest retailer in the UK.) And it led to other misunderstandings, too.

In order to get DataCash approved for connection to a banking network, we had to supply the bank with a qualification checklist for our shiny new EPOS system. At which point, we had trouble ticking the right boxes:

Q: "Does your EPOS terminal run over POTS or ISDN?"

A: "Neither: it uses a dedicated X.25 circuit to go direct to your mainframe."

Q: "Does your EPOS terminal have an LCD display or use a CRT screen?"

A: "Neither. It's a headless server running on a UNIX system."

Q: "Does your vehicle have two, three, four, or more than four wheels?"

A: "It's a hovercraft ..."

Now, one of the truisms of software engineering as a field is that version 2.0 of any program sucks. And I freely admit — as its designer and programmer — that version 2.0 of DataCash's server software sucked. Second System Effect usually cuts in as additional requirements are loaded onto a software system that just about worked the first time round, causing it to bloat, slow down, and providing space for lots of lurking bugs.

DataCash 2.0 had feature creep in spades. It needed to talk to different banks who had creatively mangled a "standard" into near-unrecognizability. And it was stuck on top of a SQL database. DataCash 1.0's database was thrown together in an afternoon, fixed format and inflexible; for DataCash 2.0 I tried writing a storage class to abstract the working code from the underlying data repositories. I wrote high level classes to reflect the various banking protocols, with the goal of subclassing them to handle the specific mutilations imposed by each institution. There was a front-end sever, to be embedded in the outside of our firewall, which customers could talk to using a simple protocol (and a perl module to encrypt the transaction details); a back-end server on the inside would handle the actual sensitive job of decrypting transactions and talking to the banks. I was about halfway through implementing this pile of object-oriented spaghetti when Gavin began selling it to customers, and actually expecting it to work. Not good ...

I nursed it along through early 1998, patching bugs and adding functionality — the first extra feature worth mentioning was support for Barclays Bank, if I remember correctly. The money was coming in, hand to mouth, but barely enough to offset the payroll and rent bills. In spring of 1998, David got the go-ahead from Gavin to hire another programmer; it was becoming clear that we needed someone who could write a web-based reporting front-end for the customers, so they could see what they were paying for (and being paid by their customers). Dave and I had the job of wading through job applicants' CVs; that's when we ended up hiring Sam Kington, sometime jazz pianist, philosophy student, and the guy who got the customer-facing web interface working. (Hi, Sam!)

Midway through 1998 we moved ... to a bigger room in the same early Victorian pile; the architects weren't doing well, and was obvious we'd need the space to hire more programmers in due course. My health wasn't holding up well. The start-up death march is bad enough, without second-hand smoke; I was getting frequent chest infections, and having to sit next to an open window during a Scottish winter so I could breathe would take its toll on me the following winter. Things were pretty tense. Gavin was selling the program, Dave was talking to irate customers and sorting out their configuration problems, and I was bug-fixing and patching, nursing the sick system along — it had been pressed into service probably a whole design cycle too early — and in my copious spare time trying to jump through certification hoops with the Royal Bank of Scotland.

Now, on the subject of bankers and information technology: there is a reason why I do not engage in internet banking.

Banks are traditionally (forget all the credit default swap voodoo nonsense) in the business of managing risk. You are a bank. Somebody deposits $100 with you, for safe-keeping. You promise to pay them $2 in interest after one year. To obtain this $2, and cover your operating costs — say, another $2 — you need to invest some or all of the $100 you've been given (or whatever sum of capital you can leverage with $100). Typically you do this by lending out money at an interest rate of, say, 5%. This is the critical bit: your profit depends on the probability of the recipient of your investment defaulting on the loan. If you are good at risk management, none of your borrowers default, and you make a 5% return on the investment. If 20% of them default (bad news!) you make only 4% overall — 80% of them don't default, after all, and pay you 5% — and just break even. If you make some really bad judgement calls and lose your shirt betting on toxic assets in the sub-prime crisis, maybe 50% default and you make only 2.5% — which means you make a loss.

Consider this mind set for a moment. From this angle, banking is all about controlling risk. If you make a bad loan, that's a lot worse for your business than not attracting potloads of depositors. So, bankers: people who are risk-averse by profession.

Now, in British retail banking in the 1980s and 1990s — this is a whole different kettle of fish from investment banking — there was a well-understood career ladder. (This may no longer be the case; times change. But I I am cheerfully ignorant in this regard right now, banking not being a terribly attractive career right now.) You would join the bank as a clerk, and as long as you kept your nose clean and avoided risks that could screw the parent institution you would be shunted from job to job every couple of years, promoted along the way. (The sideways moves were to some extent to prevent the staff getting too close to customers; the days of the traditional bank manager as pillar of the community, who knows everyone who banks with their branch by name, are long since over.) The goal of every aspiring banker was to rise to the level of branch manager. But you don't get to be a branch manager until you've done everything and are familiar with all the niches in the organization. And one of those niches — from their point of view — is the IT department.

I haven't worked in a banking IT department, but I had to deal with them on a day-to-day business for a couple of years. These were people who, for the most part, would make the technical management of Imperial Merchandise look hep and cool. These were not technologically savvy people; they were aspiring rural bank managers who had been unwillingly dragooned into temporarily supervising a basement full of mainframe programmers in three piece suits, and they were clearly marking time until they could make their escape.

There were exceptions, of course. I learned to recognize them and obtain their desk telephone extension numbers by any means possible. The one person in the department who actually wanted to be there, enjoyed the work, rose to the challenge, and could do in a day what would otherwise take six months: the jewel in the turd of banking IT. They exist. Alas, they're elusive and hard to locate, and when you find them they're chronically overworked — because nobody else has a clue what's going on.

But for the most part, Banking IT staff — at least, in the 1990s — didn't have a clue about the internet. I'm pretty sure some of them hadn't even heard of the internet. And in consequence, their reaction to a phone call from some guy in a dot-com wanting to jump through the certification and approval hoops to connect his EPOS server to their network varied from bovine placidity ("whatever, dude"), to frenetic confusion ("we'll have to set up a committee to discuss the relevant criteria for approving connecting your — what did you call it, can you spell that for me? — S-E-R-V-E-R to out network") to panic ("aieee! Hacker cooties are coming to get us!")

(To be continued.)




"Midway through 2008 we moved " - typoNit, ITYM 1998


Thanks - fixed.


Incidentally, in case it wasn't obvious: there's going to be at least one more article about DataCash, and another article about what happened afterwards and how I fell off the startup-monkey climbing frame for good.

But there may be some delays. This Thursday I'm off for a long weekend, and probably won't be blogging for a week.


To nitpick your sums Charlie: you are treating defaulters as though they only defaulted on the interest they owed, rather than on the capital too.


Astrolabe: whoops, true. D'oh!

Yes. But my general point about risk management still stands. Right?


I was trailing in your wake a bit -- In '96 I cobbled together a shopping cart system for my wife's virtual children's bookstore.

(good part: people don't try to defraud places that primarily sell children's products, and when they do, it's obvious: ordering 20 copies of Harry Potter to ship to Romania sends up red flags. Bad part: we really wanted to sell to the YA market but teens don't have their own credit cards and don't want mum shopping for them)

Our business was small enough that we found it OK (annoying, but OK) to work with a POS terminal, and later with a Verisign virtual terminal, without ever getting around to integrating the virtual terminal service with our app. Saved on fraud since every transaction got squinted at too. And led to upselling because we'd be able to say, "Hey, that's out of stock -- we haven't charged your credit card, but there are these four alternatives and which would you like?" and often they'd say "Oh! All of them!"

But what we really realized, especially with your talk about risk is that the credit card companies take on virtually no risk except for non-payment of the credit.
* Card is stolen? Chargeback to the vendor.
* Customer denies purchasing something? Chargeback to the vendor.
And so on. It's then up to the vendor to prove the transaction was valid (no, I don't have a signature from the customer), and even if you do, you're still out a $35 and up chargeback fee (rather nasty on an $8 transaction). And when we did find an obviously fraudulent transaction (we basically threw out just about every order shipping to Romania, Nigeria and Indonesia -- especially on an American card), there's no way for a vendor to turn 'em in. You'd think there'd be a bounty.

Flame off. But it still smolders.


EPOS is one of those fields in computing which has (had, possibly) its own weird language (as in terminology, not as in computer 'language') so that I would get CVs from people who had worked in EPOS and not be able to understand what the hell they meant. EDI was another one, I think we once were desparate enough to recruit someone that we interviewed an EDI person. Sadly, without an interpreter the interview got nowhere.


It's been almost painful reading this series. It parallels my own experience with a dot bomb in the '90s in so very many ways while at the same time reminding me how blazingly, agonizingly, terrifyingly FUN the Internet and everything to do with it was back then.

Thanks for the trip down memory lane!


Charlie @5. I agree. Your point about risk management is strengthened I would say. Bankers have to be strongly risk averse or they would soon run out of money, and who knows what might happen then?


You should be really glad you aren't trying to sell smart startup ideas to banks these days. You see nowadays startups are all SaaS and cloud computing and other jargon than basically means that all the data is stored on some server in some data center in Outer Mongolia (or North Carolina). This tends, how shall I put it politely? not to go down well with banks who have this rather firm belief in the concept of data remaining inside the bank's firewalled data center.


FrancisT: there is a comic pratfall along those lines coming in the final installment (pre-full-time-writing career, that is), but I'm not going to give the game away. Let's just say, my penultimate business card (the one before the current one that just says "writer") had my job title down as CTO ...


Charlie, I've got one of those business cards too (actually it says CIO - but its the same idea). Oddly enough it's my penultimate one too and it was another concept that didn't quite fly...

But that company wen south because we were trying an escape route to (European) chemical companies that were having their lunch eaten by Asian competition. Unfortunately the risk averse people who run European chemical plants seemed to prefer the idea of sticking to their high cost knitting while the firm goes gradually bankrupt rather than risk trying something new that might rescue them but might also hasten bankruptcy if it doesn't work.


Your thumbnail banking sketch has been done in variable detail levels in other places; I am no banker but offhand I can recall you missed reserve requirements, relending, loss of principal and interbank money borrowing/lending from the picture of what it is that bankers do that usually makes them pretty stodgy and conservative. Don't forget stingy too.


I think all startups need a "Gavin" type person. Let's call mine "C" (and it really wouldn't surprise me if he was the one who trusted his bank details to you; he was that sort of early adopter).

C was a bubbly guy; would be willing to sell anything. We were theoretically a web publishing company (a subdivision of larger publishing organisation), but we also partners with Sun, Cisco and BT so we could sell you your own in-house dev environment and publish to our servers. He almost committed us to doing the ATM networking for a large London show, even though we had no ATM experience and would basically be relying totally on Cisco.

We had just won the contract to do the Channel 4 website... when upper management (from our parent company) said "hey, we're not in the hardware selling business! We're shutting you down". Channel 4 were willing to fund us as a start-up, but I looked at the numbers and decided it was too risky. C was very unhappy with me; he wanted to take the whole team (5 people, including him) with him, but because of my sums the rest all said "no" as well.

That was my opportunity to be a startup millionaire; I played safe and turned it down :-) If I'd have forseen cheaper co-location and co-hosting thus obviating the need for expensive leased lines, then I might have reached a different conclusion.


"Hacker cooties are coming to get you" needs to be on a t-shirt.


dev nit: MySQL premiered in 1995, by early 1997 it had support in PHP even.


NotTheBuddha: yes, MySQL premiered in 1995. But we didn't move DataCash to run on top of it until 1998. Hint: we wanted a lightweight database ... but we wanted one that we knew well enough to trust, and we'd been working with msql in one capacity or another since 1993.


I think all techies have come across sales people like Gavin, myself included. But having had a go at the non-techie roles when I was trying to build a business from a one-man-band, I think having a Chinese Wall of semi-ignorance of the products or services being sold is actually an important part of having the confidence to make sales. As the techie, chef, cook and bottle-washer, I was all too aware of the potential problems to be able to inspire whole-hearted confidence in potential customers at times.