Documents as the single source of truth - Telephony Edition
Paul Hammant has an interesting article up about The document as the single source of truth - which made me think about How We Do Things.
Aeons ago, we wrote the first version of our telephony platform, and it was State Of The Art, and It Was Good. It had turbochargers and coffee-grinders, and a honking huge database into which we dumped all of our data, but first the data was broken up so that it fit well into various tables and columns and it was fast and well thought out and It Was Good.
Well, maybe Not So Good, because pretty early on we realized that it would be useful if we had some record of changes - when a user calls up and sez. "I never turned on Call Recording", its useful to be able to say "Why yes sir, you did indeed, it was on the 17th at 3:30pm, and by the way, Yes, we *are* Big Brother thank you very much".
So we added _changes tables that logged the updates to user's addresses, and profiles, and parameters, and, well, pretty much everything. Which was great, except that the more we logged, the bigger a nightmare it was to keep track of what changed when. The reason, of course, was that whenever any data was entered into the system, we took great to break it apart into chunks so that the appropriate data went to the appropriate place. User's email here, account settings there, phone numbers way over there, etc. etc. But trying to recreate these changes, and god-forbid, figure out an audit-trail of what happened when, aaaargh. And thus it came to pass that It Was Not Good.
Fast forward to a few years back, when we rewrote our entire platform from scratch, but this time it was all erlang and web-scale and distributed and cloud and It Was Good.
And, *this* time when we got around to persistence, we decided to leave all the incoming data well enough alone. The original documents - JSON - were dumped into a bigcouch backend (couchDB! w00t!), where we had a variety of views that we used to get at pretty much anything we need. A view for email addresses, another for account settings, yet another for phone numbers, etc., etc.. Old data never gets deleted, the documents just get versioned - giving us a pretty spectacular audit-trail for most everything that goes on in the system, and trust me, when you're doing anything with telephony, audit-trails are key.
Its ridiculously effective, its useful, and it makes our life vastly simpler.
In short, count me in on the "Document as the single source of truth" bandwagon…
Well, maybe Not So Good, because pretty early on we realized that it would be useful if we had some record of changes - when a user calls up and sez. "I never turned on Call Recording", its useful to be able to say "Why yes sir, you did indeed, it was on the 17th at 3:30pm, and by the way, Yes, we *are* Big Brother thank you very much".
So we added _changes tables that logged the updates to user's addresses, and profiles, and parameters, and, well, pretty much everything. Which was great, except that the more we logged, the bigger a nightmare it was to keep track of what changed when. The reason, of course, was that whenever any data was entered into the system, we took great to break it apart into chunks so that the appropriate data went to the appropriate place. User's email here, account settings there, phone numbers way over there, etc. etc. But trying to recreate these changes, and god-forbid, figure out an audit-trail of what happened when, aaaargh. And thus it came to pass that It Was Not Good.
Fast forward to a few years back, when we rewrote our entire platform from scratch, but this time it was all erlang and web-scale and distributed and cloud and It Was Good.
Its ridiculously effective, its useful, and it makes our life vastly simpler.
In short, count me in on the "Document as the single source of truth" bandwagon…
Comments