Immutability for your K-V stores
How do you guarantee that the data in your database is immutable? In particular, I’m talking about data that you don’twant to change — things like call-records, historical data, transaction logs, etc.
Of course, you could use something like Datomic, or you could architect your application/data-structures/tables/… from the ground up to be immutable, but that could be … problematic. Especially problematic, in fact, given that I’m referring to your existing databases that have all this stuff in it — good luck trying to convince the business side of the house that you need to re-architect everything!
Of course, you could use something like Datomic, or you could architect your application/data-structures/tables/… from the ground up to be immutable, but that could be … problematic. Especially problematic, in fact, given that I’m referring to your existing databases that have all this stuff in it — good luck trying to convince the business side of the house that you need to re-architect everything!
Theoretically, this data should already be immutable, but, the reality is that there are so many attack vectors.
- 1. One of your developers could go in and tweak the data — you’d never know
- 2. Oh, only the DBA has access? What if they tweak it?
- 3. Worse, what if somebody hacks into your system?
- 4. Even worse, what if somebody at the data-centers hacks the hypervisor?
And so on. You get the point, right?
Enter VeritasDB — a key value store that sits between your clients and servers, and ensures that the data that comes back from the servers has not been changed. It acts as a network proxy that sits between the unmodified client(s) and the unmodified servers, and outputs one of the following results for each operation
— success (“here’s the data you asked for!”)
— integrity violation (“something bad happened to the data!”)
— application error (“use the API correctly you idiot”)
— failure due to benign reasons (“OOM!”)
— success (“here’s the data you asked for!”)
— integrity violation (“something bad happened to the data!”)
— application error (“use the API correctly you idiot”)
— failure due to benign reasons (“OOM!”)
Behind the scenes, it is implemented as an authenticated Merkle B+ tree, which uses caching, concurrency, and compression to improve performance in most practical “real-world” workloads — performance testing shows a slowdown of around 5%.
There’s also a whole bunch of innovative stuff around Intel’s Software Guard Extensions (SGX) which makes the whole thing possible, as well as safe.
And, to top it all off, they’ve formally verified VeritasDB.
And, to top it all off, they’ve formally verified VeritasDB.
Good stuff — check it out!
Comments