You really need to *understand* what you're working with, especially at scale...
Its just the same old story, i.e., Complexity never goes away, it just moves up the food chain.
From the recent MongoSV conference, you have the following discoveries
Wordnik had MySQL issues, so switched to MongoDB. They then went through a bunch of roving bottlenecks
- disk IO in the cloud sucks, move everything to physical server
- cost of physical servers too high, move back to the cloud
- disk IO issues again,
- finally figure out that you need to account for the specific strengths and weaknesses of MongoDB as per the environment that you are running it in
Foursquare realized early on that they needed to keep everything in memory, but being on AWS, they used EBS to back up their data. They then ran into the chaos associated with the performance of EBS (moody, technically opaque, and at times downright confounding) resulting in custom application level solutions to resolving the errors - some of these as basic as "kill and restart the process".
Disney Interactive Media Group has Major Monitoring Issues. Yeah, MongoDB has the Mongo Monitoring System, but that doesn't absolve you from knowing what its strengths and weaknesses are, and from monitoring it.
Notice a common pattern across all of these? Just because something is easy to use, doesn't mean you understand it. You probably don't. In fact, the easier it is to use, the more likely it is that there are vast swathes of it that you have no clue about. Given that the total complexity in an environment is fixed (probably something to do with the Second Law of Thermodynamics), making some part of it easier just moves the complexity around - probably higher up the food chain where it is guaranteed to bite you later.
Mind you, please don't take this to be a criticism of MongoDB (though lord knows there is enough to criticize there). All I'm getting at is that you really need to understand what you're working with, especially at scale...
From the recent MongoSV conference, you have the following discoveries
Wordnik had MySQL issues, so switched to MongoDB. They then went through a bunch of roving bottlenecks
- disk IO in the cloud sucks, move everything to physical server
- cost of physical servers too high, move back to the cloud
- disk IO issues again,
- finally figure out that you need to account for the specific strengths and weaknesses of MongoDB as per the environment that you are running it in
Foursquare realized early on that they needed to keep everything in memory, but being on AWS, they used EBS to back up their data. They then ran into the chaos associated with the performance of EBS (moody, technically opaque, and at times downright confounding) resulting in custom application level solutions to resolving the errors - some of these as basic as "kill and restart the process".
Disney Interactive Media Group has Major Monitoring Issues. Yeah, MongoDB has the Mongo Monitoring System, but that doesn't absolve you from knowing what its strengths and weaknesses are, and from monitoring it.
Notice a common pattern across all of these? Just because something is easy to use, doesn't mean you understand it. You probably don't. In fact, the easier it is to use, the more likely it is that there are vast swathes of it that you have no clue about. Given that the total complexity in an environment is fixed (probably something to do with the Second Law of Thermodynamics), making some part of it easier just moves the complexity around - probably higher up the food chain where it is guaranteed to bite you later.
Mind you, please don't take this to be a criticism of MongoDB (though lord knows there is enough to criticize there). All I'm getting at is that you really need to understand what you're working with, especially at scale...

Comments