And you are here why?

If your service is running on EC2, you probably already know about Spot Instances .  But for those of you who don't, this is a service where Amazon allows you to bid on unused capacity in their data-centers.  The pricing varies based on supply and demand, but is typically somewhere between 25% and 50% the cost of a "reserved" instance (e.g., right now, a "Small" on-demand instance is $0.085/hr, while the spot cost is $0.027/hr). Amazon has done a lot of work in making these very easy to use, everything from the bidding, to the setup, to VPC integration. As you might imagine, this has not gone un-noticed, and there are any number of companies that have gone the spot right towards their infrastructure - using them to soak up the load associated with excess capacity. You even have guides out there on how to setup and use spot instances, a sure sign that its ubiquitous. So, whats the catch? As it turns out, the correct question is, what are the two catc...

WaaS is basically what Amazon has just released with their Amazon Simple Workflow (SWF) .  From the blog post This new service gives you the ability to build and run distributed, fault-tolerant applications that span multiple systems (cloud-based, on-premise, or both). Amazon Simple Workflow coordinates the flow of synchronous or asynchronous tasks (logical application steps) so that you can focus on your business and your application instead of having to worry about the infrastructure Picture a business process which consists of a bunch of individual tasks that need to be done in some specific sequence based on various inputs.  e.g.,   When an order comes in , check to see if we have inventory.  If we do, fill the order. If we don't, put the order in the "back-ordered" pile. Each of these tasks is called an Action .  A given Action is performed by a Worker . The whole business process is called a Workflow .  And the thingy doing all the d...

/via http://www.commitstrip.com/en/2018/09/24/trolling-the-ai/ This  particular episode of CommitStrip  pretty much represents the popular take on AI. Mind you, “popular take” is, as usual,  not  the same as “realistic”. After all,  when you’re designing robust systems, one of the very first things you’ll do is correct for signs that are missing, misplaced, or mis-identified , and if/when self-driving cars actually make their way into the real world, one of the things that they  will  be is robust against errors at multiple levels. Heck, that’s pretty much the way  humans  work — our vision is pretty seriously fallible, but we correlate a lot of  other  information to make sure that we don’t f**k up! Think about how you process visual feedback when driving. You are, unconsciously, running rules like these in your head all the time • A sign saying “60 mph” in a residential neighborhood is clearly wrong •  If you are wond...

/via Amazon /via Amazon

As you’ve probably heard,  IBM is about to release a pretty monster dataset  — over a million images — along with tool, all with the aim of helping get rid of bias in facial analysis. The cool part for me is actually the announcement of a  second dataset — around 36,000 images — that are “ equally distributed across skin tones, genders, and ages ”. So, why does this matter? Before answering this, let’s first take a brief diversion. Let’s say you are doing something involving Machine Learning and facial recognition. You’d need a dataset to train your models against — think about how you would select your dataset. You’d probably take into consideration the specifics of the task (“ I need to know if the face is smiling or not ”), the details of the algorithm that you’re working on (“ Can I still tell it’s a smile if the background changes? ”) and such-like. You’d then go to one of the  handy-dandy collection of facial-recognition databases , and pick the most appropri...

AT&T is getting into the OpenStack game - becoming a contributor, and also going after the general cloud-infrastructure space. (Yes, I know, they've been in the enterprise space w/ Synaptic ).  Thing is, in the bigger IaaS/PaaS space, its not just about whether you have the better Infrastructure or Platform, its also about mind-share, and what I'll call trust-share .  To be precise, (most) people trust Amazon/Heroku/Rackspace to not screw them over.  On the other hand, most people think of AT&T (and/or Verizon) as "Phone Companies" - with the concomitant baggage. Put it another way - if American / Delta / United announced that they were getting into the cloud-infrastructure space (humor me), I'd argue that most people would run screaming in the opposite direction, since the implicit assumption nowadays is that the airlines are in the fleece the passengers business. So, back to my original question - Do You Trust AT&T ? I'm not quite sure w...

InfoChimps  is offering cloud-based BigData platform to provide out-of-the-box (or is that out-of-the-cloud ?) Hadoop service.  Plenty of details at the link , as well as GigaOm commentary here . The new   Infochimps Platform   is essentially a publicly available version of what the company has built internally to process and analyze the data it stores within its marketplace. As Infochimps CEO Joe Kelly puts it, the company is “giving folks … the iPod to our iTunes.” Me, I think this is the start of something really good. The thing is, there is a ton of activity in the Hadoop space, with everybody and their brother tossing their hat into the ring providing some set of ancillary services based on Hadoop. The problem with all of these are that they presume you know what you are doing ! Needless to say, you don't. Really. You...Just...Don't. Let me be a bit clearer - the vast majority of people out there who could use Hadoop-based services fall i...

Available on Amazon , a collection of (nearly) a hundred sacred teachings that are capable of "bringing about an enlightenment experience so pure, so elevating, and so intense, you could plotz". Examples include If there is no self, whose arthritis is this? Be here now. Be someplace else later.   Is that so complicated? Wherever you go, there you are. Your luggage is another story. Accept misfortune as a blessing. Do not wish for perfect health, or a life without problems.   What would you talk about? The journey of a thousand miles begins with a single Oy. There is no escaping karma. In a previous life, you never called, you never wrote, you never visited. And whose fault was that? Zen is not easy. It takes effort to attain nothingness. And then what do you have? Bupkis. Breathe in. Breathe out. Breathe in. Breathe out.   Forget this and attaining Enlightenment will be the least of your problems. The Torah says, Love your neighbor as yourself. The Buddha sa...

Nice article by the EngineYard team regarding MongoDB best practices . Its actually a two-part article, first getting into general NoSQL, and then specifically into MongoDB. The NoSQL part is (as you might expect) somewhat general (I've got my own general contribution here ), but I love the understatement in the point about EBS you should know that the performance of Amazon’s Elastic Block Storage (EBS) can be inconsistent . Indeed!  I'd actually take it a step further and say Do not use EBS in any environment where reliability and/or performance characteristics of your disk-access are important .  Or, to put it differently, asynchronous backups - OK, disk-based databases - Not So Much.   Anyhow, the rest of the article is really quite relevant - somewhat obvious, but still extremely relevant.  It may read like a 'best-hits' list of the comments from the Don't Use MongoDB flame-war a wee bit back, but if you weren't part of that, well, read this....

There is a pretty huge meme going around right now about how "Google Drive is stealing your data".  Typical examples on twitter include @pdparticle - WOW! Google pretty much owns your data once uploaded to Google Drive. t.co/YUIqIVtM #GoogleDrive Google Drive owns everything you upload: terms of service analysis t.co/MDuZZ52R The truth, as in most of these cases, is a lot more complicated.  In a nutshell, if you want to blame anybody/anything, blame the copyright laws . First of all, just to get this out of the way, Google explicitly says   Some of our Services allow you to submit content. You retain ownership of any intellectual property rights that you hold in that content. In short, what belongs to you stays yours Moving on, consider what rights Google needs to just do what it does on a day to day basis In fact, do this backwards - if you store your data on a hosted service, anywhere , then you have to assign a whole bushel of rights to that service fo...

Yap was one of the big names in Voice Recognition - powering a lot of the voicemail transcription market -  till they suddenly went silent on 10/20/2011 (Silent Thursday?).  The rumors were rife - everything ranging from software breakdown, to Filipino strike, to (the most baffling one) Sneaky Verizon Ploy.  It turns out the truth was a lot less strange - Amazon bought 'em out to (most likely) embed in the Fire.  Its certainly an anti-Siri ploy, but I guess we'll find out about the Fire rumors Any Day Now...

For those who care... Kindle Fire just started shipping yesterday and a super easy one click root / jailbreak solution is already available. We went through the steps, were able to reproduce it and duplicate the end result and we also streamlined it a bit especially for the ones that do not excel in computer skills so here it is for your Kindle Fire rooting / jailbreaking pleasure. What is the root process bringing to the table? The possibility to install Google Android Marketplace (remember, by default Kindle Fire only comes with Amazon Marketplace), sideloading books on Kindle Fire is another enticing possibility as well. (more...)

This isn’t just IoT, it is pretty much any device in my house, and quite probably most in the office too. Think of it this way 1. The default software they are shipped with  will  have some security holes in it. 2. Even when they patch (“some of”? “most of”? definitely not “all of”!) these devices, there will still be unpatched devices in the Amazon warehouse. Which one did you get? How would you even know? 3. Assuming you  do  know that you got an unpatched device, did you actually get the correct patch/revision to update it? And what about future patches? 4. Auto-update ? WTF even is that? And even if you do know what it is, do you trust it to work? 5. So you  do  trust it — are they updating on a regular schedule? Seriously, most phone manufacturers can’t even be bothered to update any phone more than a year old, and let’s not even get started on your cable modem/router… And all this is assuming that  you  have good security practi...

From Noahpinion

Toss the following into a cuisinart / blender, and grind it coarsely  (ok, what I mean by that is - don't let 'er rip for hours, and end up with a fine paste - you want some texture in the final product!) 1 bunch cilantro  (rip off the bottom 3 inches of the stems. You don't have to, they have plenty of flavor/fibre) 1/2" grated ginger 1 serrano pepper 3 tbsp roasted split chickpeas (•) 1 tbsp peanuts (optional. I like the flavor) 1/4 cup fresh coconut, grated (••) Juice of 1 lemon 1 tbsp tamarind paste salt to taste  (probably 1/2 tsp, but it depends on how salty you like stuff...) This is your base chutney In a small sauce-pan (or ladle, or some such), make voggarne  (think "tempered spices that you add to the chutney for crunch and flavor) 1 tsp mustard seeds 1 tbsp olive oil    and heat on high till the seeds  just  start crackling. Then, add 1 tsp roasted split chickpeas  and continue on the heat till it...

Oh Nifty - for all you .NET developers out there, here is a Step-by-Step Guide to Amazon DynamoDB w/ .NET .  11 short steps to creating a table and setting up CRUD on it. . <pause while - hopefully - people go look> . <pause> . <you're back, right?> Is it just me?  Or does this seem overly complex?  Then again, I don't do Visual Studio, and I *really* don't do .NET (Does one imply the other? I don't really want to know), but this feels, well, pretty painful to me. Have I mentioned that I hate Java too? Do I sound like Andy Rooney?

So you've got a cloud-based service.  Good for you!  I bet its brilliant,  is brilliantly fault-tolerant 'cos of Erlang/OTP, and is going to make you Pots o' Money!  That is, of course, assuming that you have prepared yourself for the inevitable "Oh S**t!" moment, when It Goes Down.  Herewith a couple of pointers to help you prepare for everything not working exactly the way it is supposed to (Servers die?  really?)… 1) Make sure that you understand your entire system .  Done?  Good, now make sure that all of your developers understand the entire system.  Specialization might be good (and don't get me wrong, weed-whackers are *very* useful little tools, especially when you get into the weeds :-)  ), but if they don't grok the big picture, well, you're hosed.  Capturing vast quantities of scoring data could be really useful for GUI updates, and might be easy to implement from an API perspective, but if your persistence layer...

The Mel Brooks collection from Amazon - $29 (Insert some commentary about people moving to different media consumption patterns, etc., etc.)