Monday, February 15, 2010

Volumetric Testing Methodologies

Is it possible to test every single combination/permutation out there in a system. The world, no but an isolated system, yes, but is it worth it? I say the answer depends if it's a perfect system. 

How does Amazon ensure the wrong package doesn't go to the right addr? Let's break into 2 components. There's a possible incorrect bind of address to a product on the website end. Then there's a failure in putting the right product into the box. If the shopping site fails 10% to bind the address/prod on the website; amzn would go out of business. So, how's it prevent this: develop solid code and qa it after dev for every possible permutation. But they don't have to do this after the system is perfected. The same probably goes for the packaging side (prob. using RF tech). I can't imagine a crack team ripping up packages randomly to ensure if the right product got into the right box. They could do that for a little while but the monotomy would eventually make this process ineffective. That's why over 55% or more of the time, security experts can sneek past TSA officers a weapon. I will make a guess that Custom agents at ports differentiate by making a plot of what's coming in as that's not totally random (like we'll expecting a large shipment from Yemen at this time). The same probably can't be said of Amazon. So, what's common theme here: the need for a narrative. W.o this, complacency happens, as also security experts are also able to infritrate military bases and practically steal submarines.

How about the argument that ppl should at least test what we send out. Let's at amazon again: True Amazon doesn't test every single product in their inventory. Now, what about every single product they send out. They don't because they know their website binding system is "perfect" and the packaging side is probably "near-perfect". I would introduce tests that promote fundamental queries on the groundings of the system (like slight nuances that would cause the perfect system to stop working; like process-additions, etc). I would NOT test what the perfect system tests; I would test the perfect system itself. This would probably be done by pattern recognition. The other test is for the imperfect parts; like the creative part and that would be served best by peer review as it's mostly a publishing system.

1 comment:

  1. I would suggested a more reactive approach (but super reactive) -> you know how managers always use the cliché of proactive. So yes, we should be proactive to be super reactive. Since putting excessive amts of energy on >3% failure is ridiculous, how about putting energy on a system that changes errors instantly on that 3% (I believe that setting up a system will be less energy than the 97% spent each and every time).