Requirements Testing, or Executable Requirements

One of the issues always raised with Software Development, especially agile methods, is testing.Agile insists on developer testing. Lean recognizes automated unit tests as the rough equivalent of jigs and fixtures in manufacturing. They provide the “hard points” that you build against so that the part comes out as designed, and works with the rest of the system.But there are other classes of tests. Most people think of acceptance tests, which are usually conducted by QA, and are usually run manually. They provide the assurance to the customer that the system does what the specification says it should do. That doesn’t necessarily mean it does what the customer needs it to do, but that’s another story.Or maybe not. One approach to decrease the distance between the specification ant the true requirements (different from the stated requirements) would be do develop very detailed specifications, that are at least partially executable. Then when the executable portion of the specification passes, that part of the system meets the specification. This means that requirements have demonstrably flowed down through the code, eliminating another opportunity for error, the requirements traceability matrix. If the tests in the requirements pass, the code meets the intent of the requirements.Also, by bringing the testers into the discussions with the customer or customer representative at this early stage of the game, they will start asking questions that may not get asked otherwise, and flesh out a whole class of requirements that might otherwise get missed until late in the project, risking late delivery.So what’s the best means of accomplishing this? The best approach is to use an appropriate automated testing framework, Record and Playback tools are probably worse than useless, and hire a good senior developer to develop the customizations necessary to make the framework work with your class applications. This should require some thin glue code that work with a whole class of test cases. The testers then use data to drive the tests through the test engine. The testers will still spend time running manual tests. Mostly to run those cases that are just too expensive to automate (think 80-20 rule, or maybe 90-10, you get the idea).More importantly, the testers will have time to do exploratory testing. The types of things that will uncover those tricky corner cases that no one thought about because they didn’t think anyone would ever use the system that way. Trust me, they will.So what tools are there available? One that I know of is FitNesse. This is an open source tool that uses a wiki to wrap the FitFramework, which executes tests created in tables. I hope to talk more about this in the future, but enough for now.Technorati Tags:, ,

This entry was posted in Software Development. Bookmark the permalink.

Comments are closed.