Monday, October 25, 2010

Adv Debugging - My Program Worked Yesterday, but Not Today (WYNOT) by Andreas Zellar

Brief Notes of my understanding of on Delta Debugging as discussed in “Worked Yesterday, NOt Today" by Andreas Zeller 
Note:  To understand the basic premise and constraints/approaches of Delta-Debugging referred to the pdf link to:
  1. Analyse the tables/graphs carefully. (80% of the understanding comes from the example tables/graphs)
  2. Connect the WYNOT talk with Andreas Zellar's podcast on Software Engineering Radio.
Problem: Yesterday's code works but Today's code doesn't. How do we find the change(s) which induces the test-case failure.
Context: Looking through the CVS history we find that N changes have been added since Yesterday.

Alternative: Normally we would need run the debugger to reproduce the failure and then try to collect information which induces it.  But this requires a programmer to interactively query the program state using a debugger.
Can the debugging be automated without needing the programmer?

Strategy: Delta Debugging using a test-case to reproduce the failure.
Data Method:
Using the Scientific Method we can first try to reduce the Input Data in a binary search mechanism to isolate the minimum data which causes the failure.
Code Method:
  1. Simple Method: When failure is caused by a single change. We can try to isolate the change which causes the failure by doing a binary search through the change history. If there are N changes between Y(esterday) and T(oday) we test 'k/2'th change where 1 <= k <= N. We go on partitioning until we find a failure-free change. The latest failing change is the culprit. Complexity of this binary search is O (log N)
  2. Complex Method: When failure is caused by a combination of changes. Unfortunately it's Not always possible to use the Simple Method in the case when combination of changes cause the failure. In such a situation we need to identify at least 2 (or more) changes which together cause the failure. 
  • Divide the change history into N changes
  • Group N changes into say 4 units i.e. N = N/4 * 4
  • Generate combinations with different units.
  • Test combinations for the failure. 
  • Prune any units which don't participate in the failure at all.
  • Recursive above steps by sub-dividing the units above (quarters into octets and so on) until we finally we end up with minimal changes that reproduce the failure.
Proposed Plans to Reduce to Reduce Computation and Avoid Inconsistent Configurations :
a) Group related changes - changes on common date/file/variable are more likely to be related. Group them into a single unit to avoid dependency across units.
b) Prune configurations when they're obvious dead-ends e.g. Where change dependency is chained all the way (Today) 10->9->8->7->6->5->4->3->2->1 (Yesterday)
c) Exclude changed code which is never executed.

See Also:
Pdf: "WYNOT - Worked Yesterday, NOt Today"
Andreas Zeller
Podcast on Software Engineering Radio
Delta Debugging

No comments:

Post a Comment

Note: Only a member of this blog may post a comment.