Subject: Regression testing
Date: Sat, 23 Jan 1999 06:16:38 -0500
To: year2000-discuss@year2000.com
I wonder if someone could explain what the regression testing is? What steps do we take to
do it?
Tevy,
You will get significantly different answers from different people on that one.
I define regression testing as executing your application before and after remediation and
comparing the output to demonstrate the output only varies in ways you expect it to vary
(for example, if you expanded, you expect the output files of the remediated program to
have century digits where the older version may not have had them, if you have time
stamps, you might expect those to vary). The goal of regression testing is to demonstrate
you have not "broken" something in your application during the process of
remediation. A regression test proves nothing about the success or failure of Y2K
remediation efforts. In fact, any non-compliant application can always pass a regression
test by not touching the application and it will still fail miserably when it starts to
process Y2K data.
In a Y2K context, I differentiate that from time dimensional testing, where the remediated
program is executed with date fields that vary by a "constant" amount of time
(dates are "aged") and outputs are compared to demonstrate the output only
varies in date fields the way you expect it to vary. The goal of time dimensional testing
is to demonstrate that the application works the same "yesterday",
"today" and "tomorrow" implying that Y2K will have no effect on it
(when "tomorrow" extends into the next century).
If I broke it into steps, I suppose you would:
- Create your test data or use existing test bed/test cases.
- Run your application against the test data, saving the output
files for later comparison
- Remediate the application
- Run the remediated application against the same input data used in
(2), saving the output
- Compare the saved outputs from (2) and (4)
- if any unexpected differences show up, go back to step (3)
I put (1) and (2) first to emphasize that these two can be done
before, in parallel with or even after remediation.
Another opinion is Don Estes, who I believe defines regression testing to include aging
dates within this century and only considers time dimensional testing to include dates
that extend past 1999.
If you have expanded your data, we have a file comparison tool that will help you in the
regression comparison process. Without identifying the location or formats of any specific
dates in the file, it will accurately compare the files, except for two well understood
cases expressed on my WWW site, and provided your date formats are among the thousands we
cover (or you've had us add your specific, unique format). This identification
characteristic also makes this tool ideal for comparing otherwise hard/impossible to
compare files such as aged printer files.
When you get into testing with aged data, the same comparison tool can also tell you by
how much the two files vary (in days or years).
