<chapter id="cvs-regression"> | |
<title>How to do regression testing using Cvs</title> | |
<para> | |
written by (???) | |
</para> | |
<para> | |
(Extracted from <filename>wine/documentation/bugreports</filename>) | |
</para> | |
<para> | |
A problem that can happen sometimes is 'it used to work | |
before, now it doesn't anymore...'. Here is a step by step | |
procedure to try to pinpoint when the problem occured. This is | |
<emphasis>NOT</emphasis> for casual users. | |
</para> | |
<orderedlist> | |
<listitem> | |
<para> | |
Get the 'full cvs' archive from winehq. This archive is | |
the cvs tree but with the tags controlling the versioning | |
system. It's a big file (> 15 meg) with a name like | |
full-cvs-<last update date> (it's more than 100mb | |
when uncompressed, you can't very well do this with | |
small, old computers or slow Internet connections). | |
</para> | |
</listitem> | |
<listitem> | |
<para> | |
untar it into a repository directory: | |
<screen> | |
cd /home/gerard | |
tar -zxffull-cvs-2000-05-20.tar.gz | |
mv wine repository | |
</screen> | |
</para> | |
</listitem> | |
<listitem> | |
<para> | |
extract a new destination directory. This directory must | |
not be in a subdirectory of the repository else | |
<command>cvs</command> will think it's part of the | |
repository and deny you an extraction in the repository: | |
<screen> | |
cd /home/gerard | |
mv wine wine_current (-> this protects your current wine sandbox, if any) | |
export CVSROOT=/home/gerard/repository | |
cd /home/gerard | |
cvs -d $CVSROOT checkout wine | |
</screen> | |
</para> | |
<para> | |
Note that it's not possible to do a checkout at a given | |
date; you always do the checkout for the last date where | |
the full-cvs-xxx snapshot was generated. | |
</para> | |
</listitem> | |
<listitem> | |
<para> | |
you will have now in the <filename>~/wine</filename> | |
directory an image of the cvs tree, on the client side. | |
Now update this image to the date you want: | |
<screen> | |
cd /home/gerard/wine | |
cvs -d $CVSROOT update -D "1999-06-01" | |
</screen> | |
</para> | |
<para> | |
The date format is <literal>YYYY-MM-DD</literal>. | |
</para> | |
<para> | |
Many messages will inform you that more recent files have | |
been deleted to set back the client cvs tree to the date | |
you asked, for example: | |
<screen> | |
cvs update: tsx11/ts_xf86dga2.c is no longer in the repository | |
</screen> | |
</para> | |
<para> | |
<command>cvs update</command> is not limited to upgrade to | |
a <emphasis>newer</emphasis> version as I have believed for far too long :-( | |
</para> | |
</listitem> | |
<listitem> | |
<para> | |
Now proceed as for a normal update: | |
</para> | |
<screen> | |
./configure | |
make depend && make | |
</screen> | |
<para> | |
When you have found the exact date when a bug was added to | |
the cvs tree, use something like : | |
<screen> | |
cvs -d $CVSROOT diff -D "1999-07-10" -D "1999-07-12" | |
</screen> | |
to get all the differences between the last cvs tree | |
version known to work and code that first displayed the | |
misbehavior. | |
</para> | |
<note> | |
<para> | |
I did not include flags for <command>diff</command> | |
since they are in my <filename>.cvsrc</filename> file: | |
</para> | |
<screen> | |
cvs -z 3 | |
update -dPA | |
diff -u | |
</screen> | |
</note> | |
<para> | |
From this diff file, particularly the file names, and the | |
<filename>ChangeLog</filename>, it's usually possible to | |
find the different individual patches that were done at | |
this time. | |
</para> | |
<para> | |
If any non-programmer reads this, the fastest method to get | |
at the point where the problem occured is to use a binary | |
search, that is, if the problem occured in 1999, start at | |
mid-year, then is the problem is already here, back to 1st | |
April, if not, to 1st October, and so on. | |
</para> | |
</listitem> | |
<listitem> | |
<para> | |
The next step is to start from the last working version | |
and to dig the individual contributions from | |
<ulink url="http://www.integrita.com/cgi-local/lwgate.pl/WINE-PATCHES/"> | |
http://www.integrita.com/cgi-local/lwgate.pl/WINE-PATCHES/</ulink> | |
(where the Wine patches mailing list is archived) | |
</para> | |
<para> | |
If the patch was done by the Wine maintainer or if it was | |
sent directly to his mail address without going first through | |
<ulink url="mailto:wine-patches@winehq.com">wine-patches</ulink>, | |
you are out of luck as you will never find the patch in | |
the archive. If it is, it's often possible to apply the | |
patches one by one to last working cvs snapshot, compile and test. | |
If you have saved the next candidate as | |
<filename>/home/gerard/buggedpatch1.txt</filename>: | |
</para> | |
<screen> | |
cd /home/gerard/wine | |
patch -p 0 < /home/gerard/buggedpatch1.txt | |
</screen> | |
<para> | |
Beware that the committed patch is not always identical to | |
the patch that the author sent to wine-patches, as | |
sometimes the Wine maintainer changes things a bit. | |
</para> | |
<para> | |
If you find one patch that is getting the cvs source tree to | |
reproduce the problem, you have almost won; post the problem on | |
<systemitem>comp.emulators.windows.wine</systemitem> and there | |
is a chance that the author will jump in to suggest a fix; or | |
there is always the possibility to look hard at the patch until | |
it is coerced to reveal where is the bug :-) | |
</para> | |
</listitem> | |
</orderedlist> | |
</chapter> | |
<!-- Keep this comment at the end of the file | |
Local variables: | |
mode: sgml | |
sgml-parent-document:("wine-doc.sgml" "set" "book" "part" "chapter" "") | |
End: | |
--> |