1 A simple testing framework |
|
2 |
|
3 To run the tests, do: |
1 To run the tests, do: |
4 |
2 |
5 cd tests/ |
3 cd tests/ |
6 python run-tests.py |
4 python run-tests.py |
7 |
5 |
8 This finds all scripts in the test directory named test-* and executes |
6 See http://www.selenic.com/mercurial/wiki/index.cgi/WritingTests for |
9 them. The scripts can be either shell scripts or Python. Each test is |
7 more information on writing tests. |
10 run in a temporary directory that is removed when the test is complete. |
|
11 |
|
12 A test-<x> succeeds if the script returns success and its output |
|
13 matches test-<x>.out. If the new output doesn't match, it is stored in |
|
14 test-<x>.err. |
|
15 |
|
16 There are some tricky points here that you should be aware of when |
|
17 writing tests: |
|
18 |
|
19 - hg commit and hg merge want user interaction |
|
20 |
|
21 for commit use -m "text" |
|
22 for hg merge, set HGMERGE to something noninteractive (like true or merge) |
|
23 |
|
24 - changeset hashes will change based on user and date which make |
|
25 things like hg history output change |
|
26 |
|
27 use commit -m "test" -u test -d "1000000 0" |
|
28 |
|
29 - diff and export may show the current time |
|
30 |
|
31 use -D/--nodates to strip the dates |
|
32 |
|
33 - You can append your own hgrc settings to the file that the environment |
|
34 variable HGRCPATH points to. This file is cleared before running a test. |
|
35 |
|
36 You also need to be careful that the tests are portable from one platform |
|
37 to another. You're probably working on Linux, where the GNU toolchain has |
|
38 more (or different) functionality than on MacOS, *BSD, Solaris, AIX, etc. |
|
39 While testing on all platforms is the only sure-fire way to make sure that |
|
40 you've written portable code, here's a list of problems that have been |
|
41 found and fixed in the tests. Another, more comprehensive list may be |
|
42 found in the GNU Autoconf manual, online here: |
|
43 |
|
44 http://www.gnu.org/software/autoconf/manual/html_node/Portable-Shell.html |
|
45 |
|
46 sh: |
|
47 |
|
48 The Bourne shell is a very basic shell. /bin/sh on Linux is typically |
|
49 bash, which even in Bourne-shell mode has many features that Bourne shells |
|
50 on other Unix systems don't have (and even on Linux /bin/sh isn't |
|
51 guaranteed to be bash). You'll need to be careful about constructs that |
|
52 seem ubiquitous, but are actually not available in the least common |
|
53 denominator. While using another shell (ksh, bash explicitly, posix shell, |
|
54 etc.) explicitly may seem like another option, these may not exist in a |
|
55 portable location, and so are generally probably not a good idea. You may |
|
56 find that rewriting the test in python will be easier. |
|
57 |
|
58 - don't use pushd/popd; save the output of "pwd" and use "cd" in place of |
|
59 the pushd, and cd back to the saved pwd instead of popd. |
|
60 |
|
61 - don't use math expressions like let, (( ... )), or $(( ... )); use "expr" |
|
62 instead. |
|
63 |
|
64 grep: |
|
65 |
|
66 - don't use the -q option; redirect stdout to /dev/null instead. |
|
67 |
|
68 - don't use extended regular expressions with grep; use egrep instead, and |
|
69 don't escape any regex operators. |
|
70 |
|
71 sed: |
|
72 |
|
73 - make sure that the beginning-of-line matcher ("^") is at the very |
|
74 beginning of the expression -- it may not be supported inside parens. |
|
75 |
|
76 echo: |
|
77 |
|
78 - echo may interpret "\n" and print a newline; use printf instead if you |
|
79 want a literal "\n" (backslash + n). |
|
80 |
|
81 false: |
|
82 |
|
83 - false is guaranteed only to return a non-zero value; you cannot depend on |
|
84 it being 1. On Solaris in particular, /bin/false returns 255. Rewrite |
|
85 your test to not depend on a particular return value, or create a |
|
86 temporary "false" executable, and call that instead. |
|
87 |
|
88 diff: |
|
89 |
|
90 - don't use the -N option. There's no particularly good workaround short |
|
91 of writing a reasonably complicated replacement script, but substituting |
|
92 gdiff for diff if you can't rewrite the test not to need -N will probably |
|
93 do. |
|