git @ Cat's Eye Technologies Falderal / 665ef95
Merge pull request #9 from catseye/ensure-shell-text-quoted Ensure text is quoted when variables expanded Chris Pressey authored 3 years ago GitHub committed 3 years ago
15 changed file(s) with 186 addition(s) and 112 deletion(s). Raw diff Collapse all Expand all
+5
-0
HISTORY.markdown less more
99 * In "freestyle" format, lines beginning with `<= `, `<== `, or
1010 `<=== ` can supply a section of test input text (Thanks to
1111 James Holderness for feature suggestion and patch.)
12 * Falderal is now responsible for quoting all shell metacharacters
13 in the substituion text of `%(...)` variables when command line
14 templates are expanded, so that no quotes need be placed around
15 the variable in the template. (Again, thanks to James Holderness
16 for spotting the bug in the code that led to this improvement.)
1217 * `py-falderal` can now run under both Python 2 and Python 3
1318 (tested with 2.7, 3.4, and 3.5.)
1419 * The doctests that were previously distributed throughout the
+14
-6
doc/Falderal_Literate_Test_Format.markdown less more
161161 text of the test and produces its output, to be compared with the expected
162162 output.
163163
164 When a variable is expanded, it is Falderal's responsibility to escape any
165 shell metacharacters that occur as part of the substitution text so that
166 it appears to the shell as a single token, and cannot otherwise affect the
167 meaning of the instantiated shell command. It is not necessary for a
168 variable to be enclosed in any kind of quotes in the command line template.
169 It is also not possible for the substitution text to e.g. expand to two
170 arguments, or to form a pipe or redirection specifier, etc.
171
172 The exact method by which the Falderal implementation escapes or quotes
173 the substitution text so that it appears unambiguously as a single argument
174 is left up to the implementation.
175
164176 ##### `%(test-body-file)` #####
165177
166178 The variable `%(test-body-file)` will be replaced by the name of a file which
170182 ##### `%(test-body-text)` #####
171183
172184 The variable `%(test-body-text)` will be replaced by the actual text of the
173 test body. It is assumed that `%(test-body-text)` will appear inside single
174 quotes in the command string, so any single quotes in the text of the test will
175 be escaped by the Falderal implementation by preceding them with backslashes.
185 test body.
176186
177187 ##### `%(test-input-file)` #####
178188
183193 ##### `%(test-input-text)` #####
184194
185195 The variable `%(test-input-text)` will be replaced by the actual text of the
186 test input. It is assumed that `%(test-input-text)` will appear inside single
187 quotes in the command string, so any single quotes in the text of the test will
188 be escaped by the Falderal implementation by preceding them with backslashes.
196 test input.
189197
190198 If neither of the variables `%(test-body-file)` nor `%(test-body-text)` appear
191199 in the command string, the test body text will be provided on the standard
+16
-10
src/falderal/objects.py less more
88 unicode = unicode
99 except NameError:
1010 unicode = str
11
12 try:
13 from shlex import quote as shlex_quote
14 except ImportError:
15 from pipes import quote as shlex_quote
1116
1217 # Note: the __str__ method of all the classes defined herein should
1318 # produce a short, human-readable summary of the contents of the object,
563568 def __eq__(self, other):
564569 return self.__class__ == other.__class__ and self.command == other.command
565570
571 def subst(self, command, var_name, value):
572 """Replace all occurrences of `var_name` in `command` with
573 `value`, but make sure `value` is properly shell-escaped first."""
574 return command.replace(var_name, shlex_quote(value))
575
566576 def run(self, body=None, input=None):
567 # expand variables in the command
577 # first, expand all known variables in the command, using subst().
568578 test_filename = None
569579 output_filename = None
570580 command = self.command
582592 file.close()
583593 os.close(fd)
584594 # replace all occurrences in command
585 command = command.replace('%(test-body-file)', test_filename)
595 command = self.subst(command, '%(test-body-file)', test_filename)
586596 command_contained_test_body_file = True
587597
588598 if '%(test-body-text)' in self.command:
589 # escape all single quotes in body
590 body = re.sub(r"'", r"\'", body)
591599 # replace all occurrences in command
592 command = command.replace('%(test-body-text)', body)
600 command = self.subst(command, '%(test-body-text)', body)
593601 command_contained_test_body_text = True
594602
595603 if '%(test-input-file)' in self.command:
601609 file.close()
602610 os.close(fd)
603611 # replace all occurrences in command
604 command = command.replace('%(test-input-file)', test_input_filename)
612 command = self.subst(command, '%(test-input-file)', test_input_filename)
605613 command_contained_test_input_file = True
606614
607615 if '%(test-input-text)' in self.command:
608 # escape all single quotes in input
609 body = re.sub(r"'", r"\'", body)
610616 # replace all occurrences in command
611 command = command.replace('%(test-input-text)', input)
617 command = self.subst(command, '%(test-input-text)', input)
612618 command_contained_test_input_text = True
613619
614620 if '%(output-file)' in self.command:
616622 fd, output_filename = mkstemp()
617623 os.close(fd)
618624 # replace all occurrences in command
619 command = command.replace('%(output-file)', output_filename)
625 command = self.subst(command, '%(output-file)', output_filename)
620626
621627 # subshell the command and return the output
622628 pipe = Popen(command, shell=True,
+1
-1
test.sh less more
1212
1313 FIRST_TESTS="
1414 test-pass-fail test-no-functionality test-ill-formed test-no-test-body
15 test-var-subst-no-eol
15 test-var-subst test-no-eol
1616 test-utf8 test-crlf
1717 test-bad-indentation
1818 test-input-sections test-shared-body
+1
-1
tests/test-appliances-b.markdown less more
1212 -> shell command "python cat.py -o %(output-file)"
1313
1414 -> Functionality "Cat" is implemented by
15 -> shell command "python echo.py '%(test-body-text)'"
15 -> shell command "python echo.py %(test-body-text)"
+1
-1
tests/test-err-no-stderr.markdown less more
55 to be on `stdout`.
66
77 -> Functionality "Error on stdout" is implemented by shell command
8 -> "python fail.py '%(test-body-text)' '' 1"
8 -> "python fail.py %(test-body-text) '' 1"
99
1010 -> Tests for functionality "Error on stdout"
1111
+3
-0
tests/test-no-eol.expected less more
0 --------------------------------
1 Total test runs: 4, failures: 0
2 --------------------------------
+31
-0
tests/test-no-eol.markdown less more
0 Falderal Test: Missing EOL at end of output
1 ----------------------------
2
3 It makes no difference whether there is an EOL at the end of
4 the test output or not.
5
6 -> Functionality "Echo" is implemented by
7 -> shell command "python echo.py %(test-body-text)"
8
9 -> Tests for functionality "Echo"
10
11 | hello
12 = hello
13
14 | hi
15 | hi
16 = hi
17 = hi
18
19 -> Functionality "Echo, no newline" is implemented by
20 -> shell command "python echo.py -n %(test-body-text)"
21
22 -> Tests for functionality "Echo, no newline"
23
24 | hello
25 = hello
26
27 | hi
28 | hi
29 = hi
30 = hi
+1
-1
tests/test-no-test-body.markdown less more
0 Falderal Test 4
0 Falderal Test: no test body
11 ---------------
22
33 Another Falderal document which is ill-formed.
+4
-4
tests/test-stdout-stderr.markdown less more
44 expected text is matched against standard output.
55
66 -> Functionality "Succeed" is implemented by shell command
7 -> "python fail.py '%(test-body-text)' bar 0"
7 -> "python fail.py %(test-body-text) bar 0"
88
99 -> Tests for functionality "Succeed"
1010
1515 output and standard error, it's up to you to redirect them.
1616
1717 -> Functionality "Succeed/All" is implemented by shell command
18 -> "python fail.py '%(test-body-text)' bar 0 2>&1"
18 -> "python fail.py %(test-body-text) bar 0 2>&1"
1919
2020 -> Tests for functionality "Succeed/All"
2121
2727 expected text is matched against standard error.
2828
2929 -> Functionality "Fail" is implemented by shell command
30 -> "python fail.py foo '%(test-body-text)' 1"
30 -> "python fail.py foo %(test-body-text) 1"
3131
3232 -> Tests for functionality "Fail"
3333
3838 output and standard error, it's up to you to redirect them.
3939
4040 -> Functionality "Fail/All" is implemented by shell command
41 -> "python fail.py foo '%(test-body-text)' 1 1>&2"
41 -> "python fail.py foo %(test-body-text) 1 1>&2"
4242
4343 -> Tests for functionality "Fail/All"
4444
+1
-1
tests/test-substring-error.markdown less more
66 the expected error text.
77
88 -> Functionality "Fail" is implemented by shell command
9 -> "python fail.py foo '%(test-body-text)' 1"
9 -> "python fail.py foo %(test-body-text) 1"
1010
1111 -> Tests for functionality "Fail"
1212
+0
-3
tests/test-var-subst-no-eol.expected less more
0 --------------------------------
1 Total test runs: 11, failures: 0
2 --------------------------------
+0
-84
tests/test-var-subst-no-eol.markdown less more
0 Falderal Test 5
1 ---------------
2
3 Tests for variable substitution, and missing EOL at end
4 of output.
5
6 Note the use of single quotes around the `%(test-body-text)` variable;
7 without these, shell chaos is likely to result.
8
9 -> Functionality "Echo" is implemented by
10 -> shell command "python echo.py '%(test-body-text)'"
11
12 -> Tests for functionality "Echo"
13
14 | hello
15 = hello
16
17 | hi
18 | hi
19 = hi
20 = hi
21
22 -> Functionality "Echo, no newline" is implemented by
23 -> shell command "python echo.py -n '%(test-body-text)'"
24
25 -> Tests for functionality "Echo, no newline"
26
27 | hello
28 = hello
29
30 | hi
31 | hi
32 = hi
33 = hi
34
35 Note that when variables are expanded, backslash sequences in the
36 replacement string ("\n", etc) are not expanded.
37
38 -> Tests for functionality "Echo"
39
40 | he\nl\tl\\o
41 = he\nl\tl\\o
42
43 Note that single quotes needn't be supplied around `%(test-body-file)`
44 or `%(output-file)`.
45
46 -> Functionality "Cat, from file" is implemented by
47 -> shell command "python cat.py -f %(test-body-file)"
48
49 -> Tests for functionality "Cat, from file"
50
51 | hello
52 = hello
53
54 | hi
55 | hi
56 = hi
57 = hi
58
59 -> Functionality "Cat, to file" is implemented by
60 -> shell command "python cat.py -o %(output-file)"
61
62 -> Tests for functionality "Cat, to file"
63
64 | hello
65 = hello
66
67 | hi
68 | hi
69 = hi
70 = hi
71
72 -> Functionality "Cat, to and from file" is implemented by
73 -> shell command "python cat.py -f %(test-body-file) -o %(output-file)"
74
75 -> Tests for functionality "Cat, to and from file"
76
77 | hello
78 = hello
79
80 | hi
81 | hi
82 = hi
83 = hi
+3
-0
tests/test-var-subst.expected less more
0 --------------------------------
1 Total test runs: 13, failures: 0
2 --------------------------------
+105
-0
tests/test-var-subst.markdown less more
0 Falderal Test: Variable substitution
1 ------------------------------------
2
3 Tests for variable substitution, and quoting rules during such.
4
5 Note that Falderal is responsible for quoting the substitution text
6 of all `%(...)` variables occurring in a shell command template;
7 it is not necessary to put any quotes around them in the template string.
8
9 -> Functionality "Echo Body" is implemented by shell command
10 -> "python echo.py %(test-body-text)"
11
12 -> Tests for functionality "Echo Body"
13
14 | foo
15 + bar
16 = foo
17
18 Single quotes in the test body text are single escaped.
19
20 | don't
21 + can't
22 = don't
23
24 -> Functionality "Echo Input" is implemented by shell command
25 -> "python echo.py %(test-input-text)"
26
27 -> Tests for functionality "Echo Input"
28
29 | foo
30 + bar
31 = bar
32
33 Single quotes in the test input text are single escaped.
34
35 | don't
36 + can't
37 = can't
38
39 Note that when variables are expanded, backslash sequences in the
40 replacement string ("\n", etc) are not expanded.
41
42 -> Tests for functionality "Echo Body"
43
44 | he\nl\tl\\o
45 = he\nl\tl\\o
46
47 The rule that Falderal is responsible for quoting text substituted
48 into the command template extends to `%(test-body-file)` and
49 `%(test-input-file)` and `%(output-file)` as well.
50
51 -> Functionality "Cat, from file" is implemented by
52 -> shell command "python cat.py -f %(test-body-file)"
53
54 -> Tests for functionality "Cat, from file"
55
56 | hello
57 = hello
58
59 | hi
60 | hi
61 = hi
62 = hi
63
64 -> Functionality "Cat, to file" is implemented by
65 -> shell command "python cat.py -o %(output-file)"
66
67 -> Tests for functionality "Cat, to file"
68
69 | hello
70 = hello
71
72 | hi
73 | hi
74 = hi
75 = hi
76
77 -> Functionality "Cat, to and from file" is implemented by
78 -> shell command "python cat.py -f %(test-body-file) -o %(output-file)"
79
80 -> Tests for functionality "Cat, to and from file"
81
82 | hello
83 = hello
84
85 | hi
86 | hi
87 = hi
88 = hi
89
90 -> Functionality "Cat input, from file" is implemented by
91 -> shell command "python cat.py -f %(test-input-file)"
92
93 -> Tests for functionality "Cat input, from file"
94
95 | hekko
96 + hello
97 = hello
98
99 | hj
100 | hj
101 + hi
102 + hi
103 = hi
104 = hi