Commit 7dd8a18ca5c575e5b5a745f197305d51c9eb70be - Falderal

Unless --cavalier is given, lint loaded tests for sanity. --HG-- rename : tests/test10.markdown => tests/test-no-tests.markdown Cat's Eye Technologies 11 years ago

6 changed file(s) with 71 addition(s) and 14 deletion(s). Raw diff Collapse all Expand all

-0

HISTORY.markdown less more

18	18	are now created where the system wants them to be created (but you can set
19	19	the `TMP` environment variable to "." if you really want them to be created
20	20	in the current directory.)
	21	* `py-falderal` now complains and stops early if there were no documents
	22	specified, no functionality definitions were found in any of the documents,
	23	no implementations were found for some defined functionality, or if no
	24	tests were found in any of the documents, unless `--cavalier` is passed on
	25	the command line.
21	26
22	27	Version 0.8 "Ukranian Village" (current released version):
23	28

-9

TODO.markdown less more

131	131	having both kinds of expectations would allow non-deterministic combinations
132	132	of the two to be matched.)
133	133
134		### lint test documents by default
135
136		Unless some command line option is given (call it `--cavalier`, maybe,) if
137		any of the following is true, we should abort quickly with an error message:
138
139		- Parsing the given documents resulted in no tests. (This is implied by:
140		there were no documents given.)
141		- For one or more functionalities, there were no implementations specified.
142
143	134	### catch KeyboardInterrupt, report on all results so far
144	135
145	136	When a test suite takes a long time to run, it would be very nice if

+32

-4

src/falderal/driver.py less more

7	7	from falderal.objects import Document, FalderalSyntaxError
8	8
9	9
	10	##### Exceptions #####
	11
	12	class FalderalLintingError(ValueError):
	13	pass
	14
	15
	16	##### Main #####
	17
10	18	def main(args):
11	19	parser = OptionParser()
12	20
13	21	parser.add_option("-b", "--substring-error",
14	22	action="store_true", default=False,
15	23	help="match expected errors as substrings")
	24	parser.add_option("--cavalier",
	25	action="store_true", default=False,
	26	help="don't perform sanity linting before running tests")
16	27	parser.add_option("-d", "--dump",
17	28	action="store_true", default=False,
18	29	help="print out info about parsed tests, don't run them")

49	60	documents.append(Document.load(filename))
50	61	for document in documents:
51	62	tests += document.parse_blocks_to_tests(functionalities)
52		except FalderalSyntaxError as e:
	63
	64	if not options.cavalier:
	65	if not documents:
	66	raise FalderalLintingError("No test documents were specified")
	67	if not functionalities:
	68	raise FalderalLintingError(
	69	"No functionalities were found in any of the test documents"
	70	)
	71	for name in functionalities:
	72	functionality = functionalities[name]
	73	if not functionality.implementations:
	74	raise FalderalLintingError(
	75	"No implementations were found for the functionality '%s'" %
	76	functionality.name
	77	)
	78	if not tests:
	79	raise FalderalLintingError(
	80	"No tests were found in any of the test documents"
	81	)
	82
	83	except (FalderalSyntaxError, FalderalLintingError) as e:
53	84	# if options.show_full_exception: do that, else
54	85	sys.stderr.write('%s: %s\n' % (e.__class__.__name__, str(e)))
55	86	return 1
56	87
57		# XXX lint: check for no tests, or no implementations of a functionality
58		# that is being tested, or a functionality that is not being tested, and
59		# break with an error unless some option to suppress this is present
60	88
61	89	if options.dump:
62	90	print "Functionalities:"

+13

-1

test.sh less more

4	4	bin/falderal -t \|\| exit 1
5	5
6	6	cd tests
7		for TEST in test1 test2 test3 test4 test5 test9 test10 test11 test-utf8 test-crlf; do
	7
	8	FIRST_TESTS="
	9	test1 test2 test3 test4 test5 test9 test10 test11 test-utf8 test-crlf
	10	"
	11	for TEST in ${FIRST_TESTS}; do
	12	echo ${TEST}...
	13	../bin/falderal --cavalier ${TEST}.markdown > ${TEST}.actual 2>&1
	14	diff -u ${TEST}.expected ${TEST}.actual \|\| exit 1
	15	done
	16
	17	# tests for listing
	18	LINTING_TESTS="test-no-tests"
	19	for TEST in ${LINTING_TESTS}; do
8	20	echo ${TEST}...
9	21	../bin/falderal ${TEST}.markdown > ${TEST}.actual 2>&1
10	22	diff -u ${TEST}.expected ${TEST}.actual \|\| exit 1

-0

tests/test-no-tests.expected less more

FalderalLintingError: No functionalities were found in any of the test documents

+20

-0

tests/test-no-tests.markdown less more

	0	Falderal Test for No Tests Found
	1	--------------------------------
	2
	3	Since these blocks are not indented correctly, `py-falderal` should not
	4	load tests from them. (Ideally it should produce warnings though.)
	5
	6	-> Functionality "Cat" is implemented by
	7	-> shell command "python cat.py"
	8
	9	Cat cats.
	10
	11	\| meow
	12	= meow
	13
	14	\| purr
	15	\| prrr
	16	\| prreow
	17	= purr
	18	= prrr
	19	= prreow