 | CramA simple testing framework for command-line applications |
| Download | |
Cram Ranking & Summary
Advertisement
Cram Tags
Cram Description
A simple testing framework for command-line applications Cram is a functional testing framework for command-line applications, based on Mercurial's unified test format.Here's a snippet from cram.t in Cram's own test suite:The $PYTHON environment variable should be set when running thistest from Python. $ || PYTHON=python $ if ; then > coverage erase > alias cram='coverage run -a cram.py' > else > alias cram="$PYTHON cram.py" > fi $ command -v md5 || alias md5=md5sumUsage: $ cram -h sage: cram \ TESTS\.\.\. ptions: -h, --help show this help message and exit -v, --verbose show filenames and test status -i, --interactive interactively merge changed test output -y, --yes answer yes to all questions -n, --no answer no to all questions -D DIR, --tmpdir=DIR run tests in DIR --keep-tmpdir keep temporary directories -E don't reset common environment variables $ cram sage: cram \ TESTS\.\.\. The format in a nutshell: * Cram tests use the .t file extension. * Lines beginning with two spaces, a dollar sign, and a space are run in the shell. * Lines beginning with two spaces, a greater than sign, and a space allow multi-line commands. * All other lines beginning with two spaces are considered command output. * Command output in the test is first matched literally with the actual output. If it doesn't match, it's then compiled and matched as a Perl-compatible regular expression. * Command output in the test that ends with a percent sign will match actual output that doesn't end in a newline. * Anything else is a comment.InstallationYou can use pip to install Cram: sudo pip install cramOr you can install Cram the old fashioned way: wget http://bitheap.org/cram/cram-0.3.tar.gz tar zxvf cram-0.3.tar.gz cd cram-0.3.tar.gz sudo python setup.py installUsageCram will print a dot for each passing test. If a test fails, a unified context diff is printed showing the test's expected output and the actual output.For example, if we run cram on its own example tests: cram examples..--- examples/fail.t+++ examples/fail.t.out@@ -3,11 +3,11 @@ $ echo 1 1 $ echo 1- 2+ 1 $ echo 1 1 Invalid regex: $ echo 1- ++++ 1..Cram will also write the test with its actual output to examples/fail.t.err.When you're first writing a test, you might just write the commands and run the test to see what happens. If you run Cram with -i or --interactive, you'll be prompted to merge the actual output back into the test. This makes it easy to quickly prototype new tests.Note that the following environment variables are reset before tests are run: * TMPDIR, TEMP, and TMP are set to the test runner's tmp directory. * LANG, LC_ALL, and LANGUAGE are set to C. * TZ is set to GMT. * COLUMNS is set to 80. * CDPATH and GREP_OPTIONS are set to an empty string.Cram also provides the following environment variables to tests: * RUNDIR, set to the directory Cram was run from. * TESTDIR, set to the test runner's temporary directory.DevelopmentDownload the official development repository using Mercurial:hg clone http://bitbucket.org/brodie/cramTest Cram using Cram:make testsGet a test coverage report using coverage.py:make coverageVisit Bitbucket if you'd like to fork the project, watch for new changes, or report issues. Requirements: · Python What's New in This Release: · The test format has changed: Output lines containing regular expressions must now end in "`` (re)``" or they'll be matched literally. Lines ending with keywords are matched literally first, however. · Regular expressions are now matched from beginning to end. In other words "`` d (re)``" is matched as "^\d$". · In addition to (re), (glob) has been added. It supports *, ?, and escaping both characters (and backslashes) using \. · Environment settings have changed: The -D flag has been removed, $TESTDIR is now set to the directory containing the .t file, and $CRAMTMP is set to the test runner's temporary directory. · -i/--interactive now requires patch(1). Instead of the .err files replacing the original test file when answering yes to a merge, the diff output is ran through patch(1) this prevents matching regular expressions and globs from getting clobbered. · Previous .err files are now removed when tests pass. · Cram now exits with return code 1 if any tests failed. · If a test exits with return code 80, it's considered a skipped a test. This is useful for intentionally disabling tests when they only work on certain platforms or in certain settings. · The number of tests, the number of skipped tests, and the number of failed tests are now printed after all tests are finished. · Added -q/--quiet to suppress diff output. · Added contrib/cram.vim syntax file for Vim. Contributed by Steve Losh.
Cram Related Software