Test without optional packages

Tox:
* Add Tox factor for extras (all, none, individual ESP).
  For now, only break out ESPs that have specific extra
  dependencies (amazon_ses, sparkpost).
* Install most package dependencies (including extras)
  through the package itself.
* Use new runtests.py environment vars to limit test tags
  when Tox isn't installing all extras.

Travis:
* Rework matrix to request specific TOXENVs directly;
  drop tox-travis.

Test runner (runtests.py):
* Centralize RUN_LIVE_TESTS logic in runtests.py
* Add ANYMAIL_ONLY_TEST and ANYMAIL_SKIP_TESTS env vars
  (comma-separated lists of tags)

Test implementations:
* Tag all ESP-specific tests with ESP
* Tag live tests with "live"
* Don't import ESP-specific packages at test module level. 
  (Test discovery imports test modules before tag-based filtering.)

Closes #104
This commit is contained in:
Mike Edmunds
2019-02-09 15:04:08 -08:00
committed by GitHub
parent 653fdac3cc
commit 978996d7b8
38 changed files with 237 additions and 146 deletions

View File

@@ -71,7 +71,7 @@ and Python versions. Tests are run at least once a week, to check whether ESP AP
and other dependencies have changed out from under Anymail.
For local development, the recommended test command is
:shell:`tox -e django20-py36,django18-py27,lint`, which tests a representative
:shell:`tox -e django21-py36-all,django111-py27-all,lint`, which tests a representative
combination of Python and Django versions. It also runs :pypi:`flake8` and other
code-style checkers. Some other test options are covered below, but using this
tox command catches most problems, and is a good pre-pull-request check.
@@ -104,10 +104,10 @@ with those, `pyenv`_ is a helpful way to install and manage multiple Python vers
.. code-block:: console
$ pip install tox # (if you haven't already)
$ tox -e django20-py36,django18-py27,lint # test recommended environments
$ tox -e django21-py36-all,django111-py27-all,lint # test recommended environments
## you can also run just some test cases, e.g.:
$ tox -e django20-py36,django18-py27 tests.test_mailgun_backend tests.test_utils
$ tox -e django21-py36-all,django111-py27-all tests.test_mailgun_backend tests.test_utils
## to test more Python/Django versions:
$ tox # ALL 20+ envs! (grab a coffee, or use `detox` to run tests in parallel)
@@ -121,7 +121,7 @@ API keys or other settings. For example:
$ export MAILGUN_TEST_API_KEY='your-Mailgun-API-key'
$ export MAILGUN_TEST_DOMAIN='mail.example.com' # sending domain for that API key
$ tox -e django20-py36 tests.test_mailgun_integration
$ tox -e django21-py36-all tests.test_mailgun_integration
Check the ``*_integration_tests.py`` files in the `tests source`_ to see which variables
are required for each ESP. Depending on the supported features, the integration tests for