Test without optional packages

Tox:
* Add Tox factor for extras (all, none, individual ESP).
  For now, only break out ESPs that have specific extra
  dependencies (amazon_ses, sparkpost).
* Install most package dependencies (including extras)
  through the package itself.
* Use new runtests.py environment vars to limit test tags
  when Tox isn't installing all extras.

Travis:
* Rework matrix to request specific TOXENVs directly;
  drop tox-travis.

Test runner (runtests.py):
* Centralize RUN_LIVE_TESTS logic in runtests.py
* Add ANYMAIL_ONLY_TEST and ANYMAIL_SKIP_TESTS env vars
  (comma-separated lists of tags)

Test implementations:
* Tag all ESP-specific tests with ESP
* Tag live tests with "live"
* Don't import ESP-specific packages at test module level. 
  (Test discovery imports test modules before tag-based filtering.)

Closes #104
This commit is contained in:
Mike Edmunds
2019-02-09 15:04:08 -08:00
committed by GitHub
parent 653fdac3cc
commit 978996d7b8
38 changed files with 237 additions and 146 deletions

62
tox.ini
View File

@@ -1,17 +1,20 @@
[tox]
envlist =
# Factors: django-python-extras
# Test these environments first, to catch most errors early...
lint
django21-py36
django111-py27
django21-py36-all
django111-py27-all
docs
# ... then test all the other supported combinations:
django21-py{35,37,py3}
django20-py{35,36,py3}
django111-py{34,35,36,py}
django21-py{35,37,py3}-all
django20-py{35,36,py3}-all
django111-py{34,35,36,py}-all
# ... then prereleases (if available):
django22-py{35,36,37,py3}
djangoMaster-py{36,37}
django22-py{35,36,37,py3}-all
djangoMaster-py{36,37}-all
# ... then partial installation (limit extras):
django21-py37-{none,amazon_ses,sparkpost}
[testenv]
deps =
@@ -20,19 +23,27 @@ deps =
django21: django~=2.1.0
django22: django>=2.2a1
djangoMaster: https://github.com/django/django/tarball/master
# testing dependencies (duplicates setup.py tests_require):
# testing dependencies (duplicates setup.py tests_require, less optional extras):
mock
boto3
sparkpost
extras =
all,amazon_ses: amazon_ses
all,sparkpost: sparkpost
setenv =
# tell runtests.py to limit some test tags based on extras factor
none: ANYMAIL_SKIP_TESTS=amazon_ses,sparkpost
amazon_ses: ANYMAIL_ONLY_TEST=amazon_ses
sparkpost: ANYMAIL_ONLY_TEST=sparkpost
ignore_outcome =
djangoMaster: True
usedevelop = True
args_are_paths = False
# CI that wants to handle errors itself can set TOX_FORCE_IGNORE_OUTCOME=false
djangoMaster: {env:TOX_FORCE_IGNORE_OUTCOME:true}
args_are_paths = false
commands_pre =
python -VV
commands =
python --version
# pip install .[mailgun,...,sparkpost] ## usedevelop=True + manual deps is much faster on repeat runs
python runtests.py {posargs}
passenv =
ANYMAIL_ONLY_TEST
ANYMAIL_SKIP_TESTS
RUN_LIVE_TESTS
CONTINUOUS_INTEGRATION
AMAZON_SES_TEST_*
@@ -46,7 +57,7 @@ passenv =
[testenv:lint]
basepython = python3
skip_install = True
skip_install = true
passenv =
CONTINUOUS_INTEGRATION
# (but not any of the live test API keys)
@@ -59,7 +70,7 @@ commands =
[testenv:docs]
basepython = python3
skip_install = True
skip_install = true
passenv =
CONTINUOUS_INTEGRATION
# (but not any of the live test API keys)
@@ -78,20 +89,3 @@ commands =
/bin/bash -c 'python setup.py --long-description \
| rst2html5.py --config=docs/_readme/docutils.cfg \
> {env:DOCS_BUILD_DIR}/readme.html'
[travis]
unignore_outcomes = True
python =
3.6: py36, lint, docs
[travis:env]
DJANGO =
1.11: django111
2.0: django20
2.1: django21
2.2: django22
master: djangoMaster
LINT_AND_DOCS =
true: lint, docs
docs: docs
lint: lint