Test without optional packages

Tox:
* Add Tox factor for extras (all, none, individual ESP).
  For now, only break out ESPs that have specific extra
  dependencies (amazon_ses, sparkpost).
* Install most package dependencies (including extras)
  through the package itself.
* Use new runtests.py environment vars to limit test tags
  when Tox isn't installing all extras.

Travis:
* Rework matrix to request specific TOXENVs directly;
  drop tox-travis.

Test runner (runtests.py):
* Centralize RUN_LIVE_TESTS logic in runtests.py
* Add ANYMAIL_ONLY_TEST and ANYMAIL_SKIP_TESTS env vars
  (comma-separated lists of tags)

Test implementations:
* Tag all ESP-specific tests with ESP
* Tag live tests with "live"
* Don't import ESP-specific packages at test module level. 
  (Test discovery imports test modules before tag-based filtering.)

Closes #104
This commit is contained in:
Mike Edmunds
2019-02-09 15:04:08 -08:00
committed by GitHub
parent 653fdac3cc
commit 978996d7b8
38 changed files with 237 additions and 146 deletions

View File

@@ -10,9 +10,15 @@ branches:
- master
- /^v\d+\.\d+(\.\d+)?(-\S*)?$/
env:
global:
# Let Travis report failures that tox.ini would normally ignore:
- TOX_FORCE_IGNORE_OUTCOME=false
matrix:
include:
- { env: LINT_AND_DOCS=true, python: 3.6 }
- python: 3.6
env: TOXENV="lint,docs"
# Anymail supports the same Python versions as Django, plus PyPy.
# https://docs.djangoproject.com/en/dev/faq/install/#what-python-version-can-i-use-with-django
@@ -21,36 +27,41 @@ matrix:
# combinations, to avoid rapidly consuming the testing accounts' entire send allotments.
# Django 1.11: Python 2.7, 3.4, 3.5, or 3.6
- { env: DJANGO=1.11 RUN_LIVE_TESTS=true, python: 2.7 }
- { env: DJANGO=1.11, python: 3.4 }
- { env: DJANGO=1.11, python: 3.5 }
- { env: DJANGO=1.11, python: 3.6 }
- { env: DJANGO=1.11, python: pypy2.7-6.0 }
- { env: TOXENV=django111-py27-all RUN_LIVE_TESTS=true, python: 2.7 }
- { env: TOXENV=django111-py34-all, python: 3.4 }
- { env: TOXENV=django111-py35-all, python: 3.5 }
- { env: TOXENV=django111-py36-all, python: 3.6 }
- { env: TOXENV=django111-pypy-all, python: pypy2.7-6.0 }
# Django 2.0: Python 3.5+
- { env: DJANGO=2.0, python: 3.5 }
- { env: DJANGO=2.0, python: 3.6 }
- { env: DJANGO=2.0, python: pypy3.5-6.0 }
- { env: TOXENV=django20-py35-all, python: 3.5 }
- { env: TOXENV=django20-py36-all, python: 3.6 }
- { env: TOXENV=django20-pypy3-all, python: pypy3.5-6.0 }
# Django 2.1: Python 3.5, 3.6, or 3.7
- { env: DJANGO=2.1, python: 3.5 }
- { env: DJANGO=2.1 RUN_LIVE_TESTS=true, python: 3.6 }
- { env: DJANGO=2.1, python: 3.7 }
- { env: DJANGO=2.1, python: pypy3.5-6.0 }
- { env: TOXENV=django21-py35-all, python: 3.5 }
- { env: TOXENV=django21-py36-all RUN_LIVE_TESTS=true, python: 3.6 }
- { env: TOXENV=django21-py37-all, python: 3.7 }
- { env: TOXENV=django21-pypy3-all, python: pypy3.5-6.0 }
# Django 2.2: Python 3.5, 3.6, or 3.7
- { env: DJANGO=2.2, python: 3.5 }
- { env: DJANGO=2.2, python: 3.6 }
- { env: DJANGO=2.2, python: 3.7 }
- { env: DJANGO=2.2, python: pypy3.5-6.0 }
- { env: TOXENV=django22-py35-all, python: 3.5 }
- { env: TOXENV=django22-py36-all, python: 3.6 }
- { env: TOXENV=django22-py37-all, python: 3.7 }
- { env: TOXENV=django22-pypy3-all, python: pypy3.5-6.0 }
# Django development master (direct from GitHub source):
- { env: DJANGO=master, python: 3.6 }
- { env: TOXENV=djangoMaster-py36-all, python: 3.6 }
# Install without optional extras (don't need to cover entire matrix)
- { env: TOXENV=django21-py37-none, python: 3.7 }
- { env: TOXENV=django21-py37-amazon_ses, python: 3.7 }
- { env: TOXENV=django21-py37-sparkpost, python: 3.7 }
allow_failures:
- env: DJANGO=master
- env: TOXENV=djangoMaster-py36-all
python: 3.6
cache: pip
install:
- pip install tox-travis
# pin tox to avoid https://github.com/tox-dev/tox/issues/1160
- pip install tox~=3.6.1
script:
- tox