Tests: limit live API integration tests in Travis runs

To conserve our ESP test accounts' send quotas, don't run
the live API integration tests 13 times in every Travis run.
Instead, just run them twice, on a representative set
of Python/Django combinations:

* Once on Python 2.7 (currently with Django 1.8)
* Once on Python 3.x (currently 3.5 with Django 1.9)

(Prep for running weekly tests on Travis cron.)

The *non*-integration tests still run on all combos.

* Introduce RUN_LIVE_TESTS environment var to control
  whether live API integration test cases should run.
  Default True, except in Travis-CI runs default False.
* Enable RUN_LIVE_TESTS in .travis.yml matrix for the
  Python/Django combos listed above.
This commit is contained in:
medmunds
2016-06-23 15:21:40 -07:00
parent a4f43d17d2
commit 0c5911ca34
7 changed files with 39 additions and 7 deletions

View File

@@ -4,8 +4,12 @@ matrix:
include:
# Anymail supports the same python versions as Django, excluding Python 3.2, but adding pypy.
# https://docs.djangoproject.com/en/dev/faq/install/#what-python-version-can-i-use-with-django
# Live API integration tests are only run on a few, representative Python/Django version
# cominbations, to avoid rapidly consuming the testing accounts' entire send allotments.
# Django 1.8: Python 2.7, 3.2 (until the end of 2016), 3.3, 3.4, 3.5
- { env: DJANGO=django==1.8, python: 2.7 }
- { env: DJANGO=django==1.8 RUN_LIVE_TESTS=true, python: 2.7 }
- { env: DJANGO=django==1.8, python: 3.3 }
- { env: DJANGO=django==1.8, python: 3.4 }
- { env: DJANGO=django==1.8, python: 3.5 }
@@ -13,7 +17,7 @@ matrix:
# Django 1.9: Python 2.7, 3.4, 3.5
- { env: DJANGO=django==1.9, python: 2.7 }
- { env: DJANGO=django==1.9, python: 3.4 }
- { env: DJANGO=django==1.9, python: 3.5 }
- { env: DJANGO=django==1.9 RUN_LIVE_TESTS=true, python: 3.5 }
- { env: DJANGO=django==1.9, python: pypy }
# Django 1.10 (prerelease): Python 2.7, 3.4, 3.5
- { env: DJANGO="--pre django", python: 2.7 }