And decide not to work around a Python 3.3 bug accessing MIME headers
that have non-ASCII characters in params. The bug is fixed in the
Python 3.4 email package (and didn't exist in Python 2.7). Python 3.3
was only supported with Django 1.8.
In AnymailInboundMessage, work around Python 2 email.parser.Parser's
lack of handling for RFC2047-encoded email headers. (The Python 3 email
package already decodes these automatically.)
Improves inbound handling on Python 2 for all ESPs that provide raw
MIME email or raw headers with inbound events. (Mailgun, Mandrill,
SendGrid, SparkPost.)
Work around Python 2 email.parser.Parser bug handling RFC5322 folded
headers. Fixes problems where long headers in inbound mail (e.g.,
Subject) get truncated or have unexpected spaces.
This change also updates AnymailInboundMessage.parse_raw_mime to use
the improved "default" email.policy on Python 3 (rather than the
default "compat32" policy). This likely fixes several other parsing
bugs that will still affect code running on Python 2.
Improves inbound parsing for all ESPs that provide raw MIME email.
(Mailgun, Mandrill, SendGrid, SparkPost)
* Add vertical space between items in "open" lists
(rtfd/sphinx_rtd_theme#590)
* Distinguish shell prompts in console examples,
and omit them if the code is copied
* Add css and js extras directly from Sphinx conf.py
(no need to override template)
* Set up tox for testing supported Django/Python combinations
* Also include tox env for checking and building docs
* Use tox-travis for Travis CI integration
* Add tests against Django master
* Document building docs and running tests with tox
Using undeliverable @example.com recipient addresses leads some ESPs
to flag the Anymail test accounts. Switch all live integration tests
to mailinator.com recipients (unless they were already using the ESP's
own "test sink" addresses).
* Don't send *quite* so many emails during live integration tests.
(Our test account is throttled to 40/hour.)
* Relax message_id check in integration tests. SendinBlue appears
to use both @smtp-relay.mail.fr and @smtp-relay.sendinblue.com
Message-IDs.
* Note requirement for HTML message body in docs.
* "SSL" --> "https"
* "authorization" --> "authentication"
(e.g., "HTTP basic authentication" -- except when referring
specifically to the HTTP "Authorization" header used to send it)
* add a sidebar with more details on why it matters
* Flesh out SendinBlue docs, add a readme mention
* Stop trying to list all the supported ESPs in the project short
description and similar headlines -- it was becoming unwieldy.
* Support `pip install django-anymail[sendinblue]`
and use it in Travis tests (for consistency; SendinBlue
doesn't require any extra packages)
Drop support for the WEBHOOK_AUTHORIZATION setting deprecated in v1.4.
Only the WEBHOOK_SECRET replacement is allowed now.
Most Django management commands will now issue a system check error
if the old name is still used in settings.py
* Follow current setup.py recommendations from the pypa sample project
(utf-8 encoding on file reads, ensure files are read relative to
own location)
* Add/update some missing classifiers and other metadata
* Read _version.py constants into an isolated dict (rather than
the global setup.py context)
* Add setup.cfg specifying universal bdist_wheel
Django's SMTP EmailBackend allows spoofing the To header by setting
`message.extra_headers["To"]`` different from `message.to`.
No current Anymail ESP supports this. Treat extra_headers["To"] as
an unsupported ESP feature, to flag attempts to use it.
Also document Anymail's special header handling that replicates
Django's SMTP EmailBackend behavior.
New EmailMessage attribute `envelope_sender` controls ESP's sender,
sending domain, or return path where supported:
* Mailgun: overrides SENDER_DOMAIN on individual message
(domain portion only)
* Mailjet: becomes `Sender` API param
* Mandrill: becomes `return_path_domain` API param
(domain portion only)
* SparkPost: becomes `return_path` API param
* Other ESPs: not believed to be supported
Also support undocumented Django SMTP backend behavior, where envelope
sender is given by `message.from_email` when
`message.extra_headers["From"]` is set. Fixes#91.
Django allows setting the reply address with either message.reply_to
or message.extra_headers["Reply-To"]. If both are supplied, the extra
headers version takes precedence. (See EmailMessage.message().)
Several Anymail backends had duplicate logic to handle conflicting
properties. Move that logic into the base Payload.
(Also prepares for common handling of extra_headers['From'], later.)
Related changes:
* Use CaseInsensitiveDict for processing extra_headers.
This is potentially a breaking change, but any code that was trying
to send multiple headers differing only in case was likely already
broken. (Email header field names are case-insensitive, per RFC-822.)
* Handle CaseInsensitiveDict in RequestsPayload.serialize_json().
(Several backends had duplicate code for handling this, too.)
* Fixes SparkPost backend, which had been incorrectly treating
message.reply_to and message.extra_headers['Reply-To'] differently.
Add support for sending transactional email through SendinBlue. (Thanks to @RignonNoel.)
Partially implements #84. (Tracking webhooks will be a separate PR. SendinBlue doesn't support inbound handling.)
This fixes a low severity security issue affecting Anymail v0.2--v1.3.
Django error reporting includes the value of your Anymail
WEBHOOK_AUTHORIZATION setting. In a properly-configured deployment,
this should not be cause for concern. But if you have somehow exposed
your Django error reports (e.g., by mis-deploying with DEBUG=True or by
sending error reports through insecure channels), anyone who gains
access to those reports could discover your webhook shared secret. An
attacker could use this to post fabricated or malicious Anymail
tracking/inbound events to your app, if you are using those Anymail
features.
The fix renames Anymail's webhook shared secret setting so that
Django's error reporting mechanism will [sanitize][0] it.
If you are using Anymail's event tracking and/or inbound webhooks, you
should upgrade to this release and change "WEBHOOK_AUTHORIZATION" to
"WEBHOOK_SECRET" in the ANYMAIL section of your settings.py. You may
also want to [rotate the shared secret][1] value, particularly if you
have ever exposed your Django error reports to untrusted individuals.
If you are only using Anymail's EmailBackends for sending email and
have not set up Anymail's webhooks, this issue does not affect you.
The old WEBHOOK_AUTHORIZATION setting is still allowed in this release,
but will issue a system-check warning when running most Django
management commands. It will be removed completely in a near-future
release, as a breaking change.
Thanks to Charlie DeTar (@yourcelf) for responsibly reporting this
security issue through private channels.
[0]: https://docs.djangoproject.com/en/stable/ref/settings/#debug
[1]: https://anymail.readthedocs.io/en/1.4/tips/securing_webhooks/#use-a-shared-authorization-secret
Anymail's webhook validation was vulnerable to a timing attack.
An attacker could have used this to recover your WEBHOOK_AUTHORIZATION
shared secret, potentially allowing them to post fabricated or malicious
email tracking events to your app.
There have not been any reports of attempted exploit in the wild. (The
vulnerability was discovered through code review.) Attempts would be
visible in http logs as a very large number of 400 responses on
Anymail's webhook urls, or in Python error monitoring as a very large
number of AnymailWebhookValidationFailure exceptions.
If you are using Anymail's webhooks, you should upgrade to this release.
In addition, you may want to rotate to a new WEBHOOK_AUTHORIZATION
secret ([docs](http://anymail.readthedocs.io/en/stable/tips/securing_webhooks/#use-a-shared-authorization-secret)),
particularly if your logs indicate attempted exploit.
* Un-hardcode status message_id in test backend
For the test EmailBackend, get message ID's based on array position in
`mail.outbox`, so that tests can predict the message ID.
* Add a console backend for use in development
Adds an EmailBackend derived from both Anymail's test backend and
Django's console backend, to provide anymail statuses and signal
handling while printing messages to the console. For use during
development on localhost.
Closes#87
Use a default timeout of 30 seconds for all requests, and add a
REQUESTS_TIMEOUT Anymail setting to override.
(I'm making a judgement call that this is not a breaking change in the
real world, and not bumping the major version. Theoretically, it could
affect you if your network somehow takes >30s to connect to your ESP,
but eventually succeeds. If so, set REQUESTS_TIMEOUT to None to restore
the earlier behavior.)
Fixes#80.
get_api_call_arg had incorrectly returned None if a kwarg was passed
to the mocked function with a False-y value (e.g., [] or {})
get_api_call_json had only considered data param, ignoring json param
requests added a while back
Within an EmailAddress (previously ParsedEmail object), properties
now match Python 3.6 email.headerregistry.Address naming:
* .email --> .addr_spec
* .name --> .display_name
* .localpart --> .username
(Completes work started in 386668908423d1d4eade90cf7a21a546a1e96514;
this updates remaining uses of old names and removes them.)
Mailgun merges user-variables (metadata) into the webhook post data
interspersed with the actual event params. This can lead to ambiguity
interpreting post data.
To extract metadata from an event, Anymail had been attempting to avoid
that ambiguity by instead using X-Mailgun-Variables fields found in the
event's message-headers param. But message-headers isn't included in
some tracking events (opened, clicked, unsubscribed), resulting in
empty metadata for those events. (#76)
Also, conflicting metadata keys could confuse Anymail's Mailgun event
parsing, leading to unexpected values in the normalized event. (#77)
This commit:
* Cleans up Anymail's tracking webhook to be explicit about which
multi-value params it uses, avoiding conflicts with metadata keys.
Fixes#77.
* Extracts metadata from post params for opened, clicked and
unsubscribed events. All unknown event params are assumed to be
metadata. Fixes#76.
* Documents a few metadata key names where it's impossible (or likely
to be unreliable) for Anymail to extract metadata from the post data.
For reference, the order of params in the Mailgun's post data *appears*
to be (from live testing):
* For the timestamp, token and signature params, any user-variable with
the same name appears *before* the corresponding event data.
* For all other params, any user-variable with the same name as a
Mailgun event param appears *after* the Mailgun data.
Update internal-use ParsedEmail to be more like Python 3.6+
email.headerregistry.Address, and remove "internal use only"
recommendation.
(Prep for exposing inbound email headers in a convenient form.
Old names remain temporarily available for internal use;
should clean up at some point.)
SendGrid requires extra headers and metadata values be strings.
Anymail has always coerced int and float; this treats Python 2's
`long` integer type the same.
Fixes#74
* Remove "pre-1.0" warnings in docs, readme
* Jump trove classifer from pre-alpha all the way up to stable
(arguably this should have been "beta" for the past several months)
