Fix a crash or text-mangling issue when an inbound message
uses a charset other than utf-8 for its text or html body,
and SendGrid's "post raw" inbound parse option is *not*
enabled.
Update docs to recommend "post raw" option.
Fixes#187
Additional changes related to SendinBlue improvements in #158:
* Support multiple tags in webhooks (closes#162)
* Remove additional outdated template code in backend
* Update integration tests
* Update docs and changelog; note breaking changes as discussed in #161
Map Mailgun severity: temporary failure event to Anymail "deferred" event, to distinguish it from severity: permanent failures which will show up as Anymail "bounced".
Also remap Mailgun reason: generic failure to Anymail "other" reject reason (rather than "bounced").
Closes#130
SendGrid does not always correctly provide the sent Message-ID header value
to a tracking webhook's smtp-id field, making it unreliable to use for Anymail's
`message_id`.
Instead, generate a UUID `message_id` for Anymail tracking, and pass it from
send to webhooks in SendGrid custom args as anymail_id.
Webhooks will fall back to smtp-id for compatibility with previously-sent
messages that didn't have an anymail_id custom arg.
Fixes#108
Delay raising AnymailImproperlyInstalled from webhooks.amazon_ses
until an SES webhook view is instantiated. Allows anymail.urls
to import webhooks.amazon_ses without error.
Fixes#103
Simplify Postmark tracking webhook code by using new "RecordType"
field introduced with Postmark "modular webhooks". (Rather than
looking for fields that are probably only in certain events.)
Also issue configuration error on inbound url installed as tracking
webhook (and vice versa).
Drop support for the WEBHOOK_AUTHORIZATION setting deprecated in v1.4.
Only the WEBHOOK_SECRET replacement is allowed now.
Most Django management commands will now issue a system check error
if the old name is still used in settings.py
This fixes a low severity security issue affecting Anymail v0.2--v1.3.
Django error reporting includes the value of your Anymail
WEBHOOK_AUTHORIZATION setting. In a properly-configured deployment,
this should not be cause for concern. But if you have somehow exposed
your Django error reports (e.g., by mis-deploying with DEBUG=True or by
sending error reports through insecure channels), anyone who gains
access to those reports could discover your webhook shared secret. An
attacker could use this to post fabricated or malicious Anymail
tracking/inbound events to your app, if you are using those Anymail
features.
The fix renames Anymail's webhook shared secret setting so that
Django's error reporting mechanism will [sanitize][0] it.
If you are using Anymail's event tracking and/or inbound webhooks, you
should upgrade to this release and change "WEBHOOK_AUTHORIZATION" to
"WEBHOOK_SECRET" in the ANYMAIL section of your settings.py. You may
also want to [rotate the shared secret][1] value, particularly if you
have ever exposed your Django error reports to untrusted individuals.
If you are only using Anymail's EmailBackends for sending email and
have not set up Anymail's webhooks, this issue does not affect you.
The old WEBHOOK_AUTHORIZATION setting is still allowed in this release,
but will issue a system-check warning when running most Django
management commands. It will be removed completely in a near-future
release, as a breaking change.
Thanks to Charlie DeTar (@yourcelf) for responsibly reporting this
security issue through private channels.
[0]: https://docs.djangoproject.com/en/stable/ref/settings/#debug
[1]: https://anymail.readthedocs.io/en/1.4/tips/securing_webhooks/#use-a-shared-authorization-secret
Anymail's webhook validation was vulnerable to a timing attack.
An attacker could have used this to recover your WEBHOOK_AUTHORIZATION
shared secret, potentially allowing them to post fabricated or malicious
email tracking events to your app.
There have not been any reports of attempted exploit in the wild. (The
vulnerability was discovered through code review.) Attempts would be
visible in http logs as a very large number of 400 responses on
Anymail's webhook urls, or in Python error monitoring as a very large
number of AnymailWebhookValidationFailure exceptions.
If you are using Anymail's webhooks, you should upgrade to this release.
In addition, you may want to rotate to a new WEBHOOK_AUTHORIZATION
secret ([docs](http://anymail.readthedocs.io/en/stable/tips/securing_webhooks/#use-a-shared-authorization-secret)),
particularly if your logs indicate attempted exploit.
Mailgun merges user-variables (metadata) into the webhook post data
interspersed with the actual event params. This can lead to ambiguity
interpreting post data.
To extract metadata from an event, Anymail had been attempting to avoid
that ambiguity by instead using X-Mailgun-Variables fields found in the
event's message-headers param. But message-headers isn't included in
some tracking events (opened, clicked, unsubscribed), resulting in
empty metadata for those events. (#76)
Also, conflicting metadata keys could confuse Anymail's Mailgun event
parsing, leading to unexpected values in the normalized event. (#77)
This commit:
* Cleans up Anymail's tracking webhook to be explicit about which
multi-value params it uses, avoiding conflicts with metadata keys.
Fixes#77.
* Extracts metadata from post params for opened, clicked and
unsubscribed events. All unknown event params are assumed to be
metadata. Fixes#76.
* Documents a few metadata key names where it's impossible (or likely
to be unreliable) for Anymail to extract metadata from the post data.
For reference, the order of params in the Mailgun's post data *appears*
to be (from live testing):
* For the timestamp, token and signature params, any user-variable with
the same name appears *before* the corresponding event data.
* For all other params, any user-variable with the same name as a
Mailgun event param appears *after* the Mailgun data.
Mandrill's webhook signature calculation uses the
*exact url* Mandrill is posting to. If HTTP basic
auth is also used, that auth is included in the url.
Anymail was using Django's request.build_absolute_uri,
which doesn't include HTTP basic auth. Anymail now
includes the auth in the calculation, if it was present
in the request.
This should eliminate the need to use the
ANYMAIL_MANDRILL_WEBHOOK_URL override,
if Django's SECURE_PROXY_SSL_HEADER and
USE_X_FORWARDED_HOST (and/or
USE_X_FORWARDED_PROTO) settings are correct
for your server.
(The calculated url is now also included in
the validation failure error message, to aid
debugging.)
Fixes#48
Anymail was requiring Mandrill's webhook authentication key for the initial webhook url validation request from Mandrill, but Mandrill doesn't issue the key until that validation request succeeds.
* Defer complaining about missing Mandrill webhook key until actual event post.
* Document the double-deploy process required to set up Mandrill webhooks.
Fixes#46.
Add support for Postmark's recently-released [delivery tracking webhook] to Anymail's normailized status event handling. The existing Anymail tracking webhook URL can be copied to "Delivery webhook" in your Postmark outbound server settings.
Closes#45.
Compatibility with Python 2.7 versions older than 2.7.7
* Use Django's constant_time_compare method
* Include sparkpost in test requirements
* Don't use non-public `EnvironmentVarGuard` in tests
Fixes#41
* csrf_exempt must be applied to View.dispatch,
not View.post.
* In base WebhookTestCase, enable Django test Client
enforce_csrf_checks. (Test Client by default disables
CSRF protection.)
Closes#19
