This fixes a low severity security issue affecting Anymail v0.2--v1.3.
Django error reporting includes the value of your Anymail
WEBHOOK_AUTHORIZATION setting. In a properly-configured deployment,
this should not be cause for concern. But if you have somehow exposed
your Django error reports (e.g., by mis-deploying with DEBUG=True or by
sending error reports through insecure channels), anyone who gains
access to those reports could discover your webhook shared secret. An
attacker could use this to post fabricated or malicious Anymail
tracking/inbound events to your app, if you are using those Anymail
features.
The fix renames Anymail's webhook shared secret setting so that
Django's error reporting mechanism will [sanitize][0] it.
If you are using Anymail's event tracking and/or inbound webhooks, you
should upgrade to this release and change "WEBHOOK_AUTHORIZATION" to
"WEBHOOK_SECRET" in the ANYMAIL section of your settings.py. You may
also want to [rotate the shared secret][1] value, particularly if you
have ever exposed your Django error reports to untrusted individuals.
If you are only using Anymail's EmailBackends for sending email and
have not set up Anymail's webhooks, this issue does not affect you.
The old WEBHOOK_AUTHORIZATION setting is still allowed in this release,
but will issue a system-check warning when running most Django
management commands. It will be removed completely in a near-future
release, as a breaking change.
Thanks to Charlie DeTar (@yourcelf) for responsibly reporting this
security issue through private channels.
[0]: https://docs.djangoproject.com/en/stable/ref/settings/#debug
[1]: https://anymail.readthedocs.io/en/1.4/tips/securing_webhooks/#use-a-shared-authorization-secret
Use a default timeout of 30 seconds for all requests, and add a
REQUESTS_TIMEOUT Anymail setting to override.
(I'm making a judgement call that this is not a breaking change in the
real world, and not bumping the major version. Theoretically, it could
affect you if your network somehow takes >30s to connect to your ESP,
but eventually succeeds. If so, set REQUESTS_TIMEOUT to None to restore
the earlier behavior.)
Fixes#80.
Mailgun merges user-variables (metadata) into the webhook post data
interspersed with the actual event params. This can lead to ambiguity
interpreting post data.
To extract metadata from an event, Anymail had been attempting to avoid
that ambiguity by instead using X-Mailgun-Variables fields found in the
event's message-headers param. But message-headers isn't included in
some tracking events (opened, clicked, unsubscribed), resulting in
empty metadata for those events. (#76)
Also, conflicting metadata keys could confuse Anymail's Mailgun event
parsing, leading to unexpected values in the normalized event. (#77)
This commit:
* Cleans up Anymail's tracking webhook to be explicit about which
multi-value params it uses, avoiding conflicts with metadata keys.
Fixes#77.
* Extracts metadata from post params for opened, clicked and
unsubscribed events. All unknown event params are assumed to be
metadata. Fixes#76.
* Documents a few metadata key names where it's impossible (or likely
to be unreliable) for Anymail to extract metadata from the post data.
For reference, the order of params in the Mailgun's post data *appears*
to be (from live testing):
* For the timestamp, token and signature params, any user-variable with
the same name appears *before* the corresponding event data.
* For all other params, any user-variable with the same name as a
Mailgun event param appears *after* the Mailgun data.
* Remove "pre-1.0" warnings in docs, readme
* Jump trove classifer from pre-alpha all the way up to stable
(arguably this should have been "beta" for the past several months)
* Change Anymail's test EmailBackend to collect sent messages in
django.core.mail.outbox, same as Django's own locmem EmailBackend.
(So Django's test runner will automatically clear accumulated mail
between test cases.)
* Rename EmailMessage `test_response` attr to `anymail_test_response`
to avoid conflicts, and record merged ESP send params in
new `anymail_send_params` attr.
* Add docs
Closes#36.
* Mandrill now refers to "whitelist change" events
(used to be "whitelist sync").
* More details on solving validation failures due to
webhook url mismatches.
* **Future breaking change:**
Rename all Anymail backends to just `EmailBackend`,
matching Django's naming convention.
(E.g., switch to "anymail.backends.mailgun.EmailBackend"
rather than "anymail.backends.mailgun.MailgunBackend".)
The old names still work, but will issue a DeprecationWarning
and will be removed in some future release.
(Apologies for this change; the old naming convention was
a holdover from Djrill, and I wanted consistency with
other Django EmailBackends before hitting 1.0.)
Fixes#49.
SendGrid: update to v3 send API
**SendGrid:** **[possibly-breaking]** Update SendGrid backend to newer Web API v3. This should be a transparent change for most projects. Exceptions: if you use SendGrid username/password auth, esp_extra with "x-smtpapi", or multiple Reply-To addresses, please review the [porting notes](http://anymail.readthedocs.io/en/latest/esps/sendgrid/#sendgrid-v3-upgrade).
Closes#28
Anymail was requiring Mandrill's webhook authentication key for the initial webhook url validation request from Mandrill, but Mandrill doesn't issue the key until that validation request succeeds.
* Defer complaining about missing Mandrill webhook key until actual event post.
* Document the double-deploy process required to set up Mandrill webhooks.
Fixes#46.
Add support for Postmark's recently-released [delivery tracking webhook] to Anymail's normailized status event handling. The existing Anymail tracking webhook URL can be copied to "Delivery webhook" in your Postmark outbound server settings.
Closes#45.
A message's `from_email` and each address in its `to`, `cc`, and `bcc` lists must contain exactly one email address. Previous code would silently ignore additional addresses, leading to unusual behavior. Now, raises new `AnymailInvalidAddress` exception.
Example: `from_email='Widgets, Inc. <widgets@example.com>'` is invalid: it needs double-quotes around the "Widgets, Inc." display-name portion. In earlier versions, this probably would have sent the message from something like "From: Widgets <@localhost>". Now, it will raise an exception.
**Potentially-breaking change:** If your code is using an unquoted display-name containing a comma in an email address, it will now raise an error. In earlier versions, this may have appeared to succeed, but was almost certainly not doing what you intended.
Fixes#44.
* Trailing comma after "anymail" (see #40)
* Note order doesn't matter
* Change tuple to list (match examples to
Django 1.9+ project template)
[ci skip]
Allows clients that use AnymailMessage or AnymailMessageMixin
to access (e.g.) message.anymail_status.message_id without
an AttributeError, even when using a non-Anymail backend
(e.g., during testing).
Also clarify docs of anymail_status attribute to note it only gets
attached to a normal Django EmailMessage when sent through
an Anymail backend.
(May help with situation described in #36)
Allow custom MAILGUN_SENDER_DOMAIN in Anymail
settings. (Replaces need to use global esp_extra.)
Improve docs to cover cases where this is needed.
(esp_extra sender_domain is still supported for
overriding individual messages.)
Fixes#26.
When using a stored template, SparkPost disallows
subject, text, and html. Django's EmailMessage default
empty strings are enough to provoke "Both content
object and template_id are specified" from SparkPost,
so remove them (if empty) when using stored templates.
Update docs and tests; add integration test for template_id.
Fixes#24
Install default warnings filter in test runs, to avoid
python's default suppression of DeprecationWarning
and other "intended for developer" warnings.
(Avoids need to provide -Wd or -Wall arg to runtests.)
* Merge esp_extra with Mandrill send payload
* Handle pythonic forms of `recipient_metadata`
and `template_content` in esp_extra
* DeprecationWarning for Mandrill EmailMessage
attributes inherited from Djrill
* message.template_id to use ESP stored templates
* message.merge_data and merge_global_data
to supply per-recipient/global merge variables
(with or without an ESP stored template)
* When using per-recipient merge_data, tell ESP to use
batch send: individual message per "to" address.
(Mailgun does this automatically; SendGrid requires
using a different "to" field; Mandrill requires
`preserve_recipients=False`; Postmark doesn't
support *this type* of batch sending with merge data.)
* Allow message.from_email=None (must be set after
init) and message.subject=None to suppress those
fields in API calls (for ESPs that allow "From" and
"Subject" in their template definitions).
Mailgun:
* Emulate merge_global_data by copying to
recipient-variables for each recipient.
SendGrid:
* Add delimiters to merge field names via
esp_extra['merge_field_format'] or
ANYMAIL_SENDGRID_MERGE_FIELD_FORMAT setting.
Mandrill:
* Remove Djrill versions of these features;
update migration notes.
Closes#5.
Previously, setting esp_extra['x-smtpapi']['filters']
would override the entire filters setting, potentially
undoing other Anymail options that use SendGrid
filters (like track_opens).
Now, 'filters' is special-cased, and merged with
any other Anymail filter options.
(We don't do a fully deep merge, because otherwise
there would be no way to use esp_extra to *clear*
Anymail settings.)
* Update utils.get_anymail_setting to support
kwargs override of django.conf.settings values
* Use the updated version everywhere
* Switch from ImproperlyConfigured to
AnymailConfigurationError exception
(anticipates feature_wehooks change)
Closes#12
* Add smtp-id in unique_args (metadata), to ensure
it shows up in click and open events.
* Add SENDGRID_GENERATE_MESSAGE_ID setting,
default True, to control auto-Message-ID behavior.
* Document it.
