pacnpal/django-anymail
1
0
Fork 0
mirror of https://github.com/pacnpal/django-anymail.git synced 2025-12-20 03:41:05 -05:00
Code Issues Packages Projects Releases Wiki Activity

Event-tracking webhooks

Browse Source 
Closes #3
This commit is contained in:
medmunds
2016-04-20 17:13:55 -07:00
parent 36461e57b9
commit d3f914be12
31 changed files with 2451 additions and 428 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
docs/contributing.rst
View File

@@ -24,6 +24,8 @@ folks from `brack3t`_ who developed the original version of Djrill.
.. _Djrill: https://github.com/brack3t/Djrill
.. _reporting-bugs:
Bugs
----

5
docs/esps/index.rst
View File

@@ -36,13 +36,13 @@ Email Service Provider |Mailgun| |Mandrill| |Postmark|
:attr:`~AnymailMessage.track_clicks` Yes Yes No Yes
:attr:`~AnymailMessage.track_opens` Yes Yes Yes Yes
.. rubric:: :ref:`Status <esp-send-status>` and tracking
.. rubric:: :ref:`Status <esp-send-status>` and :ref:`event tracking <event-tracking>`
-------------------------------------------------------------------------------------------
:attr:`~AnymailMessage.anymail_status` Yes Yes Yes Yes
|AnymailTrackingEvent| from webhooks Yes Yes Yes Yes
=========================================== ========= ========== ========== ==========
.. Status tracking webhooks (coming)...
.. .. rubric:: :ref:`inbound`
.. -------------------------------------------------------------------------------------------
.. Inbound webhooks (coming)...
@@ -56,6 +56,7 @@ meaningless. (And even specific features don't matter if you don't plan to use t
.. |Mandrill| replace:: :ref:`mandrill-backend`
.. |Postmark| replace:: :ref:`postmark-backend`
.. |SendGrid| replace:: :ref:`sendgrid-backend`
.. |AnymailTrackingEvent| replace:: :class:`~anymail.signals.AnymailTrackingEvent`
Other ESPs

32
docs/esps/mailgun.rst
View File

@@ -103,3 +103,35 @@ values directly to Mailgun. You can use any of the (non-file) parameters listed
}
.. _Mailgun sending docs: https://documentation.mailgun.com/api-sending.html#sending
.. _mailgun-webhooks:
Status tracking webhooks
------------------------
If you are using Anymail's normalized :ref:`status tracking <event-tracking>`, enter
the url in your `Mailgun dashboard`_ on the "Webhooks" tab. Mailgun allows you to enter
a different URL for each event type: just enter this same Anymail tracking URL
for all events you want to receive:
:samp:`https://{random}:{random}@{yoursite.example.com}/anymail/mailgun/tracking/`
* *random:random* is an :setting:`ANYMAIL_WEBHOOK_AUTHORIZATION` shared secret
* *yoursite.example.com* is your Django site
If you use multiple Mailgun sending domains, you'll need to enter the webhook
URLs for each of them, using the selector on the left side of Mailgun's dashboard.
Mailgun implements a limited form of webhook signing, and Anymail will verify
these signatures (based on your :setting:`MAILGUN_API_KEY <ANYMAIL_MAILGUN_API_KEY>`
Anymail setting).
Mailgun will report these Anymail :attr:`~anymail.signals.AnymailTrackingEvent.event_type`\s:
delivered, rejected, bounced, complained, unsubscribed, opened, clicked.
The event's :attr:`~anymail.signals.AnymailTrackingEvent.esp_event` field will be
a Django :class:`~django.http.QueryDict` object of `Mailgun event fields`_.
.. _Mailgun dashboard: https://mailgun.com/app/dashboard
.. _Mailgun event fields: https://documentation.mailgun.com/user_manual.html#webhooks

92
docs/esps/mandrill.rst
View File

@@ -56,6 +56,31 @@ root of the settings file if neither ``ANYMAIL["MANDRILL_API_KEY"]``
nor ``ANYMAIL_MANDRILL_API_KEY`` is set.
.. setting:: ANYMAIL_MANDRILL_WEBHOOK_KEY
.. rubric:: MANDRILL_WEBHOOK_KEY
Required if using Anymail's webhooks. The "webhook authentication key"
issued by Mandrill.
`More info <https://mandrill.zendesk.com/hc/en-us/articles/205583257>`_
in Mandrill's KB.
.. setting:: ANYMAIL_MANDRILL_WEBHOOK_URL
.. rubric:: MANDRILL_WEBHOOK_URL
Required only if using Anymail's webhooks *and* the hostname your
Django server sees is different from the public webhook URL
you provided Mandrill. (E.g., if you have a proxy in front
of your Django server that forwards
"https\://yoursite.example.com" to "http\://localhost:8000/").
If you are seeing :exc:`AnymailWebhookValidationFailure` errors
from your webhooks, set this to the exact webhook URL you entered
in Mandrill's settings.
.. setting:: ANYMAIL_MANDRILL_API_URL
.. rubric:: MANDRILL_API_URL
@@ -76,6 +101,41 @@ Anymail's Mandrill backend does not yet implement the
:attr:`~anymail.message.AnymailMessage.esp_extra` feature.
.. _mandrill-webhooks:
Status tracking webhooks
------------------------
If you are using Anymail's normalized :ref:`status tracking <event-tracking>`,
follow `Mandrill's instructions`_ to add Anymail's webhook URL:
:samp:`https://{random}:{random}@{yoursite.example.com}/anymail/mandrill/tracking/`
* *random:random* is an :setting:`ANYMAIL_WEBHOOK_AUTHORIZATION` shared secret
* *yoursite.example.com* is your Django site
Be sure to check the boxes in the Mandrill settings for the event types you want to receive.
The same Anymail tracking URL can handle all Mandrill "message" and "sync" events.
Mandrill implements webhook signing on the entire event payload, and Anymail will
verify the signature. You must set :setting:`ANYMAIL_MANDRILL_WEBHOOK_KEY` to the
webhook key authentication key issued by Mandrill. You may also need to set
:setting:`ANYMAIL_MANDRILL_WEBHOOK_URL` depending on your server config.
Mandrill will report these Anymail :attr:`~anymail.signals.AnymailTrackingEvent.event_type`\s:
sent, rejected, deferred, bounced, opened, clicked, complained, unsubscribed. Mandrill does
not support delivered events. Mandrill "whitelist" and "blacklist" sync events will show up
as Anymail's unknown event_type.
The event's :attr:`~anymail.signals.AnymailTrackingEvent.esp_event` field will be
a `dict` of Mandrill event fields, for a single event. (Although Mandrill calls
webhooks with batches of events, Anymail will invoke your signal receiver separately
for each event in the batch.)
.. _Mandrill's instructions:
https://mandrill.zendesk.com/hc/en-us/articles/205583217-Introduction-to-Webhooks
.. _migrating-from-djrill:
Migrating from Djrill
@@ -123,6 +183,16 @@ Changes to settings
(or just `IGNORE_RECIPIENT_STATUS` in the :setting:`ANYMAIL`
settings dict).
``DJRILL_WEBHOOK_SECRET`` and ``DJRILL_WEBHOOK_SECRET_NAME``
Replaced with HTTP basic auth. See :ref:`securing-webhooks`.
``DJRILL_WEBHOOK_SIGNATURE_KEY``
Use :setting:`ANYMAIL_MANDRILL_WEBHOOK_KEY` instead.
``DJRILL_WEBHOOK_URL``
Use :setting:`ANYMAIL_MANDRILL_WEBHOOK_URL`, or eliminate if
your Django server is not behind a proxy that changes hostnames.
Changes to EmailMessage attributes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -182,3 +252,25 @@ Changes to EmailMessage attributes
Or better yet, use Anymail's new :ref:`inline-images`
helper functions to attach your inline images.
Changes to webhooks
~~~~~~~~~~~~~~~~~~~
Anymail uses HTTP basic auth as a shared secret for validating webhook
calls, rather than Djrill's "secret" query parameter. See
:ref:`securing-webhooks`. (A slight advantage of basic auth over query
parameters is that most logging and analytics systems are aware of the
need to keep auth secret.)
Anymail replaces `djrill.signals.webhook_event` with
`anymail.signals.tracking` and (in a future release)
`anymail.signals.inbound`. Anymail parses and normalizes
the event data passed to the signal receiver: see :ref:`event-tracking`.
The equivalent of Djrill's ``data`` parameter is available
to your signal receiver as
:attr:`event.esp_event <anymail.signals.AnymailTrackingEvent.esp_event>`,
and for most events, the equivalent of Djrill's ``event_type`` parameter
is `event.esp_event['event']`. But consider working with Anymail's
normalized :class:`~anymail.signals.AnymailTrackingEvent` instead.

34
docs/esps/postmark.rst
View File

@@ -113,3 +113,37 @@ see :ref:`unsupported-features`.
**No delayed sending**
Postmark does not support :attr:`~anymail.message.AnymailMessage.send_at`.
.. _postmark-webhooks:
Status tracking webhooks
------------------------
If you are using Anymail's normalized :ref:`status tracking <event-tracking>`, enter
the url in your `Postmark account settings`_, under Servers > *your server name* >
Settings > Outbound > Webhooks. You should enter this same Anymail tracking URL
for both the "Bounce webhook" and "Opens webhook" (if you want to receive both
types of events):
:samp:`https://{random}:{random}@{yoursite.example.com}/anymail/postmark/tracking/`
* *random:random* is an :setting:`ANYMAIL_WEBHOOK_AUTHORIZATION` shared secret
* *yoursite.example.com* is your Django site
Anymail doesn't care about the "include bounce content" and "post only on first open"
Postmark webhook settings: whether to use them is your choice.
If you use multiple Postmark servers, you'll need to repeat entering the webhook
settings for each of them.
Postmark will report these Anymail :attr:`~anymail.signals.AnymailTrackingEvent.event_type`\s:
rejected, failed, bounced, deferred, autoresponded, opened, complained, unsubscribed, subscribed.
(Postmark does not support sent, delivered, or clicked events.)
The event's :attr:`~anymail.signals.AnymailTrackingEvent.esp_event` field will be
a `dict` of Postmark `bounce <http://developer.postmarkapp.com/developer-bounce-webhook.html>`_
or `open <http://developer.postmarkapp.com/developer-open-webhook.html>`_ webhook data.
.. _Postmark account settings: https://account.postmarkapp.com/servers

28
docs/esps/sendgrid.rst
View File

@@ -174,3 +174,31 @@ Limitations and quirks
actually OK with that.)
(Tested March, 2016)
.. _sendgrid-webhooks:
Status tracking webhooks
------------------------
If you are using Anymail's normalized :ref:`status tracking <event-tracking>`, enter
the url in your `SendGrid mail settings`_, under "Event Notification":
:samp:`https://{random}:{random}@{yoursite.example.com}/anymail/sendgrid/tracking/`
* *random:random* is an :setting:`ANYMAIL_WEBHOOK_AUTHORIZATION` shared secret
* *yoursite.example.com* is your Django site
Be sure to check the boxes in the SendGrid settings for the event types you want to receive.
SendGrid will report these Anymail :attr:`~anymail.signals.AnymailTrackingEvent.event_type`\s:
queued, rejected, bounced, deferred, delivered, opened, clicked, complained, unsubscribed,
subscribed.
The event's :attr:`~anymail.signals.AnymailTrackingEvent.esp_event` field will be
a `dict` of `Sendgrid event`_ fields, for a single event. (Although SendGrid calls
webhooks with batches of events, Anymail will invoke your signal receiver separately
for each event in the batch.)
.. _SendGrid mail settings: https://app.sendgrid.com/settings/mail_settings
.. _Sendgrid event: https://sendgrid.com/docs/API_Reference/Webhooks/event.html

6
docs/inbound.rst
View File

@@ -8,11 +8,5 @@ Receiving inbound email
Normalized inbound email handling is coming soon to Anymail.
.. _inbound-webhooks:
Configuring inbound webhooks
----------------------------
Inbound email signals
---------------------

158
docs/installation.rst
View File

@@ -41,6 +41,9 @@ To use Anymail for sending email, edit your Django project's :file:`settings.py`
"MAILGUN_API_KEY" = "<your Mailgun key>",
}
The exact settings vary by ESP.
See the :ref:`supported ESPs <supported-esps>` section for specifics.
3. Change your existing Django :setting:`EMAIL_BACKEND` to the Anymail backend
for your ESP. For example, to send using Mailgun by default:
@@ -52,39 +55,140 @@ To use Anymail for sending email, edit your Django project's :file:`settings.py`
use :ref:`multiple Anymail backends <multiple-backends>` to send particular
messages through different ESPs.)
The exact backend name and required settings vary by ESP.
See the :ref:`supported ESPs <supported-esps>` section for specifics.
Also, if you don't already have a :setting:`DEFAULT_FROM_EMAIL` in your settings,
Finally, if you don't already have a :setting:`DEFAULT_FROM_EMAIL` in your settings,
this is a good time to add one. (Django's default is "webmaster\@localhost",
which some ESPs will reject.)
With the settings above, you are ready to send outgoing email through your ESP.
If you also want to enable status tracking or inbound email, continue with the
optional settings below. Otherwise, skip ahead to :ref:`sending-email`.
Configuring status tracking webhooks
------------------------------------
Anymail can optionally connect to your ESPs event webhooks to notify your app
.. _webhooks-configuration:
Configuring status tracking webhooks (optional)
-----------------------------------------------
Anymail can optionally connect to your ESP's event webhooks to notify your app
of status like bounced and rejected emails, successful delivery, message opens
and clicks, and other tracking.
If you aren't using Anymail's webhooks, skip this section.
.. warning::
Webhooks are ordinary urls, and are wide open to the internet.
You must use care to **avoid creating security vulnerabilities**
that could expose your users' emails and other private information,
or subject your app to malicious input data.
At a minimum, your site should **use SSL** (https), and you should
configure **webhook authorization** as described below.
See :ref:`securing-webhooks` for additional information.
If you want to use Anymail's status tracking webhooks, follow the steps above
to :ref:`configure an Anymail backend <backend-configuration>`, and then
follow the instructions in the :ref:`event-tracking` section to set up
the delivery webhooks.
to :ref:`configure an Anymail backend <backend-configuration>`, and then:
1. In your :file:`settings.py`, add
:setting:`WEBHOOK_AUTHORIZATION <ANYMAIL_WEBHOOK_AUTHORIZATION>`
to the ``ANYMAIL`` block:
.. code-block:: python
ANYMAIL = {
...
'WEBHOOK_AUTHORIZATION': '<a random string>:<another random string>',
}
This setting should be a string with two sequences of random characters,
separated by a colon. It is used as a shared secret, known only to your ESP
and your Django app, to ensure nobody else can call your webhooks.
We suggest using 16 characters (or more) for each half of the
secret. Always generate a new, random secret just for this purpose.
(*Don't* use your Django secret key or ESP's API key.)
An easy way to generate a random secret is to run this command in
a shell:
.. code-block:: console
$ python -c "from django.utils import crypto; print(':'.join(crypto.get_random_string(16) for _ in range(2)))"
(This setting is actually an HTTP basic auth string. You can also set it
to a list of auth strings, to simplify credential rotation or use different auth
with different ESPs. See :setting:`ANYMAIL_WEBHOOK_AUTHORIZATION` in the
:ref:`securing-webhooks` docs for more details.)
Configuring inbound email
-------------------------
2. In your project's :file:`urls.py`, add routing for the Anymail webhook urls:
Anymail can optionally connect to your ESPs inbound webhook to notify your app
of inbound messages.
.. code-block:: python
If you want to use inbound email with Anymail, first follow the first two
:ref:`backend configuration <backend-configuration>` steps above. (You can
skip changing your :setting:`EMAIL_BACKEND` if you don't want to us Anymail
for *sending* messages.) Then follow the instructions in the
:ref:`inbound-webhooks` section to set up the inbound webhooks.
from django.conf.urls import include, url
urlpatterns = [
...
url(r'^anymail/', include('anymail.urls')),
]
(You can change the "anymail" prefix in the first parameter to
:func:`~django.conf.urls.url` if you'd like the webhooks to be served
at some other URL. Just match whatever you use in the webhook URL you give
your ESP in the next step.)
3. Enter the webhook URL(s) into your ESP's dashboard or control panel.
In most cases, the URL will be:
:samp:`https://{random}:{random}@{yoursite.example.com}/anymail/{esp}/tracking/`
* "https" (rather than http) is *strongly recommended*
* *random:random* is the WEBHOOK_AUTHORIZATION string you created in step 1
* *yoursite.example.com* is your Django site
* "anymail" is the url prefix (from step 2)
* *esp* is the lowercase name of your ESP (e.g., "sendgrid" or "mailgun")
* "tracking" is used for Anymail's sent-mail event tracking webhooks
Some ESPs support different webhooks for different tracking events. You can
usually enter the same Anymail webhook URL for all of them (or all that you
want to receive). But be sure to check the specific details for your ESP
under :ref:`supported-esps`.
Also, some ESPs try to validate the webhook URL immediately when you enter it.
If so, you'll need to deploy your Django project to your live server before you
can complete this step.
See :ref:`event-tracking` for information on creating signal handlers and the
status tracking events you can receive.
.. _inbound-configuration:
Configuring inbound email (optional)
------------------------------------
(Coming soon -- not yet implemented)
.. Anymail can optionally connect to your ESP's inbound webhook to notify your app
.. of incoming messages.
..
.. If you aren't using your EPS's inbound email, skip this section.
..
.. If you want to use inbound email with Anymail, follow the steps above
.. for setting up :ref:`status tracking webhooks <webhooks-configuration>`,
.. but enter the webhook URL in your ESP's "inbound email" settings,
.. substituting "inbound" for "tracking" at the end of the url:
..
.. :samp:`https://{random}:{random}@{yoursite.example.com}/anymail/{esp}/inbound/`
..
.. Then see :ref:`inbound` for information on creating a signal handler
.. for receiving inbound email notifications in your code.
..
.. (Note: if you are only using your ESP for inbound email, not sending messages,
.. there's no need to change your project's EMAIL_BACKEND.)
.. setting:: ANYMAIL
@@ -163,3 +267,19 @@ Whether Anymail should raise :exc:`~anymail.exceptions.AnymailUnsupportedFeature
errors for email with features that can't be accurately communicated to the ESP.
Set to `True` to ignore these problems and send the email anyway. See
:ref:`unsupported-features`. (Default `False`.)
.. rubric:: WEBHOOK_AUTHORIZATION
A `'random:random'` shared secret string. Anymail will reject incoming webhook calls
from your ESP that don't include this authorization. You can also give a list of
shared secret strings, and Anymail will allow ESP webhook calls that match any of them
(to facilitate credential rotation). See :ref:`securing-webhooks`.
Default is unset, which leaves your webhooks insecure. Anymail
will warn if you try to use webhooks with setting up authorization.
This is actually implemented using HTTP basic authorization, and the string is
technically a "username:password" format. But you should *not* use any real
username or password for this shared secret.

396
docs/sending/tracking.rst
View File

@@ -1,178 +1,272 @@
.. module:: anymail.signals
.. _event-tracking:
Tracking sent mail status
=========================
.. note::
Anymail provides normalized handling for your ESP's event-tracking webhooks.
You can use this to be notified when sent messages have been delivered,
bounced, been opened or had links clicked, among other things.
Normalized event-tracking webhooks and signals are coming
to Anymail soon.
Webhook support is optional. If you haven't yet, you'll need to
:ref:`configure webhooks <webhooks-configuration>` in your Django
project. (You may also want to review :ref:`securing-webhooks`.)
Once you've enabled webhooks, Anymail will send a ``anymail.signals.tracking``
custom Django :mod:`signal <django.dispatch>` for each ESP tracking event it receives.
You can connect your own receiver function to this signal for further processing.
Be sure to read Django's `listening to signals`_ docs for information on defining
and connecting signal receivers.
Example:
.. code-block:: python
from anymail.signals import tracking
from django.dispatch import receiver
@receiver(tracking) # add weak=False if inside some other function/class
def handle_bounce(sender, event, esp_name, **kwargs):
if event.event_type == 'bounced':
print("Message %s to %s bounced" % (
event.message_id, event.recipient))
@receiver(tracking)
def handle_click(sender, event, esp_name, **kwargs):
if event.event_type == 'clicked':
print("Recipient %s clicked url %s" % (
event.recipient, event.click_url))
You can define individual signal receivers, or create one big one for all
event types, which ever you prefer. You can even handle the same event
in multiple receivers, if that makes your code cleaner. These
:ref:`signal receiver functions <signal-receivers>` are documented
in more detail below.
Note that your tracking signal recevier(s) will be called for all tracking
webhook types you've enabled at your ESP, so you should always check the
:attr:`~AnymailTrackingEvent.event_type` as shown in the examples above
to ensure you're processing the expected events.
Some ESPs batch up multiple events into a single webhook call. Anymail will
invoke your signal receiver once, separately, for each event in the batch.
.. `Mandrill webhooks`_ are used for notification about outbound messages
.. (bounces, clicks, etc.), and also for delivering inbound email
.. processed through Mandrill.
..
.. Djrill includes optional support for Mandrill's webhook notifications.
.. If enabled, it will send a Django signal for each event in a webhook.
.. Your code can connect to this signal for further processing.
Normalized tracking event
-------------------------
.. _Mandrill webhooks: http://help.mandrill.com/entries/21738186-Introduction-to-Webhooks
.. class:: AnymailTrackingEvent
.. _webhooks-config:
The `event` parameter to Anymail's `tracking`
:ref:`signal receiver <signal-receivers>`
is an object with the following attributes:
Configuring tracking webhooks
-----------------------------
.. attribute:: event_type
.. warning:: Webhook Security
A normalized `str` identifying the type of tracking event.
Webhooks are ordinary urls---they're wide open to the internet.
You must take steps to secure webhooks, or anyone could submit
random (or malicious) data to your app simply by invoking your
webhook URL. For security:
.. note::
* Your webhook should only be accessible over SSL (https).
(This is beyond the scope of Anymail.)
Most ESPs will send some, but *not all* of these event types.
Check the :ref:`specific ESP <supported-esps>` docs for more
details. In particular, very few ESPs implement the "sent" and
"delivered" events.
* Your webhook must include a random, secret key, known only to your
app and your ESP. Anymail will verify calls to your webhook, and will
reject calls without the correct key.
One of:
.. * You can, optionally include the two settings :setting:`DJRILL_WEBHOOK_SIGNATURE_KEY`
.. and :setting:`DJRILL_WEBHOOK_URL` to enforce `webhook signature`_ checking
* `'queued'`: the ESP has accepted the message
and will try to send it (possibly at a later time).
* `'sent'`: the ESP has sent the message
(though it may or may not get successfully delivered).
* `'rejected'`: the ESP refused to send the messsage
(e.g., because of a suppression list, ESP policy, or invalid email).
Additional info may be in :attr:`reject_reason`.
* `'failed'`: the ESP was unable to send the message
(e.g., because of an error rendering an ESP template)
* `'bounced'`: the message was rejected or blocked by receiving MTA
(message transfer agent---the receiving mail server).
* `'deferred'`: the message was delayed by in transit
(e.g., because of a transient DNS problem, a full mailbox, or
certain spam-detection strategies).
The ESP will keep trying to deliver the message, and should generate
a separate `'bounced'` event if later it gives up.
* `'delivered'`: the message was accepted by the receiving MTA.
(This does not guarantee the user will see it. For example, it might
still be classified as spam.)
* `'autoresponded'`: a robot sent an automatic reply, such as a vacation
notice, or a request to prove you're a human.
* `'opened'`: the user opened the message (used with your ESP's
:attr:`~anymail.message.AnymailMessage.track_opens` feature).
* `'clicked'`: the user clicked a link in the message (used with your ESP's
:attr:`~anymail.message.AnymailMessage.track_clicks` feature).
* `'complained'`: the recipient reported the message as spam.
* `'unsubscribed'`: the recipient attempted to unsubscribe
(when you are using your ESP's subscription management features).
* `'subscribed'`: the recipient attempted to subscribe to a list,
or undo an earlier unsubscribe (when you are using your ESP's
subscription management features).
* `'unknown'`: anything else. Anymail isn't able to normalize this event,
and you'll need to examine the raw :attr:`esp_event` data.
.. _webhook signature: http://help.mandrill.com/entries/23704122-Authenticating-webhook-requests
.. attribute:: message_id
A `str` unique identifier for the message, matching the
:attr:`message.anymail_status.message_id <anymail.message.AnymailStatus.message_id>`
attribute from when the message was sent.
The exact format of the string varies by ESP. (It may or may not be
an actual "Message-ID", and is often some sort of UUID.)
.. attribute:: timestamp
A `~datetime.datetime` indicating when the event was generated.
(The timezone is often UTC, but the exact behavior depends on your ESP and
account settings. Anymail ensures that this value is an *aware* datetime
with an accurate timezone.)
.. attribute:: event_id
A `str` unique identifier for the event, if available; otherwise `None`.
Can be used to avoid processing the same event twice. Exact format varies
by ESP, and not all ESPs provide an event_id for all event types.
.. attribute:: recipient
The `str` email address of the recipient. (Just the "recipient\@example.com"
portion.)
.. attribute:: metadata
A `dict` of unique data attached to the message, or `None`.
(See :attr:`AnymailMessage.metadata <anymail.message.AnymailMessage.metadata>`.)
.. attribute:: tags
A `list` of `str` tags attached to the message, or `None`.
(See :attr:`AnymailMessage.tags <anymail.message.AnymailMessage.tags>`.)
.. attribute:: reject_reason
For `'bounced'` and `'rejected'` events, a normalized `str` giving the reason
for the bounce/rejection. Otherwise `None`. One of:
* `'invalid'`: bad email address format.
* `'bounced'`: bounced recipient. (In a `'rejected'` event, indicates the
recipient is on your ESP's prior-bounces suppression list.)
* `'timed_out'`: your ESP is giving up after repeated transient
delivery failures (which may have shown up as `'deferred'` events).
* `'blocked'`: your ESP's policy prohibits this recipient.
* `'spam'`: the receiving MTA or recipient determined the message is spam.
(In a `'rejected'` event, indicates the recipient is on your ESP's
prior-spam-complaints suppression list.)
* `'unsubscribed'`: the recipient is in your ESP's unsubscribed
suppression list.
* `'other'`: some other reject reason; examine the raw :attr:`esp_event`.
* `None`: Anymail isn't able to normalize a reject/bounce reason for
this ESP.
.. note::
Not all ESPs provide all reject reasons, and this area is often
under-documented by the ESP. Anymail does its best to interpret
the ESP event, but you may find (e.g.,) that it will report
`'timed_out'` for one ESP, and `'bounced'` for another, sending
to the same non-existent mailbox.
We appreciate :ref:`bug reports <reporting-bugs>` with the raw
:attr:`esp_event` data in cases where Anymail is getting it wrong.
.. attribute:: description
If available, a `str` with a (usually) human-readable description of the event.
Otherwise `None`. For example, might explain why an email has bounced. Exact
format varies by ESP (and sometimes event type).
.. attribute:: mta_response
If available, a `str` with a raw (intended for email administrators) response
from the receiving MTA. Otherwise `None`. Often includes SMTP response codes,
but the exact format varies by ESP (and sometimes receiving MTA).
.. attribute:: user_agent
For `'opened'` and `'clicked'` events, a `str` identifying the browser and/or
email client the user is using, if available. Otherwise `None`.
.. attribute:: click_url
For `'clicked'` events, the `str` url the user clicked. Otherwise `None`.
.. attribute:: esp_event
The "raw" event data from the ESP, deserialized into a python data structure.
For most ESPs this is either parsed JSON (as a `dict`), or HTTP POST fields
(as a Django :class:`~django.http.QueryDict`).
This gives you (non-portable) access to additional information provided by
your ESP. For example, some ESPs include geo-IP location information with
open and click events.
.. To enable Djrill webhook processing you need to create and set a webhook
.. secret in your project settings, include the Djrill url routing, and
.. then add the webhook in the Mandrill control panel.
..
.. 1. In your project's :file:`settings.py`, add a :setting:`DJRILL_WEBHOOK_SECRET`:
..
.. .. code-block:: python
..
.. DJRILL_WEBHOOK_SECRET = "<create your own random secret>"
..
.. substituting a secret you've generated just for Mandrill webhooks.
.. (Do *not* use your Mandrill API key or Django SECRET_KEY for this!)
..
.. An easy way to generate a random secret is to run the command below in a shell:
..
.. .. code-block:: console
..
.. $ python -c "from django.utils import crypto; print crypto.get_random_string(16)"
..
..
.. 2. In your base :file:`urls.py`, add routing for the Djrill urls:
..
.. .. code-block:: python
..
.. urlpatterns = patterns('',
.. ...
.. url(r'^djrill/', include(djrill.urls)),
.. )
..
..
.. 3. Now you need to tell Mandrill about your webhook:
..
.. * For receiving events on sent messages (e.g., bounces or clickthroughs),
.. you'll do this in Mandrill's `webhooks control panel`_.
.. * For setting up inbound email through Mandrill, you'll add your webhook
.. to Mandrill's `inbound settings`_ under "Routes" for your domain.
.. * And if you want both, you'll need to add the webhook in both places.
..
.. In all cases, the "Post to URL" is
.. :samp:`{https://yoursite.example.com}/djrill/webhook/?secret={your-secret}`
.. substituting your app's own domain, and changing *your-secret* to the secret
.. you created in step 1.
..
.. (For sent-message webhooks, don't forget to tick the "Trigger on Events"
.. checkboxes for the events you want to receive.)
..
..
.. Once you've completed these steps and your Django app is live on your site,
.. you can use the Mandrill "Test" commands to verify your webhook configuration.
.. Then see the next section for setting up Django signal handlers to process
.. the webhooks.
..
.. Incidentally, you have some control over the webhook url.
.. If you'd like to change the "djrill" prefix, that comes from
.. the url config in step 2. And if you'd like to change
.. the *name* of the "secret" query string parameter, you can set
.. :setting:`DJRILL_WEBHOOK_SECRET_NAME` in your :file:`settings.py`.
..
.. For extra security, Mandrill provides a signature in the request header
.. X-Mandrill-Signature. If you want to verify this signature, you need to provide
.. the settings :setting:`DJRILL_WEBHOOK_SIGNATURE_KEY` with the webhook-specific
.. signature key that can be found in the Mandrill admin panel and
.. :setting:`DJRILL_WEBHOOK_URL` where you should enter the exact URL, including
.. that you entered in Mandrill when creating the webhook.
.. _signal-receivers:
.. _webhooks control panel: https://mandrillapp.com/settings/webhooks
.. _inbound settings: https://mandrillapp.com/inbound
Signal receiver functions
-------------------------
Your Anymail signal receiver must be a function with this signature:
.. _webhook-usage:
.. function:: def my_handler(sender, event, esp_name, **kwargs):
Tracking event signals
----------------------
(You can name it anything you want.)
.. Once you've enabled webhooks, Djrill will send a ``djrill.signals.webhook_event``
.. custom `Django signal`_ for each Mandrill event it receives.
.. You can connect your own receiver function to this signal for further processing.
..
.. Be sure to read Django's `listening to signals`_ docs for information on defining
.. and connecting signal receivers.
..
.. Examples:
..
.. .. code-block:: python
..
.. from djrill.signals import webhook_event
.. from django.dispatch import receiver
..
.. @receiver(webhook_event)
.. def handle_bounce(sender, event_type, data, **kwargs):
.. if event_type == 'hard_bounce' or event_type == 'soft_bounce':
.. print "Message to %s bounced: %s" % (
.. data['msg']['email'],
.. data['msg']['bounce_description']
.. )
..
.. @receiver(webhook_event)
.. def handle_inbound(sender, event_type, data, **kwargs):
.. if event_type == 'inbound':
.. print "Inbound message from %s: %s" % (
.. data['msg']['from_email'],
.. data['msg']['subject']
.. )
..
.. @receiver(webhook_event)
.. def handle_whitelist_sync(sender, event_type, data, **kwargs):
.. if event_type == 'whitelist_add' or event_type == 'whitelist_remove':
.. print "Rejection whitelist update: %s email %s (%s)" % (
.. data['action'],
.. data['reject']['email'],
.. data['reject']['reason']
.. )
..
..
.. Note that your webhook_event signal handlers will be called for all Mandrill
.. webhook callbacks, so you should always check the `event_type` param as shown
.. in the examples above to ensure you're processing the expected events.
..
.. Mandrill batches up multiple events into a single webhook call.
.. Djrill will invoke your signal handler once for each event in the batch.
..
.. The available fields in the `data` param are described in Mandrill's documentation:
.. `sent-message webhooks`_, `inbound webhooks`_, and `whitelist/blacklist sync webooks`_.
:param class sender: The source of the event. (One of the
:mod:`anymail.webhook.*` View classes, but you
generally won't examine this parameter; it's
required by Django's signal mechanism.)
:param AnymailTrackingEvent event: The normalized tracking event.
Almost anything you'd be interested in
will be in here.
:param str esp_name: e.g., "SendMail" or "Postmark". If you are working
with multiple ESPs, you can use this to distinguish
ESP-specific handling in your shared event processing.
:param \**kwargs: Required by Django's signal mechanism
(to support future extensions).
.. _Django signal: https://docs.djangoproject.com/en/stable/topics/signals/
.. _inbound webhooks:
http://help.mandrill.com/entries/22092308-What-is-the-format-of-inbound-email-webhooks-
:returns: nothing
:raises: any exceptions in your signal receiver will result
in a 400 HTTP error to the webhook. See discussion
below.
If (any of) your signal receivers raise an exception, Anymail
will discontinue processing the current batch of events and return
an HTTP 400 error to the ESP. Most ESPs respond to this by re-sending
the event(s) later, a limited number of times.
This is the desired behavior for transient problems (e.g., your
Django database being unavailable), but can cause confusion in other
error cases. You may want to catch some (or all) exceptions
in your signal receiver, log the problem for later follow up,
and allow Anymail to return the normal 200 success response
to your ESP.
Some ESPs impose strict time limits on webhooks, and will consider
them failed if they don't respond within (say) five seconds.
And will retry sending the "failed" events, which could cause duplicate
processing in your code.
If your signal receiver code might be slow, you should instead
queue the event for later, asynchronous processing (e.g., using
something like `Celery`_).
If your signal receiver function is defined within some other
function or instance method, you *must* use the `weak=False`
option when connecting it. Otherwise, it might seem to work at first,
but will unpredictably stop being called at some point---typically
on your production server, in a hard-to-debug way. See Django's
`listening to signals`_ docs for more information.
.. _Celery: http://www.celeryproject.org/
.. _listening to signals:
https://docs.djangoproject.com/en/stable/topics/signals/#listening-to-signals
.. _sent-message webhooks: http://help.mandrill.com/entries/21738186-Introduction-to-Webhooks
.. _whitelist/blacklist sync webooks:
https://mandrill.zendesk.com/hc/en-us/articles/205583297-Sync-Event-Webhook-format

1
docs/tips/index.rst
View File

@@ -9,6 +9,7 @@ done with Anymail:
multiple_backends
django_templates
securing_webhooks
.. TODO:
.. Working with django-mailer(2)

109
docs/tips/securing_webhooks.rst Normal file
View File

@@ -0,0 +1,109 @@
.. _securing-webhooks:
Securing webhooks
=================
If not used carefully, webhooks can create security vulnerabilities
in your Django application.
At minimum, you should **use SSL** and a **shared authorization secret**
for your Anymail webhooks. (Really, for *any* webhooks.)
Use SSL
-------
Your Django site must use SSL, and the webhook URLs you
give your ESP should start with "https" (not http).
Without https, the data your ESP sends your webhooks is exposed in transit.
This can include your customers' email addresses, the contents of messages
you receive through your ESP, the shared secret used to authorize calls
to your webhooks (described in the next section), and other data you'd
probably like to keep private.
Configuring SSL is beyond the scope of Anymail, but there are many good
tutorials on the web.
If you aren't able to use https on your Django site, then you should
not set up your ESP's webhooks.
.. setting:: ANYMAIL_WEBHOOK_AUTHORIZATION
Use a shared authorization secret
---------------------------------
A webhook is an ordinary URL---anyone can post anything to it.
To avoid receiving random (or malicious) data in your webhook,
you should use a shared random secret that your ESP can present
with webhook data, to prove the post is coming from your ESP.
Most ESPs recommend using HTTP basic authorization as this shared
secret. Anymail includes support for this, via the
:setting:`!ANYMAIL_WEBHOOK_AUTHORIZATION` setting.
Basic usage is covered in the
:ref:`webhooks configuration <webhooks-configuration>` docs.
If something posts to your webhooks without the required shared
secret as basic auth in the HTTP_AUTHORIZATION header, Anymail will
raise an :exc:`AnymailWebhookValidationFailure` error, which is
a subclass of Django's :exc:`~django.core.exceptions.SuspiciousOperation`.
This will result in an HTTP 400 response, without further processing
the data or calling your signal receiver function.
In addition to a single "random:random" string, you can give a list
of authorization strings. Anymail will permit webhook calls that match
any of the authorization strings:
.. code-block:: python
ANYMAIL = {
...
'WEBHOOK_AUTHORIZATION': [
'abcdefghijklmnop:qrstuvwxyz0123456789',
'ZYXWVUTSRQPONMLK:JIHGFEDCBA9876543210',
],
}
This facilitates credential rotation: first, append a new authorization
string to the list, and deploy your Django site. Then, update the webhook
URLs at your ESP to use the new authorization. Finally, remove the old
(now unused) authorization string from the list and re-deploy.
.. warning::
If your webhook URLs don't use https, this shared authorization
secret won't stay secret, defeating its purpose.
Signed webhooks
---------------
Some ESPs implement webhook signing, which is another method of verifying
the webhook data came from your ESP. Anymail will verify these signatures
for ESPs that support them. See the docs for your
:ref:`specific ESP <supported-esps>` for more details and configuration
that may be required.
Even with signed webhooks, it doesn't hurt to also use a shared secret.
Additional steps
----------------
Webhooks aren't unique to Anymail or to ESPs. They're used for many
different types of inter-site communication, and you can find additional
recommendations for improving webhook security on the web.
For example, you might consider:
* Tracking :attr:`~anymail.signals.AnymailTrackingEvent.event_id`,
to avoid accidental double-processing of the same events (or replay attacks)
* Checking the webhook's :attr:`~anymail.signals.AnymailTrackingEvent.timestamp`
is reasonably close the current time
* Configuring your firewall to reject webhook calls that come from
somewhere other than your ESP's documented IP addresses (if your ESP
provides this information)
But you should start with using SSL and a random shared secret via HTTP auth.