Mailgun: fix event/metadata param extraction in tracking webhook

Mailgun merges user-variables (metadata) into the webhook post data
interspersed with the actual event params. This can lead to ambiguity
interpreting post data.

To extract metadata from an event, Anymail had been attempting to avoid
that ambiguity by instead using X-Mailgun-Variables fields found in the
event's message-headers param. But message-headers isn't included in
some tracking events (opened, clicked, unsubscribed), resulting in
empty metadata for those events. (#76)

Also, conflicting metadata keys could confuse Anymail's Mailgun event
parsing, leading to unexpected values in the normalized event. (#77)

This commit:
* Cleans up Anymail's tracking webhook to be explicit about which
  multi-value params it uses, avoiding conflicts with metadata keys.
  Fixes #77.
* Extracts metadata from post params for opened, clicked and
  unsubscribed events. All unknown event params are assumed to be
  metadata. Fixes #76.
* Documents a few metadata key names where it's impossible (or likely
  to be unreliable) for Anymail to extract metadata from the post data.

For reference, the order of params in the Mailgun's post data *appears*
to be (from live testing):
* For the timestamp, token and signature params, any user-variable with
  the same name appears *before* the corresponding event data.
* For all other params, any user-variable with the same name as a
  Mailgun event param appears *after* the Mailgun data.
This commit is contained in:
medmunds
2017-10-27 13:26:37 -07:00
parent 636c8a5d80
commit bb68f3dd6d
5 changed files with 207 additions and 33 deletions

View File

@@ -4,6 +4,7 @@ import base64
from unittest import skipIf
import six
from django.http import QueryDict
from django.test import SimpleTestCase, RequestFactory, override_settings
from django.utils.translation import ugettext_lazy
@@ -22,7 +23,7 @@ from anymail.utils import (
parse_address_list, EmailAddress,
is_lazy, force_non_lazy, force_non_lazy_dict, force_non_lazy_list,
update_deep,
get_request_uri, get_request_basic_auth, parse_rfc2822date)
get_request_uri, get_request_basic_auth, parse_rfc2822date, querydict_getfirst)
class ParseAddressListTests(SimpleTestCase):
@@ -299,6 +300,21 @@ class RequestUtilsTests(SimpleTestCase):
"https://user:pass@secret.example.com:8989/path/to/?query")
class QueryDictUtilsTests(SimpleTestCase):
def test_querydict_getfirst(self):
q = QueryDict("a=one&a=two&a=three")
q.getfirst = querydict_getfirst.__get__(q)
self.assertEqual(q.getfirst('a'), "one")
# missing key exception:
with self.assertRaisesMessage(KeyError, "not a key"):
q.getfirst("not a key")
# defaults:
self.assertEqual(q.getfirst('not a key', "beta"), "beta")
self.assertIsNone(q.getfirst('not a key', None))
class ParseRFC2822DateTests(SimpleTestCase):
def test_with_timezones(self):
dt = parse_rfc2822date("Tue, 24 Oct 2017 10:11:35 -0700")