Brevo: support proxy open, complained, error events

Add support for Brevo's new "Complained," "Error" and 
"Loaded by proxy" events in Brevo tracking webhook.

Closes #385.

---------

Co-authored-by: Mike Edmunds <medmunds@gmail.com>

CHANGELOG.rst

@@ -26,6 +26,18 @@ Release history
     ..  This extra heading level keeps the ToC from becoming unmanageably long
 
 
+vNext
+-----
+
+*unreleased changes*
+
+Features
+~~~~~~~~
+
+* **Brevo:** Support Brevo's new "Complaint," "Error" and "Loaded by proxy"
+  tracking events. (Thanks to `@originell`_ for the update.)
+
+
 v11.0.1
 -------
 
@@ -1695,6 +1707,7 @@ Features
 .. _@mounirmesselmeni: https://github.com/mounirmesselmeni
 .. _@mwheels: https://github.com/mwheels
 .. _@nuschk: https://github.com/nuschk
+.. _@originell: https://github.com/originell
 .. _@puru02: https://github.com/puru02
 .. _@RignonNoel: https://github.com/RignonNoel
 .. _@sblondon: https://github.com/sblondon

anymail/webhooks/brevo.py

@@ -40,23 +40,34 @@ class BrevoTrackingWebhookView(BrevoBaseWebhookView):
             )
         return [self.esp_to_anymail_event(esp_event)]
 
+    # Map Brevo event type -> Anymail normalized (event type, reject reason).
     event_types = {
-        # Map Brevo event type: Anymail normalized (event type, reject reason)
+        # Treat "request" as QUEUED rather than SENT, because it may be received
-        # received even if message won't be sent (e.g., before "blocked"):
+        # even if message won't actually be sent (e.g., before "blocked").
         "request": (EventType.QUEUED, None),
         "delivered": (EventType.DELIVERED, None),
         "hard_bounce": (EventType.BOUNCED, RejectReason.BOUNCED),
         "soft_bounce": (EventType.BOUNCED, RejectReason.BOUNCED),
         "blocked": (EventType.REJECTED, RejectReason.BLOCKED),
         "spam": (EventType.COMPLAINED, RejectReason.SPAM),
+        "complaint": (EventType.COMPLAINED, RejectReason.SPAM),
         "invalid_email": (EventType.BOUNCED, RejectReason.INVALID),
         "deferred": (EventType.DEFERRED, None),
-        "opened": (EventType.OPENED, None),  # see also unique_opened below
+        # Brevo has four types of opened events:
+        #   - "unique_opened": first time opened
+        #   - "opened": subsequent opens
+        #   - "unique_proxy_opened": first time opened via proxy (e.g., Apple Mail)
+        #   - "proxy_open": subsequent opens via proxy
+        # Treat all of these as OPENED.
+        "unique_opened": (EventType.OPENED, None),
+        "opened": (EventType.OPENED, None),
+        "unique_proxy_open": (EventType.OPENED, None),
+        "proxy_open": (EventType.OPENED, None),
         "click": (EventType.CLICKED, None),
         "unsubscribe": (EventType.UNSUBSCRIBED, None),
-        # shouldn't occur for transactional messages:
+        "error": (EventType.FAILED, None),
+        # ("list_addition" shouldn't occur for transactional messages.)
         "list_addition": (EventType.SUBSCRIBED, None),
-        "unique_opened": (EventType.OPENED, None),  # first open; see also opened above
     }
 
     def esp_to_anymail_event(self, esp_event):

docs/esps/brevo.rst

@@ -332,20 +332,36 @@ Be sure to select the checkboxes for all the event types you want to receive. (A
 sure you are in the "Transactional" section of their site; Brevo has a separate set
 of "Campaign" webhooks, which don't apply to messages sent through Anymail.)
 
-If you are interested in tracking opens, note that Brevo has both "First opening"
+If you are interested in tracking opens, note that Brevo has four different
-and an "Known open" event types. The latter seems to be generated only for the second
+open event types:
-and subsequent opens. Anymail normalizes both types to "opened." To track unique opens
+
-enable only "First opening," or to track all message opens enable both. (Brevo used to
+* "First opening": the first time a message is opened by a particular recipient.
-deliver both events for the first open, so be sure to check their current behavior
+  (Brevo event type "opened")
-if duplicate first open events might cause problems for you. You might be able to use
+* "Known open": the second and subsequent opens. (Brevo event type "unique_opened")
-the event timestamp to de-dupe.)
+* "Loaded by proxy": a message's tracking pixel is loaded by a proxy service
+  intended to protect users' IP addresses. See Brevo's article on
+  `Apple's Mail Privacy Protection`_ for more details. As of July, 2024, Brevo
+  seems to deliver this event only for the second and subsequent loads by the
+  proxy service. (Brevo event type "proxy_open")
+* "First open but loaded by proxy": the first time a message's tracking pixel
+  is loaded by a proxy service for a particular recipient. As of July, 2024,
+  this event has not yet been exposed in Brevo's webhook control panel, and
+  you must contact Brevo support to enable it. (Brevo event type "unique_proxy_opened")
+
+Anymail normalizes all of these to "opened." If you need to distinguish the
+specific Brevo event types, examine the raw
+:attr:`~anymail.signals.AnymailTrackingEvent.esp_event`, e.g.:
+``if event.esp_event["event"] == "unique_opened": …``.
 
 Brevo will report these Anymail :attr:`~anymail.signals.AnymailTrackingEvent.event_type`\s:
 queued, rejected, bounced, deferred, delivered, opened (see note above), clicked, complained,
-unsubscribed, subscribed (though this should never occur for transactional email).
+failed, unsubscribed, subscribed (though subscribed should never occur for transactional email).
 
 For events that occur in rapid succession, Brevo frequently delivers them out of order.
 For example, it's not uncommon to receive a "delivered" event before the corresponding "queued."
+Also, note that "queued" may be received even if Brevo will not actually send the message.
+(E.g., if a recipient is on your blocked list due to a previous bounce, you may receive
+"queued" followed by "rejected.")
 
 The event's :attr:`~anymail.signals.AnymailTrackingEvent.esp_event` field will be
 a `dict` of raw webhook data received from Brevo.
@@ -356,8 +372,14 @@ a `dict` of raw webhook data received from Brevo.
     than "brevo". The old URL will still work, but is deprecated. See :ref:`brevo-rename`
     below.
 
+.. versionchanged:: 11.1
+
+    Added support for Brevo's "Complaint," "Error" and "Loaded by proxy" events.
+
 
 .. _Transactional > Email > Settings > Webhook: https://app-smtp.brevo.com/webhook
+.. _Apple's Mail Privacy Protection:
+    https://help.brevo.com/hc/en-us/articles/4406537065618-How-to-handle-changes-in-Apple-s-Mail-Privacy-Protection
 
 
 .. _brevo-inbound:

tests/test_brevo_webhooks.py

@@ -226,6 +226,36 @@ class BrevoDeliveryTestCase(WebhookTestCase):
         )
         event = kwargs["event"]
         self.assertEqual(event.event_type, "complained")
+        self.assertEqual(event.reject_reason, "spam")
+
+    def test_complaint(self):
+        # Sadly, this is not well documented in the official Brevo API documentation.
+        raw_event = {
+            "event": "complaint",
+            "email": "example@domain.com",
+            "id": "xxxxx",
+            "date": "2020-10-09 00:00:00",
+            "ts": 1604933619,
+            "message-id": "201798300811.5787683@relay.domain.com",
+            "ts_event": 1604933654,
+            "X-Mailin-custom": '{"meta": "data"}',
+            "tags": ["transac_messages"],
+        }
+        response = self.client.post(
+            "/anymail/brevo/tracking/",
+            content_type="application/json",
+            data=json.dumps(raw_event),
+        )
+        self.assertEqual(response.status_code, 200)
+        kwargs = self.assert_handler_called_once_with(
+            self.tracking_handler,
+            sender=BrevoTrackingWebhookView,
+            event=ANY,
+            esp_name="Brevo",
+        )
+        event = kwargs["event"]
+        self.assertEqual(event.event_type, "complained")
+        self.assertEqual(event.reject_reason, "spam")
 
     def test_invalid_email(self):
         # "If a ISP again indicated us that the email is not valid or if we discovered
@@ -258,6 +288,38 @@ class BrevoDeliveryTestCase(WebhookTestCase):
             event.mta_response, "(guessing invalid_email includes a reason)"
         )
 
+    def test_error_email(self):
+        # Sadly, this is not well documented in the official Brevo API documentation.
+        raw_event = {
+            "event": "error",
+            "email": "example@domain.com",
+            "id": "xxxxx",
+            "date": "2020-10-09 00:00:00",
+            "ts": 1604933619,
+            "message-id": "201798300811.5787683@relay.domain.com",
+            "ts_event": 1604933654,
+            "subject": "My first Transactional",
+            "X-Mailin-custom": '{"meta": "data"}',
+            "template_id": 22,
+            "tags": ["transac_messages"],
+            "ts_epoch": 1604933623,
+        }
+        response = self.client.post(
+            "/anymail/brevo/tracking/",
+            content_type="application/json",
+            data=json.dumps(raw_event),
+        )
+        self.assertEqual(response.status_code, 200)
+        kwargs = self.assert_handler_called_once_with(
+            self.tracking_handler,
+            sender=BrevoTrackingWebhookView,
+            event=ANY,
+            esp_name="Brevo",
+        )
+        event = kwargs["event"]
+        self.assertEqual(event.event_type, "failed")
+        self.assertEqual(event.reject_reason, None)
+
     def test_deferred_event(self):
         # Note: the example below is an actual event capture (with 'example.com'
         # substituted for the real receiving domain). It's pretty clearly a bounce, not
@@ -341,6 +403,71 @@ class BrevoDeliveryTestCase(WebhookTestCase):
         event = kwargs["event"]
         self.assertEqual(event.event_type, "opened")
 
+    def test_proxy_open_event(self):
+        # Equivalent to "Loaded via Proxy" in the Brevo UI.
+        # This is sent when a tracking pixel is loaded via a 'privacy proxy server'.
+        # This technique is used by Apple Mail, for example, to protect user's IP
+        # addresses.
+        raw_event = {
+            "event": "proxy_open",
+            "email": "example@domain.com",
+            "id": 1,
+            "date": "2020-10-09 00:00:00",
+            "message-id": "201798300811.5787683@relay.domain.com",
+            "subject": "My first Transactional",
+            "tag": ["transactionalTag"],
+            "sending_ip": "xxx.xxx.xxx.xxx",
+            "s_epoch": 1534486682000,
+            "template_id": 1,
+        }
+        response = self.client.post(
+            "/anymail/brevo/tracking/",
+            content_type="application/json",
+            data=json.dumps(raw_event),
+        )
+        self.assertEqual(response.status_code, 200)
+        kwargs = self.assert_handler_called_once_with(
+            self.tracking_handler,
+            sender=BrevoTrackingWebhookView,
+            event=ANY,
+            esp_name="Brevo",
+        )
+        event = kwargs["event"]
+        self.assertEqual(event.event_type, "opened")
+
+    def test_unique_proxy_open_event(self):
+        # Sadly, undocumented in Brevo.
+        # Equivalent to "First Open but loaded via Proxy".
+        # This is sent when a tracking pixel is loaded via a 'privacy proxy server'.
+        # This technique is used by Apple Mail, for example, to protect user's IP
+        # addresses.
+        raw_event = {
+            "event": "unique_proxy_open",
+            "email": "example@domain.com",
+            "id": 1,
+            "date": "2020-10-09 00:00:00",
+            "message-id": "201798300811.5787683@relay.domain.com",
+            "subject": "My first Transactional",
+            "tag": ["transactionalTag"],
+            "sending_ip": "xxx.xxx.xxx.xxx",
+            "s_epoch": 1534486682000,
+            "template_id": 1,
+        }
+        response = self.client.post(
+            "/anymail/brevo/tracking/",
+            content_type="application/json",
+            data=json.dumps(raw_event),
+        )
+        self.assertEqual(response.status_code, 200)
+        kwargs = self.assert_handler_called_once_with(
+            self.tracking_handler,
+            sender=BrevoTrackingWebhookView,
+            event=ANY,
+            esp_name="Brevo",
+        )
+        event = kwargs["event"]
+        self.assertEqual(event.event_type, "opened")
+
     def test_clicked_event(self):
         raw_event = {
             "event": "click",