Feature: Implement merge_headers

Implement and document `merge_headers`
for all other ESPs that can support it. (See #371
for base and Amazon SES implementation.)

Closes #374

docs/esps/amazon_ses.rst

@@ -96,6 +96,14 @@ Limitations and quirks
 **No delayed sending**
   Amazon SES does not support :attr:`~anymail.message.AnymailMessage.send_at`.
 
+**Merge features require template_id**
+  Anymail's :attr:`~anymail.message.AnymailMessage.merge_headers`,
+  :attr:`~anymail.message.AnymailMessage.merge_metadata`,
+  :attr:`~anymail.message.AnymailMessage.merge_data`, and
+  :attr:`~anymail.message.AnymailMessage.merge_global_data` are only supported
+  when sending :ref:`templated messages <amazon-ses-templates>`
+  (using Anymail's :attr:`~anymail.message.AnymailMessage.template_id`).
+
 **No global send defaults for non-Anymail options**
   With the Amazon SES backend, Anymail's :ref:`global send defaults <send-defaults>`
   are only supported for Anymail's added message options (like

docs/esps/esp-feature-matrix.csv

@@ -1,8 +1,9 @@
 Email Service Provider,:ref:`amazon-ses-backend`,:ref:`brevo-backend`,:ref:`mailersend-backend`,:ref:`mailgun-backend`,:ref:`mailjet-backend`,:ref:`mandrill-backend`,:ref:`postal-backend`,:ref:`postmark-backend`,:ref:`resend-backend`,:ref:`sendgrid-backend`,:ref:`sparkpost-backend`,:ref:`unisender-go-backend`
 .. rubric:: :ref:`Anymail send options <anymail-send-options>`,,,,,,,,,,,,
 :attr:`~AnymailMessage.envelope_sender`,Yes,No,No,Domain only,Yes,Domain only,Yes,No,No,No,Yes,No
+:attr:`~AnymailMessage.merge_headers`,Yes*,Yes,No,Yes,Yes,No,No,Yes,Yes,Yes,Yes*,Yes*
 :attr:`~AnymailMessage.metadata`,Yes,Yes,No,Yes,Yes,Yes,No,Yes,Yes,Yes,Yes,Yes
-:attr:`~AnymailMessage.merge_metadata`,No,Yes,No,Yes,Yes,Yes,No,Yes,Yes,Yes,Yes,Yes
+:attr:`~AnymailMessage.merge_metadata`,Yes*,Yes,No,Yes,Yes,Yes,No,Yes,Yes,Yes,Yes,Yes
 :attr:`~AnymailMessage.send_at`,No,Yes,Yes,Yes,No,Yes,No,No,No,Yes,Yes,Yes
 :attr:`~AnymailMessage.tags`,Yes,Yes,Yes,Yes,Max 1 tag,Yes,Max 1 tag,Max 1 tag,Yes,Yes,Max 1 tag,Yes
 :attr:`~AnymailMessage.track_clicks`,No,No,Yes,Yes,Yes,Yes,No,Yes,No,Yes,Yes,Yes
@@ -10,8 +11,8 @@ Email Service Provider,:ref:`amazon-ses-backend`,:ref:`brevo-backend`,:ref:`mail
 :ref:`amp-email`,Yes,No,No,Yes,No,No,No,No,No,Yes,Yes,Yes
 .. rubric:: :ref:`templates-and-merge`,,,,,,,,,,,,
 :attr:`~AnymailMessage.template_id`,Yes,Yes,Yes,Yes,Yes,Yes,No,Yes,No,Yes,Yes,Yes
-:attr:`~AnymailMessage.merge_data`,Yes,Yes,Yes,Yes,Yes,Yes,No,Yes,No,Yes,Yes,Yes
-:attr:`~AnymailMessage.merge_global_data`,Yes,Yes,(emulated),(emulated),Yes,Yes,No,Yes,No,Yes,Yes,Yes
+:attr:`~AnymailMessage.merge_data`,Yes*,Yes,Yes,Yes,Yes,Yes,No,Yes,No,Yes,Yes,Yes
+:attr:`~AnymailMessage.merge_global_data`,Yes*,Yes,(emulated),(emulated),Yes,Yes,No,Yes,No,Yes,Yes,Yes
 .. rubric:: :ref:`Status <esp-send-status>` and :ref:`event tracking <event-tracking>`,,,,,,,,,,,,
 :attr:`~AnymailMessage.anymail_status`,Yes,Yes,Yes,Yes,Yes,Yes,Yes,Yes,Yes,Yes,Yes,Yes
 :class:`~anymail.signals.AnymailTrackingEvent` from webhooks,Yes,Yes,Yes,Yes,Yes,Yes,Yes,Yes,Yes,Yes,Yes,Yes

docs/esps/index.rst

@@ -48,6 +48,8 @@ The table below summarizes the Anymail features supported for each ESP.
     :widths: auto
     :class: sticky-left
 
+\* See ESP detail page for limitations and clarifications
+
 Trying to choose an ESP? Please **don't** start with this table. It's far more
 important to consider things like an ESP's deliverability stats, latency, uptime,
 and support for developers. The *number* of extra features an ESP offers is almost

docs/esps/mailersend.rst

@@ -219,6 +219,10 @@ see :ref:`unsupported-features`.
   Any other extra headers will raise an
   :exc:`~anymail.exceptions.AnymailUnsupportedFeature` error.
 
+**No merge headers support**
+  MailerSend's API does not provide a way to support Anymail's
+  :attr:`~anymail.message.AnymailMessage.merge_headers`.
+
 **No metadata support**
   MailerSend does not support Anymail's
   :attr:`~anymail.message.AnymailMessage.metadata` or

docs/esps/mailgun.rst

@@ -247,18 +247,23 @@ Limitations and quirks
   obvious reasons, only the domain portion applies. You can use anything before
   the @, and it will be ignored.
 
-**Using merge_metadata with merge_data**
+**Using merge_metadata and merge_headers with merge_data**
   If you use both Anymail's :attr:`~anymail.message.AnymailMessage.merge_data`
   and :attr:`~anymail.message.AnymailMessage.merge_metadata` features, make sure your
-  merge_data keys do not start with ``v:``. (It's a good idea anyway to avoid colons
-  and other special characters in merge_data keys, as this isn't generally portable
-  to other ESPs.)
+  :attr:`~!anymail.message.AnymailMessage.merge_data` keys do not start with ``v:``.
+
+  Similarly, if you use Anymail's :attr:`~anymail.message.AnymailMessage.merge_headers`
+  together with :attr:`~anymail.message.AnymailMessage.merge_data`, make sure your
+  :attr:`~!anymail.message.AnymailMessage.merge_data` keys do not start with ``h:``.
+
+  (It's a good idea anyway to avoid colons and other special characters in merge data
+  keys, as this isn't generally portable to other ESPs.)
 
   The same underlying Mailgun feature ("recipient-variables") is used to implement
-  both Anymail features. To avoid conflicts, Anymail prepends ``v:`` to recipient
-  variables needed for merge_metadata. (This prefix is stripped as Mailgun prepares
-  the message to send, so it won't be present in your Mailgun API logs or the metadata
-  that is sent to tracking webhooks.)
+  all three Anymail features. To avoid conflicts, Anymail prepends ``v:`` to recipient
+  variables needed for merge metadata, and ``h:`` for merge headers recipient variables.
+  (These prefixes are stripped as Mailgun prepares the message to send, so won't appear
+  in your Mailgun API logs or the metadata that is sent to tracking webhooks.)
 
 **Additional limitations on merge_data with template_id**
   If you are using Mailgun's stored handlebars templates (Anymail's

docs/esps/mandrill.rst

@@ -143,6 +143,10 @@ Limitations and quirks
   (Verified and reported to MailChimp support 4/2022;
   see `Anymail discussion #257`_ for more details.)
 
+**No merge headers support**
+  Mandrill's API does not provide a way to support Anymail's
+  :attr:`~anymail.message.AnymailMessage.merge_headers`.
+
 **Envelope sender uses only domain**
   Anymail's :attr:`~anymail.message.AnymailMessage.envelope_sender` is used to
   populate Mandrill's `'return_path_domain'`---but only the domain portion.

docs/esps/postal.rst

@@ -119,6 +119,13 @@ see :ref:`unsupported-features`.
 **Attachments must be named**
   Postal issues an `AttachmentMissingName` error when trying to send an attachment without name.
 
+**No merge features**
+  Because Postal does not support batch sending, Anymail's
+  :attr:`~anymail.message.AnymailMessage.merge_headers`,
+  :attr:`~anymail.message.AnymailMessage.merge_metadata`,
+  and :attr:`~anymail.message.AnymailMessage.merge_data`
+  are not supported.
+
 
 .. _postal-templates:
 

docs/esps/sparkpost.rst

@@ -206,6 +206,15 @@ Limitations and quirks
 
   .. versionadded:: 8.0
 
+**Extra header limitations**
+  SparkPost's API silently ignores certain email headers (specified via
+  Django's :ref:`headers or extra_headers <message-headers>` or Anymail's
+  :attr:`~anymail.message.AnymailMessage.merge_headers`). In particular,
+  attempts to provide a custom :mailheader:`List-Unsubscribe` header will
+  not work; the message will be sent with SparkPost's own subscription
+  management headers. (The list of allowed custom headers does not seem
+  to be documented.)
+
 **Envelope sender may use domain only**
   Anymail's :attr:`~anymail.message.AnymailMessage.envelope_sender` is used to
   populate SparkPost's `'return_path'` parameter. Anymail supplies the full

docs/esps/unisender_go.rst

@@ -212,6 +212,13 @@ Limitations and quirks
   :attr:`~anymail.message.AnymailMessage.merge_data` or
   :attr:`~anymail.message.AnymailMessage.merge_global_data`.
 
+**Limited merge headers support**
+  Unisender Go supports per-recipient :mailheader:`List-Unsubscribe` headers
+  (if your account has been approved to disable their unsubscribe link),
+  but trying to include any other field in Anymail's
+  :attr:`~anymail.message.AnymailMessage.merge_headers` will raise
+  an :exc:`~anymail.exceptions.AnymailUnsupportedFeature` error.
+
 **No envelope sender overrides**
   Unisender Go does not support overriding a message's
   :attr:`~anymail.message.AnymailMessage.envelope_sender`.

docs/sending/anymail_additions.rst

@@ -101,6 +101,49 @@ an :ref:`unsupported feature <unsupported-features>` error.
         .. _how envelope sender relates to return path:
             https://www.postmastery.com/blog/about-the-return-path-header/
 
+    .. attribute:: merge_headers
+
+        .. versionadded:: 11.0
+
+        On a message with multiple recipients, if your ESP supports it,
+        you can set this to a `dict` of *per-recipient* extra email headers.
+        Each key in the dict is a recipient email (address portion only),
+        and its value is a dict of header fields and values for that recipient:
+
+        .. code-block:: python
+
+            message.to = ["wile@example.com", "R. Runner <rr@example.com>"]
+            message.extra_headers = {
+                # Headers for all recipients
+                "List-Unsubscribe-Post": "List-Unsubscribe=One-Click",
+            }
+            message.merge_headers = {
+                # Per-recipient headers
+                "wile@example.com": {
+                    "List-Unsubscribe": "<https://example.com/unsubscribe/12345>",
+                },
+                "rr@example.com": {
+                    "List-Unsubscribe": "<https://example.com/unsubscribe/98765>",
+                },
+            }
+
+        When :attr:`!merge_headers` is set, Anymail will use the ESP's
+        :ref:`batch sending <batch-send>` option, so that each :attr:`to` recipient gets
+        an individual message (and doesn't see the other emails on the :attr:`to` list).
+
+        Many ESPs restrict which headers are allowed. Be sure to check Anymail's
+        :ref:`ESP-specific docs <supported-esps>` for your ESP.
+        (Also, :ref:`special handling <message-headers>` for :mailheader:`From`,
+        :mailheader:`To` and :mailheader:`Reply-To` headers does *not* apply
+        to :attr:`!merge_headers`.)
+
+        If :attr:`!merge_headers` defines a particular header for only some
+        recipients, the default for other recipients comes from the message's
+        :ref:`extra_headers <message-headers>`. If not defined there, behavior
+        varies by ESP: some will include the header field only for recipients
+        where you have provided it; other ESPs will send an empty header field
+        to the other recipients.
+
     .. attribute:: metadata
 
         If your ESP supports tracking arbitrary metadata, you can set this to