.. _sendgrid-backend: SendGrid ======== Anymail integrates with the `SendGrid`_ email service, using their `Web API v3`_. .. important:: **Troubleshooting:** If your SendGrid messages aren't being delivered as expected, be sure to look for "drop" events in your SendGrid `activity feed`_. SendGrid detects certain types of errors only *after* the send API call appears to succeed, and reports these errors as drop events. .. _SendGrid: https://sendgrid.com/ .. _Web API v3: https://sendgrid.com/docs/API_Reference/Web_API_v3/Mail/index.html .. _activity feed: https://app.sendgrid.com/email_activity?events=drops Settings -------- .. rubric:: EMAIL_BACKEND To use Anymail's SendGrid backend, set: .. code-block:: python EMAIL_BACKEND = "anymail.backends.sendgrid.EmailBackend" in your settings.py. .. setting:: ANYMAIL_SENDGRID_API_KEY .. rubric:: SENDGRID_API_KEY A SendGrid API key with "Mail Send" permission. (Manage API keys in your `SendGrid API key settings`_.) Required. .. code-block:: python ANYMAIL = { ... "SENDGRID_API_KEY": "", } Anymail will also look for ``SENDGRID_API_KEY`` at the root of the settings file if neither ``ANYMAIL["SENDGRID_API_KEY"]`` nor ``ANYMAIL_SENDGRID_API_KEY`` is set. .. _SendGrid API key settings: https://app.sendgrid.com/settings/api_keys .. setting:: ANYMAIL_SENDGRID_GENERATE_MESSAGE_ID .. rubric:: SENDGRID_GENERATE_MESSAGE_ID Whether Anymail should generate a UUID for each message sent through SendGrid, to facilitate status tracking. The UUID is attached to the message as a SendGrid custom arg named "anymail_id" and made available as :attr:`anymail_status.message_id ` on the sent message. Default ``True``. You can set to ``False`` to disable this behavior, in which case sent messages will have a `message_id` of ``None``. See :ref:`Message-ID quirks ` below. .. setting:: ANYMAIL_SENDGRID_MERGE_FIELD_FORMAT .. rubric:: SENDGRID_MERGE_FIELD_FORMAT If you use :ref:`merge data ` with SendGrid's legacy transactional templates, set this to a :meth:`str.format` formatting string that indicates how merge fields are delimited in your legacy templates. For example, if your templates use the ``-field-`` hyphen delimiters suggested in some SendGrid docs, you would set: .. code-block:: python ANYMAIL = { ... "SENDGRID_MERGE_FIELD_FORMAT": "-{}-", } The placeholder `{}` will become the merge field name. If you need to include a literal brace character, double it up. (For example, Handlebars-style ``{{field}}`` delimiters would take the format string `"{{{{{}}}}}"`.) The default `None` requires you include the delimiters directly in your :attr:`~anymail.message.AnymailMessage.merge_data` keys. You can also override this setting for individual messages. See the notes on SendGrid :ref:`templates and merge ` below. This setting is not used (or necessary) with SendGrid's newer dynamic transactional templates, which always use Handlebars syntax. .. setting:: ANYMAIL_SENDGRID_API_URL .. rubric:: SENDGRID_API_URL The base url for calling the SendGrid API. The default is ``SENDGRID_API_URL = "https://api.sendgrid.com/v3/"`` (It's unlikely you would need to change this.) .. _sendgrid-esp-extra: esp_extra support ----------------- To use SendGrid features not directly supported by Anymail, you can set a message's :attr:`~anymail.message.AnymailMessage.esp_extra` to a `dict` of parameters for SendGrid's `v3 Mail Send API`_. Your :attr:`esp_extra` dict will be deeply merged into the parameters Anymail has constructed for the send, with `esp_extra` having precedence in conflicts. Anymail has special handling for `esp_extra["personalizations"]`. If that value is a `dict`, Anymail will merge that personalizations dict into the personalizations for each message recipient. (If you pass a `list`, that will override the personalizations Anymail normally constructs from the message, and you will need to specify each recipient in the personalizations list yourself.) Example: .. code-block:: python message.open_tracking = True message.esp_extra = { "asm": { # SendGrid subscription management "group_id": 1, "groups_to_display": [1, 2, 3], }, "tracking_settings": { "open_tracking": { # Anymail will automatically set `"enable": True` here, # based on message.open_tracking. "substitution_tag": "%%OPEN_TRACKING_PIXEL%%", }, }, # Because "personalizations" is a dict, Anymail will merge "future_feature" # into the SendGrid personalizations array for each message recipient "personalizations": { "future_feature": {"future": "data"}, }, } (You can also set `"esp_extra"` in Anymail's :ref:`global send defaults ` to apply it to all messages.) .. _v3 Mail Send API: https://sendgrid.com/docs/API_Reference/Web_API_v3/Mail/index.html#-Request-Body-Parameters Limitations and quirks ---------------------- .. _sendgrid-message-id: **Message-ID** SendGrid does not return any sort of unique id from its send API call. Knowing a sent message's ID can be important for later queries about the message's status. To work around this, Anymail generates a UUID for each outgoing message, provides it to SendGrid as a custom arg named "anymail_id" and makes it available as the message's :attr:`anymail_status.message_id ` attribute after sending. The same UUID will be passed to Anymail's :ref:`tracking webhooks ` as :attr:`event.message_id `. To disable attaching tracking UUIDs to sent messages, set :setting:`SENDGRID_GENERATE_MESSAGE_ID ` to False in your Anymail settings. .. versionchanged:: 6.0 In batch sends, Anymail generates a distinct anymail_id for *each* "to" recipient. (Previously, a single id was used for all batch recipients.) Check :attr:`anymail_status.recipients[to_email].message_id ` for individual batch-send tracking ids. .. versionchanged:: 3.0 Previously, Anymail generated a custom :mailheader:`Message-ID` header for each sent message. But SendGrid's "smtp-id" event field does not reliably reflect this header, which complicates status tracking. (For compatibility with messages sent in earlier versions, Anymail's webhook :attr:`message_id` will fall back to "smtp-id" when "anymail_id" isn't present.) **Single Reply-To** SendGrid's v3 API only supports a single Reply-To address. If your message has multiple reply addresses, you'll get an :exc:`~anymail.exceptions.AnymailUnsupportedFeature` error---or if you've enabled :setting:`ANYMAIL_IGNORE_UNSUPPORTED_FEATURES`, Anymail will use only the first one. **Invalid Addresses** SendGrid will accept *and send* just about anything as a message's :attr:`from_email`. (And email protocols are actually OK with that.) (Tested March, 2016) **Wrong character set on text attachments** Under some conditions, SendGrid incorrectly identifies text attachments (text/plain, text/calendar, etc.) as using ISO-8859-1 encoding, and forces ``charset="iso-8859-1"`` into the attachments' MIME headers. This generally causes any non-ASCII characters in the attachments to be replaced with incorrect or illegal characters in the recipient's email client. The behavior is unpredictable, and may vary by SendGrid account or change over time. There is no reliable, general workaround that Anymail could implement. You may be able to counteract the issue by enabling open and/or click tracking in your SendGrid account. The only way to completely avoid the problem is switching to a non-text attachment type (e.g., application/pdf) or limiting your text attachments to use only ASCII characters. See `issue 150 `_ for more information and other possible workarounds. If this impacts your usage, it's helpful to report it to SendGrid support, so they can quantify customers affected and prioritize a fix. (Noted June, 2019 and December, 2019) **No envelope sender overrides** SendGrid does not support overriding :attr:`~anymail.message.AnymailMessage.envelope_sender` on individual messages. .. _sendgrid-templates: Batch sending/merge and ESP templates ------------------------------------- SendGrid offers both :ref:`ESP stored templates ` and :ref:`batch sending ` with per-recipient merge data. SendGrid has two types of stored templates for transactional email: * Dynamic transactional templates, which were introduced in July, 2018, use Handlebars template syntax and allow complex logic to be coded in the template itself. * Legacy transactional templates, which allow only simple key-value substitution and don't specify a particular template syntax. [Legacy templates were originally just called "transactional templates," and many older references still use this terminology. But confusingly, SendGrid's dashboard and some recent articles now use "transactional templates" to mean the newer, dynamic templates.] .. versionchanged:: 4.1 Added support for SendGrid dynamic transactional templates. (Earlier Anymail releases work only with SendGrid's legacy transactional templates.) You can use either type of SendGrid stored template by setting a message's :attr:`~anymail.message.AnymailMessage.template_id` to the template's unique id (*not* its name). Supply the merge data values with Anymail's normalized :attr:`~anymail.message.AnymailMessage.merge_data` and :attr:`~anymail.message.AnymailMessage.merge_global_data` message attributes. .. code-block:: python message = EmailMessage( ... # omit subject and body (or set to None) to use template content to=["alice@example.com", "Bob "] ) message.template_id = "d-5a963add2ec84305813ff860db277d7a" # SendGrid dynamic id message.merge_data = { 'alice@example.com': {'name': "Alice", 'order_no': "12345"}, 'bob@example.com': {'name': "Bob", 'order_no': "54321"}, } message.merge_global_data = { 'ship_date': "May 15", } When you supply per-recipient :attr:`~anymail.message.AnymailMessage.merge_data`, Anymail automatically changes how it communicates the "to" list to SendGrid, so that each recipient sees only their own email address. (Anymail creates a separate "personalization" for each recipient in the "to" list; any cc's or bcc's will be duplicated for *every* to-recipient.) See the `SendGrid's transactional template overview`_ for more information. .. _SendGrid's transactional template overview: https://sendgrid.com/docs/ui/sending-email/create-and-edit-transactional-templates/ .. _sendgrid-legacy-templates: Legacy transactional templates ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ With *legacy* transactional templates (only), SendGrid doesn't have a pre-defined merge field syntax, so you must tell Anymail how substitution fields are delimited in your templates. There are three ways you can do this: * Set `'merge_field_format'` in the message's :attr:`~anymail.message.AnymailMessage.esp_extra` to a python :meth:`str.format` string, as shown in the example below. (This applies only to that particular EmailMessage.) * *Or* set :setting:`SENDGRID_MERGE_FIELD_FORMAT ` in your Anymail settings. This is usually the best approach, and will apply to all legacy template messages sent through SendGrid. (You can still use esp_extra to override for individual messages.) * *Or* include the field delimiters directly in *all* your :attr:`~anymail.message.AnymailMessage.merge_data` and :attr:`~anymail.message.AnymailMessage.merge_global_data` keys. E.g.: ``{'-name-': "Alice", '-order_no-': "12345"}``. (This can be error-prone, and makes it difficult to transition to other ESPs or to SendGrid's dynamic templates.) .. code-block:: python # ... message.template_id = "5997fcf6-2b9f-484d-acd5-7e9a99f0dc1f" # SendGrid legacy id message.merge_data = { 'alice@example.com': {'name': "Alice", 'order_no': "12345"}, 'bob@example.com': {'name': "Bob", 'order_no': "54321"}, } message.esp_extra = { # Tell Anymail this SendGrid legacy template uses "-field-" for merge fields. # (You could instead set SENDGRID_MERGE_FIELD_FORMAT in your ANYMAIL settings.) 'merge_field_format': "-{}-" } SendGrid legacy templates allow you to mix your EmailMessage's `subject` and `body` with the template subject and body (by using `<%subject%>` and `<%body%>` in your SendGrid template definition where you want the message-specific versions to appear). If you don't want to supply any additional subject or body content from your Django app, set those EmailMessage attributes to empty strings or `None`. On-the-fly templates ~~~~~~~~~~~~~~~~~~~~ Rather than define a stored ESP template, you can refer to merge fields directly in an EmailMessage's subject and body, and SendGrid will treat this as an on-the-fly, legacy-style template definition. (The on-the-fly template can't contain any dynamic template logic, and like any legacy template you must specify the merge field format in either Anymail settings or esp_extra as described above.) .. code-block:: python # on-the-fly template using merge fields in subject and body: message = EmailMessage( subject="Your order {{order_no}} has shipped", body="Dear {{name}}:\nWe've shipped order {{order_no}}.", to=["alice@example.com", "Bob "] ) # note: no template_id specified message.merge_data = { 'alice@example.com': {'name': "Alice", 'order_no': "12345"}, 'bob@example.com': {'name': "Bob", 'order_no': "54321"}, } message.esp_extra = { # here's how to get Handlebars-style {{merge}} fields with Python's str.format: 'merge_field_format': "{{{{{}}}}}" # "{{ {{ {} }} }}" without the spaces } .. _sendgrid-webhooks: Status tracking webhooks ------------------------ If you are using Anymail's normalized :ref:`status tracking `, enter the url in your `SendGrid mail settings`_, under "Event Notification": :samp:`https://{random}:{random}@{yoursite.example.com}/anymail/sendgrid/tracking/` * *random:random* is an :setting:`ANYMAIL_WEBHOOK_SECRET` shared secret * *yoursite.example.com* is your Django site Be sure to check the boxes in the SendGrid settings for the event types you want to receive. SendGrid will report these Anymail :attr:`~anymail.signals.AnymailTrackingEvent.event_type`\s: queued, rejected, bounced, deferred, delivered, opened, clicked, complained, unsubscribed, subscribed. The event's :attr:`~anymail.signals.AnymailTrackingEvent.esp_event` field will be a `dict` of `Sendgrid event`_ fields, for a single event. (Although SendGrid calls webhooks with batches of events, Anymail will invoke your signal receiver separately for each event in the batch.) .. _SendGrid mail settings: https://app.sendgrid.com/settings/mail_settings .. _Sendgrid event: https://sendgrid.com/docs/API_Reference/Webhooks/event.html .. _sendgrid-inbound: Inbound webhook --------------- If you want to receive email from SendGrid through Anymail's normalized :ref:`inbound ` handling, follow SendGrid's `Inbound Parse Webhook`_ guide to set up Anymail's inbound webhook. The Destination URL setting will be: :samp:`https://{random}:{random}@{yoursite.example.com}/anymail/sendgrid/inbound/` * *random:random* is an :setting:`ANYMAIL_WEBHOOK_SECRET` shared secret * *yoursite.example.com* is your Django site Be sure the URL has a trailing slash. (SendGrid's inbound processing won't follow Django's :setting:`APPEND_SLASH` redirect.) If you want to use Anymail's normalized :attr:`~anymail.inbound.AnymailInboundMessage.spam_detected` and :attr:`~anymail.inbound.AnymailInboundMessage.spam_score` attributes, be sure to enable the "Check incoming emails for spam" checkbox. In most cases, you should enable SendGrid's "POST the raw, full MIME message" checkbox. Anymail should work either way (and you can change the option at any time), but enabling raw MIME will give the most accurate representation of *any* received email (including complex forms like multi-message mailing list digests). .. _Inbound Parse Webhook: https://sendgrid.com/docs/Classroom/Basics/Inbound_Parse_Webhook/setting_up_the_inbound_parse_webhook.html