Closed-loop interface

This topic describes the closed-loop interface that is used for bi-directional handling of actions and data exchange between Episerver Campaign and external systems.

Installation

Contact Episerver customer support regarding setup of the closed-loop interface, and terms and conditions that apply when accessing this type of non-anonymized action-based data.

Because the response data consists of non-anonymized action-based data, you have to indemnify and hold Episerver harmless in advance from any potential liabilities and third-party claims which may result from making this functionality available to you.

How it works

The external system (the customer's system integrated with Episerver Campaign) automatically triggers mailings in Episerver Campaign. Action-based mailing data is logged in Episerver Campaign and returned to the external system. This response data can be further processed within a third system, for example a data warehouse.

The closed-loop interface consists of these modules:

  • The import module automatically transfers recipient data like product recommendations from for example a data warehouse system, to Episerver Campaign, and automatically triggers mailings.
  • The response data export module transfers action-based mailing data such as openings, clicks, responses, unsubscribes, back to an external system.

The interface modules can be configured and used independently of each other.

Image: Data Exchange: External system I — External system II (data warehouse)

A service processes received data within Episerver Campaign. When a file is stored on the server, various jobs are being executed: Recipients are imported, mailings are created and sent, and a status notification is sent. The response data export module is responsible for the return channel. This module exports statistical data to the customer's system. The log data can be integrated into the data warehouse.

Image: Sequence diagram: External system I — External system II (data warehouse)

Exchanging data

The data exchange is done using an SFTP server set up at Episerver, and recipient data is transferred to that server. Response data is available for download from the same server.

Running transfers must have a temporary file name to avoid incomplete file exchange, see Data transfer troubleshooting. Naming conventions are described below.

The data exchange does not respect strict separation of clients. All user data can be transferred through one SFTP account. You can set up client-specific SFTP accounts if required for security reasons. Alternatively, you can set up separate sub-directories in one client. In both cases, the recipient data import module must then commit one file per client/directory.

Transferring recipient data

Recipient data can be transferred at any time to the Episerver Campaign SFTP server as previously described.

The name convention of the transferred files must be as follows:

YYYYMMDDHHMMSS_subscribers.csv

The recipient file is a CSVStands for "comma-separated values"; tabular data in a plain text file separated by the comma character. file with the following properties:

  • Phrase and field lengths are variable within the limits of the recipient list.
  • A semicolon (;) is used as separator.
  • Line feed is used for line breaks.
  • Quotation marks ("") are used as field boundaries. To designate quotation marks within a field, wrap them in quotation marks.
  • The file does not contain declarations regarding columns and field lengths.
  • Empty fields (NULL) remain empty in a data set. The number of field separators must be constant.

Recipient data is processed automatically every 10 minutes. The SFTP server of Episerver Campaign provides log files for download within 30 days after creation.

According to the specifications, fields of the main recipient list must be committed. Fields supposed to not contain any value are committed as empty columns. Existing fields and the field sequence are fixed. Do not change them without the consent of both parties.

Using personalization and recommendations

It is possible to enhance the delivered recipient data with personalization, for example by adding product recommendations to the mailing.

One or more fields reserved for a product ID are added to the CSV file with the recipient data. Product information (such as title, description, image-URL) for recommended products is transferred in a second CSV file. Product information is inserted into the mailing using field functions and a placeholder. Assigning product IDs to recipients is done by the customer.

With web analysis software, recommendations are available via a CSV file. Other recommended products are assigned to each product. Product information is rendered into the mailing using a special field function, similar to the process described above. You do not need to assign products to a recipient, as this is automatically done by the web analysis software in accordance with the initial configuration.

To add personalized content, you need a new template. Contact customer support.

Creating source mailings and sending mailings

As a basis for the campaign you are going to send, a source mailing has to be created in Smart Campaigns. A so called master list serves as recipient list for the source mailing. Placeholders referencing a specific field of the recipient list are inserted into the mailing paragraphs. Using these placeholders you can individually assign and insert product data and other information into the mailing.

To send multiple newsletters, or send newsletters at the same time, you have to create multiple source mailings. The source mailing is never sent and must remain in status New or respectively Activation required. Do not activate the mailing, otherwise the copy cannot be created. Instead the system automatically creates a copy of it and sends that, so the source mailing can be reused.

If you create a mailing using the REST API, you cannot use the mailing for the closed-loop interface. Create the source mailing only in Smart Campaigns.

Triggering the dispatch

When a file containing recipient data is stored in the corresponding directory on the SFTP server, the mailing is sent. Every 10 minutes, a Cron job checks this directory for new files. If a new file exists, it is imported into the recipient list of the corresponding client, and assigned to the source mailing using the parameter BROADMAIL_ID.

There are different ways to add content to a mailing:

  • Directly from the file. The content is imported along with the recipient data in a CSV file. This method is only applicable for texts and hyperlinks.
  • Referenced content. URLs in the imported CSV files are used to reference the content. When sending the mailing, the content is retrieved from the external server of the customer through the content interface. You can add texts, pictures, attachments, and hyperlinks types of content to a mailing.

When the data is imported, the mailing is automatically sent, and gets a unique ID.

You can send test emails using the closed-loop interface by storing test data instead of real data on the server. The CSV file must have the same structure as the file containing the real recipient data. Every time a test email is sent, a new mailing is created in Episerver Campaign.

Mailings are triggered only by storing a file on the server configured for the closed-loop interface. You cannot send mailings by using the Episerver Campaign user interface.

You can customize the source mailing for different newsletters. Only change the source mailing after the previous mailing is sent. Additionally, ensure that the correct mailing ID of the source mailing is used in the recipient CSV file.

Troubleshooting data transfer

If a problem occurs transferring the data, a notification is sent to an email address configured beforehand. An error file is generated and the transfer stopped. During implementation you can configure the exact interface behavior in case of an error. When the implementation is completed, you can switch off this setting.

If an error occurs, the interface must be restarted in accordance with the error file. These are the cases:

  • Data import failed.
  • Error when mailing is started after data import.

Errors during data transfer are handled by customer support, since these can only be resolved manually.

Monitoring and sending status

After a recipient data import, the status of the process is sent via email and registered in a log file. This assures that actions within the interface are properly logged and documented in Episerver Campaign.

Structure of the main recipient list

Column name Data type Example Remarks
Email Varchar(255) [email protected] Email address of the user.
broadmail_ID Bigint/Long 44018617811 Mailing ID in Episerver Campaign (broadmail_ID).
WAVE_ID Bigint/Long 46623317811 Identifies the selection/sending wave for unambiguous assignment of user actions. This parameter is optionally set by the external system when importing the recipient data.
Salutation Varchar(255) Field from existing recipient list.
First name Varchar(255) Field from existing recipient list.
Last name Varchar(255) Field from existing recipient list.

The list configuration described above is an example. More fields can be added if required for personalization purposes. Contact customer support beforehand, and document them in the interface setup.

Specifying response data

Response data is generated daily from the log files, and is available for download at a configured time via an SFTP server within Episerver Campaign. The files contain the most recent data. The files containing the mailing data are available for download for 30 days after starting the mailing.

The response data consists of 7 CSV files. Each of these can be read as a table as described in the following entity relationship (ER) diagram. See detailed type information for each file below.

File names can be configured. The names used in the following sections are examples.

Image: Simplified ER diagram of the response data (PK = primary key, FK = foreign key)

Sent mailings

YYYYMMDD_mailings.csv contains mailings sent in the referenced period.

Column name Data type Example Remarks
id Bigint/Long 51106527229 Mailing ID in Episerver Campaign
mailingGroupId Bigint/Long 44018617811 Client ID in Episerver Campaign (broadmail_ID)
started DateTime 2007-10-24 12:07:55 Start time of the mailing
finished DateTime 2007-10-24 12:09:55 End time of the mailing

Sending Log

YYYYMMDD_mailingrecipients.csv contains recipient-related data. The sending log refers to the mailings contained in the file YYYYMMDD_mailings.csv.

Column name Data type Example Remarks
mailingId Bigint/Long 51106527229 Mailing ID in Episerver Campaign
userId Varchar(255) [email protected] Email address of the recipient
created DateTime 2007-10-24 12:07:55 Sending date
response.category Varchar(20)
  • unknown – The recipient sent a response to the mailing.
  • bounce – Soft bounce
  • temporary_bounce – Soft bounce
  • fatal_bounce – Hard bounce
  • autoresponder – Autoresponder
 Delivery status

YYYYMMDD_links.csv contains links from the mailing. Data is only available for links with activated link tracking.

Column name Data type Example Remarks
mailingId Bigint/Long 51106527229 Mailing ID in Episerver Campaign
id Bigint/Long 44018617936 Link ID in Episerver Campaign
Name Varchar(255) Imprint Name of the link
url Varchar(255) http://www.example.com Link target (URL)

Clicks

YYYYMMDD_clicks.csv contains mailing clicks generated by recipients, within the referenced period. Clicked links/URLs can be found in the file YYYYMMDD_links.csv.

Column name Data type Example Remarks
mailingId Bigint/Long 51106527229 Mailing ID in Episerver Campaign
userId Varchar(255) [email protected] Email address of the recipient
linkId Bigint/Long 44018617936 Link ID in Episerver Campaign
created DateTime 2007-10-24 12:07:55 Date and time of the click.

There can be multiple clicks from one recipient within one second.

Opens

YYYYMMDD_opens.csv contains mailings opened by recipients in the referenced period.

Column name Data type Example Remarks
mailingId Bigint / Long 51106527229 Mailing ID in Episerver Campaign
userId Varchar(255) [email protected] Email address of the recipient
created DateTime 2007-10-24 12:07:55 Time and date when the mailing was opened by a recipient.

One recipient can open a mailing several times. In this case, each opening generates a singular data set.

Unsubscribes

YYYYMMDD_unsubscribes.csv contains unsubscribes by recipients in the referenced period.

Column name Data type Example Remarks
mailingId Bigint/Long 51106527229 Mailing ID in Episerver Campaign
userId Varchar(255) [email protected] Email address of the recipient
created DateTime 2007-10-24 12:07:55 Date and time of the unsubscribe

Responses

YYYYMMDD_responses.csv contains emails returned to the sender address from any user within the referenced period.

Column name Data type Example Remarks
mailingId Bigint/Long 51106527229 Mailing ID in Episerver Campaign.
mailingToUserId Varchar(255) 25951836752 Encoded assignment of the recipient.
userId Varchar(255) [email protected] Email address of the recipient.
category Varchar(20)
  • unknown – The recipient sent a response to the mailing.
  • bounce – Soft bounce.
  • temporary_bounce – Soft bounce.
  • fatal_bounce – Hard bounce.
  • autoresponder – Autoresponder.
 Delivery status.
created DateTime 2007-10-24 12:07:55 Date and time of the response.