Skip navigation

Before diving into the flowchart for exporting all Eloqua activities using the the Bulk API, let's cover some best practices using the Bulk API focused on exporting activities:

  • Use filters to ensure fewer than 5,000,000 records are exported per Bulk API sync
    • A common method of filtering is to use the Activity Date to export a finite time period of activity, such as one month (use larger or smaller time periods based on volume of activity generation)
  • Export one Activity Type at a time
    • Do not run syncs for all Activity Types simultaneously
  • Make multiple export requests sequentially
    • Allow the sync to finish before executing the next sync

 

Here are the main resources related to exporting activities using the Bulk API on the Developer Help Center:

 

Exporting All Eloqua Activities Using the Bulk API Flowchart

(click to enlarge)

Exporting All Activities Using the Bulk API Flow

 

Notes and Resources by Flowchart Step

Determine Bulk API Base URL and Select Authentication

Select Activity Type to Export

Create an Activity Export Definition for the selected Activity Type

Create the Sync with the Activity Export Definition URI

Retrieve Sync Logs with the Sync URI's Logs Endpoint

    • Syncs Logs Endpoint
    • If exporting to CSV:
      • The total number of records returned can only be retrieved using the Syncs Logs Endpoint
    • If exporting to JSON:
      • When retrieving data with the Sync URI's Data Endpoint in JSON, the following parameters are returned along with the data, either of which could be used to determine how many calls are necessary to retrieve the remaining data:
        • totalResults
        • hasMore
    • When retrieving the count of total records exported from the Syncs Logs endpoint, it's important to note the "message" within that item will be "Successfully exported members to csv file."
    • Example Syncs Logs Response with items "count" and "message" highlighted:

              Example Sync Logs Response

Retrieve the Data with the Sync URI's Data Endpoint

Retrieve the Remaining Data with the Sync URI's Data Endpoint Using Offset

    • Example with URL parameters: GET /syncs/{id}/data?limit=50000&offset=50000
    • Example calls to retrieve all data if there were exactly 150,000 activity records:
      • GET /syncs/{id}/data?limit=50000&offset=0 (This is part of "Retrieve the Data with the Sync URI's Data Endpoint" step)
      • GET /syncs/{id}/data?limit=50000&offset=50000
      • GET /syncs/{id}/data?limit=50000&offset=100000
    • Data returned in JSON unless Accept header specifics CSV (API Call Format)
      • Example: Accept: text/csv
    • A validation check could also be added here.

Create an Activity Export Definition Filtering on Activity Created Date