Bulk sending at scale
If you rely on a high-volume and scalable distribution solution to send envelopes to your customers, with each receiving a unique copy of the document to sign, then the DocuSign Bulk Send feature is the best solution to achieve it.
The v2 version of the Bulk Send API adds the capabilities of the Advanced Bulk Send feature, already available in the DocuSign web app, to the externally available eSignature REST API. This API changes the construction model to that which the standard envelope API employs.
How it works:
Executing an envelope bulk send involves three primary operations. The first operation creates the list of bulk send envelope parameters: subject, recipients, tab values, and envelope custom field values. The second operation creates a draft envelope to serve as the basis for all the envelopes to be sent in the bulk send batch. The third operation creates and sends the batch of envelopes, using the previously created list and draft envelope.
When the third operation of sending the batch envelopes is triggered, DocuSign queues the envelopes for asynchronous processing. Because asynchronous processing results in the envelopes being created at different speeds, your app should not expect envelope creation in real time. Since it's an asynchronous process, the customer system should subscribe for notifications via DocuSign Connect to track the creation of an envelope for each unique record.
Finally, your app should monitor the failure or progress of bulk send batches.
API Operations
Create draft envelope (one time per bulk send trigger)
Purpose:
Create a draft envelope to serve as the basis for all envelopes of a bulk send batch. Since it only needs to be a draft envelope, you can use any existing envelope construction method. The draft should have roles for replacement by your bulk send list recipients. All other envelope characteristics should be included in the draft, such as defining custom envelope fields.
API:
POST https://{env}.docusign.net/restapi/v2.1/accounts/{accountID}/envelopes
The most popular method is to use a DocuSign template to upload the document into DocuSign and use tags to specify what information you need from your signers; you will get a TemplateId, which will be needed to create the draft envelope.
DocuSign recommends this approach, as the business can own template creation and can change the template in the DocuSign web app when needed without any need to modify the calling code.
Note: If your bulk send needs to be done only with one template, then you can use the template directly instead of creating a draft envelope.
The draft envelope technique can also allow developers to add more envelope custom fields if those are not present in the DocuSign server template, and you need to create a draft envelope for each template if you want to use multiple templates in an envelope.
Another option: You can create a draft version (status = “created”) of the envelope with the document, recipient roles, and tags specified without using a template.
API:
POST https://{{env}}.docusign.net/restapi/v2.1/accounts/{{acctID}}/envelopes
{
"compositeTemplates": [
{
"inlineTemplates": [
{
"recipients": {
"signers": [
{
"recipientId": 1,
"roleName": "Manager",
"name": "Multi Bulk Recipient::Manager",
"email": "multiBulk-Manager@docusign.com"
},
{
"recipientId": 2,
"roleName": "SalesRep",
"name": "Multi Bulk Recipient::SalesRep",
"email": "multiBulk-SalesRep@docusign.com"
}
]
},
"sequence": 2,
"customFields": {
"textCustomFields": [
{
"name": "batchName",
"value": ""
},
{
"name": "UseCaseId",
"value": ""
}
]
}
}
],
"serverTemplates": [
{
"templateId": "c20b4c4d-xxxx-xxxx-xxxx-c82af27bdb80",
"sequence": 1
}
]
}
],
"status": "created"
}
Notes:
It's important to specify the status of created to create a draft envelope. It's also important to create any envelope custom fields, such as batchName and UseCaseId, that will be set for each bulk send list for tracking purposes. Using an envelope custom field this way will uniquely identify each set of recipients sent in one batch.
Draft envelopes in DocuSign are deleted 30 days after creation, so you should always create a new draft envelope before the start of a new set of bulk send batches. For instance, if you are planning to process 30 batches today, then create a draft envelope and reuse the same draft envelope for these 30 batches today. Then, if you are doing another round of batches in the future, you should recreate the draft envelope for the new round of batches.
Get RecipientId (one time per bulk send trigger)
Purpose:
Get the recipientId for each recipient in the draft envelope created by DocuSign. Recipient IDs received from this call will be used in the next section.
API:
GET https://{{env}}.docusign.net/restapi/v2.1/accounts/{{acctID}}/{{draftEnvelopeId}}/recipients
Create bulk send list
Purpose:
Create the list of envelope details for the envelopes that will comprise a bulk send batch. The list is to be created on the DocuSign platform via this API operation. Once created, it can be used any number of times when sending a bulk send batch. Use the same recipientId and role mapping which you received in the previous call.
API:
POST https://{{env}}.docusign.net/restapi/v2.1/accounts/{{acctID}}/bulk_send_lists
{
"bulkCopies": [
{
"recipients": [
{
"recipientId": 1,
"role": "SalesRep",
"Name": "Sales One",
"Email": "dstechnuggetssales1@mailinator.com",
"AccessCode": "12345",
"tabs": [
{
"tabLabel": "SalesRepText1",
"initialValue": "SText1111"
}
]
},
{
"recipientId": 2,
"role": "Manager",
"Name": "Manager One",
"Email": "dstechnuggetsmanager1@mailinator.com",
"tabs": [
{
"tabLabel": "ManagerText1",
"initialValue": "MText1111"
}
]
}
],
"customFields": [
{
"name": "batchName",
"value": "NewEmployee_11_3_2020"
},
{
"name": "UseCaseId",
"value": "HR_11_3_2020_NewEmployee"
}
]
}
],
"name": "NewEmployee_11_3_2020"
}
Notes:
The response body is the bulkListId and a repeat back of the request body. The value of the listId is a GUID. Bulk send lists are retained and reusable.
Note how I've made the name and the envelope custom field batchName have the same value. Doing this enables you to maintain mapping for tracking purposes in your database and in DocuSign.
DocuSign limits each batch to no more than 1000 records. Each record means each envelope. So, in this step do not add more than 1000 objects under the BulkCopies property in the JSON body. As a best practice, if possible, try to have a batch with not more than 500 records and make this value configurable. Keeping a low number of records helps to make your batch complete quickly, and it will save some API calls to check the batch status as explained in the "Check batch status to retrieve errors" step (see section below).
Test bulk batch (optional)
Purpose:
Test a bulk send. It leverages a draft envelope as the “template” for testing the bulk-send envelopes.
API:
POST https://{{env}}.docusign.net/restapi/v2.1/accounts/{{acctID}}/bulk_send_lists/{{bulkListId}}/test
{
"envelopeOrTemplateId": "{{draftEnvID}}"
}
Sample response:
Sample failure:
{
"canBeSent": false,
"validationErrors": [
"BULK_SEND_ENVELOPE_NOT_FOUND"
],
"validationErrorDetails": [
"Envelope 063a5637-f1f4-4ed7-b904-2760a590ed92 does not exist or you do not have permission to access it."
]
}
Sample success:{ "canBeSent": true }
The canBeSent flag indicates whether or not the bulk send list is valid. If the flag returns true, then you can proceed to the next API call for the bulk send.
Create and send bulk batch
Purpose:
Executes a bulk send. It leverages a draft envelope as the “template” for creating the bulk send envelopes, and then applies the bulk send list to initiate the queue for creating envelopes for the batch
API:
POST https://{{env}}.docusign.net/restapi/v2.1/accounts/{{acctID}}/bulk_send_lists/{{bulkListId}}/send
{
"envelopeOrTemplateId": "{{draftEnvID}}"
}
Sample Response:
Sample success:
{
"envelopeOrTemplateId": "27bdba63-xxxx-xxxx-xxxxx-cbb65161e121",
"batchId": "c42dd06a-xxxx-xxxx-xxxx-09d9a1824f62",
"batchName": " NewEmployee_11_3_2020",
"batchSize": "2",
"totalQueued": "2",
"queueLimit": "2000"
}
Sample exception if queue limit is hit:
{
"envelopeOrTemplateId": "27bdba63-xxxx-xxxx-xxxx-cbb65161e121",
"batchId": "c42dd06a-xxxx-xxxx-xxxx-09d9a1824f62",
"batchName": " NewEmployee_11_3_2020",
"batchSize": "500",
"totalQueued": "1800",
"queueLimit": "2000",
"errors": [
"BULK_SEND_ACCOUNT_HAS_TOO_MANY_QUEUED_ENVELOPES"
],
"errorDetails": [
"Cannot send this bulk sending list because doing so would exceed the maximum of 2000 in-flight envelopes.
This account currently has 1800 envelopes waiting to be processed. Please try again later."
]
}
Notes:
The response body contains the envelopeOrTemplateId for the draft envelope used and batchId for this bulk send batch.
The response also shows the current queueLimit for this DocuSign account and how many records are queued (totalqueued) and how many (batchSize) are sent in this current batch.
Another limit in DocuSign for bulk send is queue size. This is a DocuSign account-level limit. By default, DocuSign does not allow you to queue more than 2000 bulk send records for an account. For instance, if you have a batch size of 500 and you have already sent four batches in succession, and then try to send another batch, then you will get a queue size limit exception like the sample below. DocuSign recommends you design your system with some tolerance to handle such exceptions; one such way is explained below.
ApiErrorCode: BULK_SEND_ACCOUNT_HAS_TOO_MANY_QUEUED_ENVELOPES
Sample ApiErrorMessage: Cannot send this bulk sending list because doing so would exceed the maximum of 2000 in-flight envelopes. This account currently has 1800 envelopes waiting to be processed. Please try again later.
If you get this exception from DocuSign from this API call, then catch the response body to know if the current BatchSize push was making totalQueued more than the queueLimit.
Check the errorDetails property in the response body, and if it contains a text of “envelopes waiting” or or if the error code is “BULK_SEND_ACCOUNT_HAS_TOO_MANY_QUEUED_ENVELOPES”, then it means the queue limit has been reached. This errorDetails property will also return the current queue limit and maximum queue limit.
To handle this exception, your calling code can put the thread to sleep for n seconds before retrying it instead of killing the process. DocuSign recommends that you create an incremental backout counter for each sleep cycle. If Thread1 is put to sleep for three minutes the first time, and on retry it again gets the same exception, then next time it should retry after 3 * backout counter, so it will have a sleep pattern like 3,6,9…
If you get an exception from DocuSign for some other reason than the queue limit being reached, then park the batch for later analysis and save the failure in your database. Save the bulkListId as well in the database along with the batchFailureDateTime.
On a successful response, save bulkListId, batchId, batchName and batchSize in the database. DocuSign recommends you save the batchStatus, such as "BatchSubmitted" and the batchSubmittedDateTime for tracking purposes.
Check batch status to retrieve errors
Purpose:
This operation is used to check on the status of an outgoing bulk send batch in terms of what is queued, what has been sent, and errors encountered.
API:
GET https://{{env}}.docusign.net/restapi/v2.1/accounts/{{acctID}}/bulk _send_batch/{{batchId}}
Notes:
This API provides a polling method to determine the status and outcome of a bulk send batch. This is mainly useful in understanding if part of the batch is still queued awaiting envelope creation, as well as knowing if any failed. However, it only provides counts.
Example response body:
{
"batchId": "c42dd06a-xxxx-xxxx-xxxx-09d9a1824f62",
"batchSize": "2",
"sent": "1",
"queued": "0",
"failed": "1",
"envelopeIdOrTemplateId": "27bdba63-xxxx-xxxx-xxxx-cbb65161e121",
"mailingListId": "0f098a23-xxxx-xxxx-xxxx-b59467c1a285",
"ownerUserId": "adbfb163-xxxx-xxxx-xxxx-f7b7f06dff5a",
"senderUserId": "adbfb163-xxxx-xxxx-xxxx-f7b7f06dff5a",
"batchName": "NewEmployee_11_3_2020",
"submittedDate": "2020-11-04T07:33:18.3570000Z",
"envelopesUri":
"/envelopes/?from_date=11-2-2020&custom_field=BulkBatchId=c42dd06a-xxxx-xxxx-xxxx-09d9a1824f62",
"bulkErrors": [
{
"errorMessage": "SMS authentication phone number cannot be empty. Recipient: Manager
Two(dstechnuggetsmanager2@mailinator.com).",
"recipientEmails": [
"dstechnuggetssales2@mailinator.com",
"dstechnuggetsmanager2@mailinator.com"
],
"created": "2020-11-03T23:33:26.7439932Z"
}
]
}
Notes:
Call (poll) this API once every 30-60 mins for each batch until batchSize is equal to the sent + failed properties returned in the response above and/or batchStatus is not "BatchFinished" in your database.
In parallel, update the value of batchStatus in your batch-related table(s) to track the progress of the batch. Once batchSize is equal to sent + failed, change the value of batchStatus to "BatchFinished" so that you do not poll for this batch ID in the next iteration.
Also save bulkErrors in the database, these errors will contain only recipient emails, so you need to trace them back to the original set of data sent in the batchName property, which was set when sending the batch as name and setting the envelope custom field batchName.
The value of the envelopesUri property returned in the response can help you in getting a list of envelopes sent in this batch, DocuSign recommends you use DocuSign Connect to retrieve envelopes linked to this batch instead of using this API endpoint.
For any API Integrations:
To prevent a single customer from impacting everyone, DocuSign eSignature API has several API request limits designed to protect the performance of your integrations. Please review eSignature API rules and resource limits for more information.
DocuSign Connect (a notification service)
Need for DS Connect:
DocuSign imposes an API-calls-per-hour limit; by default, you cannot make more than 1000 API calls per hour. If you try to breach this limit, you will get an HOURLY_APIINVOCATION_LIMIT_EXCEEDED error from DocuSign. So, as a best practice, use DocuSign Connect wherever possible instead of calling DocuSign APIs, and DocuSign Connect can also help you to build scalable systems.
Purpose:
DocuSign Connect is a push notification service that sends real‐time data updates to external applications for any envelopes (synchronously or asynchronously) created in the DocuSign account. Think of DocuSign Connect like an Amazon delivery notification message that you receive when a package is delivered at your doorstep.
In the bulk sending scenario, since envelopes are created asynchronously, DocuSign Connect sends its notifications to your application for each envelope as soon as it's created by DocuSign during batch processing. Registering for these notifications is normally done either through an account-based Connect custom configuration or via event notification on individual envelopes.
Once your app receives a notification, then it should react based on the notification, either to parse the responseto save data in the database or trigger an asynchronous job to download the document from DocuSign or put this message in an internal queuing mechanism for later processing.
DS Connect expects a response within 100 seconds from an external application, so DocuSign highly recommends that your app return an HTTP 200 response as soon as the message is received, and then process the notification message.
To keep track of the envelopes created by the batches submitted, DocuSign will send the BulkBatchId as the EnvelopeCustomField in the Connect message, and this will help you to track what envelopes are created by the BulkBatchId created earlier. Tracking each envelope’s BulkBatchId will help in analyzing to determine whether all envelopes in that batch were successfully sent by DocuSign or not, and if the batch completed or not. You should also update the batch-related tables in your database that are used by the Check Batch Status to retrieve Errors step.
Since a lot of envelope-related data (like recipient data, signature event data at the recipient and envelope level, form data, and more) are sent in the Connect message, saving such data in the database can provide lots of support for troubleshooting and providing any metrics or reports to the business owners or stakeholders.
DocuSign highly recommends you select the settings below in your DocuSign Connect configuration:
- Enable Log (maximum 100)
- Require Acknowledgement
- Time Zone Information
Selecting Enable Log will make DocuSign preserve the last 100 Connect messages in DocuSign for any troubleshooting if needed.
If the Require Acknowledgement option is selected and a publication message fails to be acknowledged, the message goes back into the queue and the system will retry delivery until a successful acknowledgement is received.
Selecting Time Zone Information will send the time zone information shown below so you can see the time zone used in any date-time element data in the Connect message.
<TimeZone>Central Standard Time</TimeZone> <TimeZoneOffset>-5</TimeZoneOffset>
Note: The order of the Connect message for an envelope is not 100% guaranteed, so it is better to use the TimeGenerated element from the Connect message to keep messages in order in your application.
There is an option to include Document PDFs in the Connect settings, but it is best practice to NOT select it and instead use the DocuSign eSignature REST API to pull documents as archive from DocuSign once your app receives a notification. If this option is selected in the Connect configuration, then DocuSign sends the documents in Base64 format only, and Base64 inflates the message by 30%; this requires more bandwidth to transfer data from DocuSign to external applications. DocuSign recommends instead that, on receiving the notification from DocuSign, your app should trigger an asynchronous job to pull documents in binary format from DocuSign via the API call below. With this single API call, you can pull one or more binary documents at a time.
GET /restapi/v2.1/accounts/{accountId}/envelopes/{envelopeId}/documents/archive
Not selecting the Document PDFs option helps to make the Connect listener a thin client, and greatly improves message processing time.
