Edit

Background operations (preview)

  • Article

[This article is prerelease documentation and is subject to change.]

Use background operations to send requests that Dataverse processes asynchronously. Background operations are useful when you don't want to maintain a connection while a request runs.

When a background operation completes, you can get notified in either of two ways:

You can retrieve the result of a background operation in either of two ways:

To run a request in the background, the operation must be defined as a custom API. Learn how to create and use custom APIs and retrieve data about custom APIs.

Custom APIs use plug-ins to perform data operations. Like all Dataverse plug-ins, these plug-ins have a two-minute execution time-out. Sending the request asynchronously doesn't provide more execution time.

Required privileges

To perform a background operation, the initiating user must have read and write access to the backgroundoperations table. Assign the prvReadbackgroundoperation and prvWritebackgroundoperation privileges to grant this access.

Learn how to edit a security role.

Request asynchronous processing

You can run asynchronous requests in the background using either the SDK for .NET or the Dataverse Web API.

Examples in this article use a custom API named sample_ExportDataUsingFetchXmlToAnnotation. This custom API is described in Sample: ExportDataUsingFetchXmlToAnnotation custom API.

Use the ExecuteBackgroundOperation message.

The SDK doesn't have ExecuteBackgroundOperation request and response classes. Until these classes are added, use the base OrganizationRequest and OrganizationResponse classes as described in Use messages with the SDK for .NET.

The following table describes the input parameters for the ExecuteBackgroundOperation message.

Name Type Description
Request OrganizationRequest (Required) Contains the request you want to have processed asynchronously. The Dataverse message for the request must be implemented as a custom API.
CallbackUri string (Optional) Dataverse sends a POST HTTP request to this URL when the operation is completed.

The following table describes the output parameters for the ExecuteBackgroundOperation message.

Name Type Description
BackgroundOperationId Guid Identifies the background operation table row you can use to monitor or cancel the processing of your request.
Location string Identifies the status monitor resource URL you can use to retrieve the status of your request or to cancel it.

The following static method uses ExecuteBackgroundOperation with the sample_ExportDataUsingFetchXmlToAnnotation custom API.

static void SendRequestAsynchronously(IOrganizationService service)
{
    //Create a request for message defined as a custom API to run in the background
    var asyncRequest = new OrganizationRequest("sample_ExportDataUsingFetchXmlToAnnotation")
    {
        Parameters =
        {
            {"FetchXml",  @"<fetch version='1.0'  
                                    output-format='xml-platform' 
                                    mapping='logical'>
                                <entity name='account'>
                                    <attribute name='accountid'/>
                                    <attribute name='name'/>  
                                </entity>
                            </fetch>" }
        }
    };

    //Create a request to execute the message in the background
    var request = new OrganizationRequest("ExecuteBackgroundOperation")
    {
        Parameters =
        {
            {"Request", asyncRequest }
        }
    };

    //Execute the background operation request
    var response = service.Execute(request);

    Console.WriteLine($"BackgroundOperationId: {response["BackgroundOperationId"]}");
    Console.WriteLine($"Location: {response["Location"]}");
}

Output:

BackgroundOperationId: <backgroundoperationid value>
Location: [Organization URI]/api/backgroundoperation/<backgroundoperationid value>

Learn more about the IOrganizationService interface and how to use messages with the SDK for .NET.

Manage background operations

When you send a request to be processed in the background, the response includes two values that represent different methods you can use to monitor or cancel background operations.

Querying the background operations table or status monitor resource to check on requests is commonly known as status polling. We recommend that you avoid excessive polling because it can negatively affect performance. If needed, we suggest polling at an interval of one minute or more.

Background operations table

The background operations table contains information about requests to process asynchronously. This table has the logical name backgroundoperation and the entity set name backgroundoperations. Learn about the backgroundoperation EntityType.

The following table describes the columns you can use to manage the status of background operations.

Display name
Schema name
Logical name		 Type Description
Background Operation
BackgroundOperationId
backgroundoperationid		 Uniqueidentifier The primary key
Status
StateCode
backgroundoperationstatecode		 Picklist State of the background operation

Options:
- Value: 0, Label: Ready
- Value: 2, Label: Locked
- Value: 3, Label: Completed
Status Reason
StatusCode
backgroundoperationstatuscode		 Picklist Status of the background operation

Options:
- Value: 0, Label: Waiting For Resources (State:Ready)
- Value: 20, Label: In Progress (State:Locked)
- Value: 22, Label: Canceling (State:Locked)
- Value: 30, Label: Succeeded (State:Completed)
- Value: 31, Label: Failed (State:Completed)
- Value: 32, Label: Canceled (State:Completed)
Name
Name
name		 String The UniqueName of the custom API used for the background operation
DisplayName
DisplayName
displayname		 String The DisplayName of the custom API used for the background operation
Input Parameters
InputParameters
inputparameters		 Memo The input parameters that were supplied to start the background operation

This string is a JSON serialized array of Key and Value.
Output Parameters
OutputParameters
outputparameters		 Memo The response of the background operation

This string is a JSON serialized array of Key and Value.
Start Time
StartTime
starttime		 DateTime When the background operation started execution
End Time
EndTime
endtime		 DateTime When the background operation finished execution
Retry Count
RetryCount
retrycount		 Integer The number of times the background operation was retried
Error Code
ErrorCode
errorcode		 Integer The error code if the background operation fails

If the error comes from Dataverse, it has an integer value that corresponds to one of the codes listed in Web service error codes. If the error didn't come from Dataverse, the value is set to zero.
Error Message
ErrorMessage
errormessage		 Memo The error message if the background operation fails
Run As
RunAs
runas		 String The systemuserid of the systemuser used to execute the background operation
Created On
CreatedOn
createdon		 DateTime When the record was created
Time to live
TTLInSeconds
ttlinseconds		 Integer Time to live in seconds, after which the record is automatically deleted; the default value is 90 days

Poll the background operations table

Make sure to include these columns in your query:

  • name
  • backgroundoperationstatecode
  • backgroundoperationstatuscode
  • outputparameters
  • errorcode
  • errormessage

How you poll the table depends on whether you're using the SDK or the Web API.

static void PollBackgroundOperationRequest(IOrganizationService service, Guid backgroundOperationId)
{
    // List of columns that will help to get status, output and error details if any
    var columnSet = new ColumnSet(
        "name",
        "backgroundoperationstatecode",
        "backgroundoperationstatuscode",
        "outputparameters",
        "errorcode",
        "errormessage");

    try
    {
        // Get the entity with all the required columns
        var backgroundOperation = service.Retrieve("backgroundoperation", backgroundOperationId, columnSet);

        Console.WriteLine($"Name: {backgroundOperation["name"]}");
        Console.WriteLine($"State Code: {backgroundOperation.FormattedValues["backgroundoperationstatecode"]}");
        Console.WriteLine($"Status Code: {backgroundOperation.FormattedValues["backgroundoperationstatuscode"]}");
        Console.WriteLine($"Output Parameters:");

        // Deserialize the Output Parameters into KeyValuePair<string, string>
        List<KeyValuePair<string, string>>? output = 
            System.Text.Json.JsonSerializer
            .Deserialize<List<KeyValuePair<string, string>>>((string)backgroundOperation["outputparameters"]);

        output.ForEach(x => {
            Console.WriteLine($"\t{x.Key}: {x.Value}");
        });

        Console.WriteLine($"Error Code: {backgroundOperation.GetAttributeValue<string>("errorcode")}");
        Console.WriteLine($"Error Message: {backgroundOperation.GetAttributeValue<string>("errormessage")}");
    }
    // Catch Dataverse errors
    catch (FaultException<OrganizationServiceFault> ex)
    {
        Console.WriteLine($"ErrorCode:{ex.Detail.ErrorCode}");
        Console.WriteLine($"Message:{ex.Detail.Message}");
    }
    // Catch other errors
    catch (Exception error)
    {
        Console.WriteLine($"Some other error occurred: '{error.Message}'");
    }
}

Waiting output:

Name: sample_ExportDataUsingFetchXmlToAnnotation
State Code: Locked
Status Code:  In Progress
Output Parameters:  
Error Code:  
Error Message:

Complete output:

Name: sample_ExportDataUsingFetchXmlToAnnotation
State Code: Completed
Status Code:  Succeeded
Output Parameters:
        AnnotationId: {value}
Error Code:  
Error Message:

Error output:

Name: sample_ExportDataUsingFetchXmlToAnnotation
State Code: Completed
Status Code:  Failed
Output Parameters: 
Error Code:  -2147187707
Error Message:  Access is denied.

If the platform produces the error, it has an integer value that corresponds to one of the codes listed in the Web service error codes. If the platform doesn't produce the error, its value is set to zero.

Id not found:

ErrorCode:-2147185406
Message:The HTTP status code of the response was not expected (404).

Status: 404
Response:
{"error":{"message":"Could not find item '110eaa68-db17-4115-ad74-d185823fc089'.","details":[{"message":"\r\nErrors : [\r\n  \"Resource Not Found. Learn more: https://aka.ms/cosmosdb-tsg-not-found\"\r\n]\r\n"}]}}

Poll the status monitor resource

You can poll the status monitor resource with a GET request, which returns the status of the background operation. If the operation is complete, it provides the output of the custom API. If there was an error during execution, you receive an error message and code.

Send a request to the status monitor resource URL that was returned with the Location response header of the original request.

Request:

GET [Organization URI]/api/backgroundoperation/{backgroundoperationid}
Content-Type: application/json

Response:

HTTP/1.1 200 OK
Content-Type: application/json

{
  backgroundOperationErrorCode: {INT},
  backgroundOperationErrorMessage: {string},
  backgroundOperationStateCode: {INT},
  backgroundOperationStatusCode: {INT},
  outputParam1: {value},
  outputParam2: {value},
  outputParam3: {value},
}

backgroundOperationErrorCode and backgroundOperationErrorMessage values are included only when an error occurs. Output parameters are included only when the operation completes successfully.

Labels aren't available with the status monitor resource.

Receive notification of result

To get a notification when a background operation completes, you can either include a callback URL with your request or subscribe to the OnBackgroundOperationComplete event.

Request a callback

You can specify a URL in your request to receive a callback when the operation is completed. Dataverse uses this URL to send a POST request with the following payload:

{
    "location": "< status monitor resource URL >",
    "backgroundOperationId": "{GUID}",
    "backgroundOperationStateCode": {INT},
    "backgroundOperationStatusCode": {INT},
    "backgroundOperationErrorCode": {INT},
    "backgroundOperationErrorMessage": {string},
}

backgroundOperationErrorCode and backgroundOperationErrorMessage are included only when an error occurs.

The callback payload doesn't include any output parameters. The site that receives the callback must send an authenticated GET request using the status monitor resource URL to get results.

If the URL requires authentication, it must be a self-sufficient shared access signature (SAS) URL. It isn't possible to include any more headers to include API keys or tokens for authentication.

You may want to use a site like webhook.site to test the callback URL.

How you request a callback depends on whether you're using the SDK or the Web API. The following examples send a request using a webhook to webhook.site for testing.

With the SDK, set the ExecuteBackgroundOperation.CallbackUri parameter to the URL to send the request.

static void SendRequestAsynchronouslyWithCallback(IOrganizationService service)
{
    //Create a request for message defined as a custom API to run in the background
    var asyncRequest = new OrganizationRequest("sample_ExportDataUsingFetchXmlToAnnotation")
    {
        Parameters =
        {
            {"FetchXml",  @"<fetch version='1.0'  
                                    output-format='xml-platform' 
                                    mapping='logical'>
                                <entity name='account'>
                                    <attribute name='accountid'/>
                                    <attribute name='name'/>  
                                </entity>
                            </fetch>" }
        }
    };

    //Create a request to execute the message in the background
    var request = new OrganizationRequest("ExecuteBackgroundOperation")
    {
        Parameters =
        {
            {"Request", asyncRequest },
            // Request a callback
            {"CallbackUri", "https://webhook.site/<id>" }
        }
    };

    //Execute the background operation request
    var response = service.Execute(request);

    Console.WriteLine($"BackgroundOperationId: {response["BackgroundOperationId"]}");
    Console.WriteLine($"Location: {response["Location"]}");
}

Subscribe to the OnBackgroundOperationComplete event

Another way to receive a notification when a background operation completes is to register a step on the OnBackgroundOperationComplete message. This message is a custom API that only allows asynchronous step registrations. It's an example of the type of messages created using a custom API to represent business events.

As the name suggests, the OnBackgroundOperationComplete event occurs each time a background operation completes. When you register an asynchronous step on this event, you can perform any type of logic you want in a plug-in or forward the data to Azure services or a webhook. Learn more:

The following tables describe the input and output parameters for the OnBackgroundOperationComplete message.

Input parameters:

Name Type Description
PayloadType Integer What type of response is sent to the callback URI when the background operation is complete; always zero for background operations

This field is internal and shouldn't be updated.
LocationUrl String Location URL
BackgroundOperationId Guid The ID of the background operation

Output parameters:

Name Type Description
OperationName String Operation name
BackgroundOperationStateCode Integer Background operation state code
BackgroundOperationStatusCode Integer Background operation status code

Configure the OnBackgroundOperationComplete message as shown in the instructions to register a plug-in. Make sure you set the message name as OnBackgroundOperationComplete. Set Auto Delete to true so that the System Job (AsyncOperation) record is automatically removed.

Cancel background operations

You can cancel a background operation that you initiated if it hasn't started.

  • If the operation hasn't started, Dataverse doesn't execute it.
  • If the operation has started, Dataverse doesn't stop it.
  • If an error occurs during execution of a background operation you canceled, Dataverse doesn't retry it.
  • If the operation has already completed, you get the following error: Canceling background operation is not allowed after it is in terminal state.

You can cancel a background operation in either of two ways:

Cancel a background operation by updating backgroundoperations

Update the row in the backgroundoperations table to set the backgroundoperationstatecode to 2 (Locked) and backgroundoperationstatuscode to 22 (Cancelling).

How you update the backgroundoperations table depends on whether you're using the SDK or the Web API.

static void CancelBackgroundOperationRequest(
    IOrganizationService service, 
    Guid backgroundOperationId)
{
    var backgroundOperation = new Entity(
        entityName: "backgroundoperation", 
        id: backgroundOperationId)
    { 
        Attributes =
        {
            //Set state as Locked
            {"backgroundoperationstatecode", new OptionSetValue(2) },
            //Set status as Cancelling
            {"backgroundoperationstatuscode", new OptionSetValue(22) }
        }            
    }; 

    service.Update(backgroundOperation);
}

Send a DELETE request to the status monitor resource

You can also cancel a background operation by sending a DELETE request to the status monitor resource.

Request:

DELETE [Organization URI]/api/backgroundoperation/{backgroundoperationid}

Response:

HTTP/1.1 200 Ok

{
    backgroundOperationStateCode: 2,
    backgroundOperationStatusCode: 22
}

Retries

If an error occurs during execution of the request, it's retried up to three times. These retries use an exponential backoff strategy.