Bidders - Troubleshooting BSS uploads

Article
02/27/2024

You can use the methods in this topic to diagnose any problems with your segment data upload.

Different Phases of Batch Segment Service

This section describes the phases of upload, so you can understand where problems may be occurring.

Starting - Request upload URL and Job ID

In this phase, the clients request an upload URL and a job ID. This stage times out in 5 minutes. If jobs are stuck in this phase, it indicates that the clients requested the URL but failed to upload anything within the allotted time.

Uploading

In this phase, the clients upload the file to the given URL. It is recommended not to exceed one upload per minute. If the clients have more than 200 jobs waiting to be processed at any given time, they will be prohibited from uploading additional jobs.

Validating and Processing

Once the clients upload the file, processing of the file occurs in the next phase where the validation of segment IDs and user IDs is carried out. If a record contains invalid segment or user IDs, the platform does not process the record. When the clients check the job status, they will be able to view statistics regarding the number of invalid users.

Completion

In this phase, the data of the file is successfully uploaded to the platform, and it is available for targeting.

Possible upload errors

Attempting to upload file larger than 0.5GB

{"response":{"status":"ERROR","error_code":"FILESIZE_LIMIT_EXCEEDED","errors":["Member
            exceeds maximum byte size allowed for a file"]}}

Error code in batch segment upload job

""batch_segment_upload_job": {
      "phase": "error",
      "start_time": "2015-08-13 18:40:32",
      "uploaded_time": null,
      "validated_time": null,
      "completed_time": null,
      "error_code": "uploading-error",
      "time_to_process": "0.00",
      "percent_complete": 0,
      "num_valid": 0,
      "num_invalid_format": 0,
      "num_valid_user": 0,
      "num_invalid_user": 0,
      "num_invalid_segment": 0,
      "num_invalid_timestamp": 0,
      "num_unauth_segment": 0,
      "num_past_expiration": 0,
      "num_inactive_segment": 0,
      "num_other_error": 0,
      "error_log_lines": null,
      "segment_log_lines": null,
      "id": 11661553,
      "job_id": "Pm3oCUf5CSVKIOt4mAqOzdt6K3qInj1431542432",
      "member_id": 958,
      "created_on": "2015-05-13 18:40:32",
      "last_modified": "2015-05-13 18:40:33"
    }

The following errors may happen when:

You have reached one of its four upload limits:

daily bytes,
hourly bytes,
daily lines, or
hourly lines

Attempting to exceed daily byte upload limit

{"response":{"status":"ERROR","error_code":"RATE_LIMIT_EXCEEDED","errors":["Member
                  exceeds maximum allowed bytes per day"]}}

Attempting to exceed hourly byte upload limit

{"response":{"status":"ERROR","error_code":"RATE_LIMIT_EXCEEDED","errors":["Member
                  exceeds maximum allowed bytes per hour"]}}

Attempting to exceed daily lines upload limit

{"response":{"status":"ERROR","error_code":"RATE_LIMIT_EXCEEDED","errors":["Member
                  exceeds maximum allowed number of lines per day"]}}

Attempting to exceed hourly lines upload limit

{"response":{"status":"ERROR","error_code":"RATE_LIMIT_EXCEEDED","errors":["Member
                  exceeds maximum allowed number of lines per hour"]}}

You have canceled the upload.

The upload phase exceeds 90 minutes.

Exceeding maximum time to upload

{"response":{"status":"ERROR","error_code":"RATE_LIMIT_EXCEEDED","errors":["Maximum
                  upload time exceeded"]}}

Possible processing errors

Invalid format

If the value of the num_invalid_format field is greater than "0", check the values in the error_log_lines field.

In the example below, the num_invalid_format field displays a value of "1", with details provided in the error_log_lines field.

In the error_log_lines field:

num_invalid_format indicates that there was a problem parsing a line in the uploaded file.
"failed with an illegal number of fields" indicates that the number of fields in a segment_fields block did not match what was defined in the batch-segment config (for more information, see Initial BSS Account Setup).

In this case, the config expects three fields to be defined in the block: SEG_ID, VALUE, EXPIRATION, but the parser only found two fields - SEG_ID and VALUE, thus showing an error.

num_invalid_format and error_log_lines example

"batch_segment_upload_job": {
phase": "completed",
"error_code": null,
"time_to_process": "0.01",
"percent_complete": 100,
"num_valid": 0,
"num_invalid_format": 1,
"num_valid_user": 0,
"num_invalid_user": 0,
"num_invalid_segment": 0,
"num_invalid_timestamp": 0,
"num_unauth_segment": 0,
"num_past_expiration": 0,
"num_inactive_segment": 0,
"num_other_error": 0,
"error_log_lines": "num_invalid_format-WINDOWSADID-USER-ID;SEG_ID:VALUE~9 failed with an illegal number of fields",
"segment_log_lines": null,
"start_time": "2015-08-13 18:40:32",
"uploaded_time": "2015-08-13 18:42:32",
"validated_time": "2015-08-13 18:42:32",
"completed_time": "2015-08-13 18:42:33",
"id": 123412341234,
"job_id": "Pm3oCUf5CSVKIOt4mAqOzdt6K3qInj1431542432",
"member_id": 958,
"created_on": "2015-08-13 18:40:32",
"last_modified": "2015-08-13 18:42:33"
}

View your file upload history

To see metadata about all your segment file uploads within the last 30 days, make a GET call to the service with your member_id specified in the query string. The JSON response will include an array of batch_segment_upload_job objects.

For more information about the specific fields of the batch_segment_upload_job object, see JSON Fields.

Note

File upload history is available for the last 30 days only.

$ curl -b cookies 'https://api.appnexus.com/batch-segment?member_id=456'
{
   "response" : {
      "batch_segment_upload_job" : [
         {
           "phase": "completed",
            "start_time": "2012-05-22 16:48:55",
            "uploaded_time": "2012-05-22 16:48:56",
            "validated_time": "2012-05-22 16:49:01",
            "completed_time": "2012-05-22 16:49:01",
            "error_code": null,
            "time_to_process": "0.04",
            "percent_complete": 100,
            "num_valid": 0,
            "num_invalid_format": 0,
            "num_invalid_user": 2,
            "num_invalid_segment": 0,
            "num_unauth_segment": 1,
            "num_past_expiration": 0,
            "num_inactive_segment": 0,
            "num_other_error": 0,
            "error_log_lines": " \n\nnum_unauth_segment-4013681496264948522;5013:0,5014:1550\nnum_invalid_user-7652266028043224430;5848:0,5849:1440,5850:1440\nnum_invalid_user-8802117132500293405;5851:0,5847:-1",
            "id": 98,
            "job_id": "T1v98eIOlCZndeLGSXD0nrs57L8ES11337705335",
            "member_id": 456,
            "created_on": "2012-05-22 16:48:55",
            "last_modified": "2012-05-22 16:49:01"
         },
         ...
    }
  }
}

Note

Our API limits responses to 100 objects via pagination. You can view additional objects by appending one of these to the API call:

&start_element=101
&sort=last_modified.desc

You can read more about pagination on our Documentation Portal here.

If you're still experiencing technical issues, you can submit a request on our Customer Support Portal. Don't forget to include the job ID in your support request.

JSON fields

HTTP Method	Endpoint	Description
`GET`	`https://api.appnexus.com/batch-segment/meta`	Use this call to find out which fields you can filter and sort by.

Fields	Type	Description
`batch_segment_upload_job`	object	This is an object whose fields contain metadata describing the upload and processing job. If you are using the Impbus API, this will be an array containing a single object. See Batch Segment Upload Job for details.
`id`	int	This is the ID of the `batch_segment_upload_job` object associated with this request. Default: An automatically generated number.
`status`	string	The status of the API call; successful calls return `"OK"`.

Batch segment upload job

When you request the status of your processing job, the system returns a batch_segment_upload_job object (if you are a data provider, this will be an array containing a single object). Depending on which request you're making to the service, it will contain some or all of the following metadata.

Note

Most metadata will only be present when "phase": "completed".

Fields	Type	Description
`completed_time`	date	The time at which file processing was completed.
`created_on`	date	The creation date of this object.
`error_code`	int	If `"phase": "error"`, this error code describes the type of error encountered. Note that an error code will only be shown here if there was an error with the uploading, validating, or processing of the file itself (i.e., does not include invalid format or invalid segment errors). Common errors are caused by unreadable files and exceeding defined object limits. Returns `null` if no errors were found.
`error_log_lines`	string	A string containing newline-separated lines. Each line lists a validation error or the reason for an error while uploading your file. You can choose how many lines appear in this field. Default: 200 lines
`id`	int	The unique identifier of this object.
`job_id`	string	A string of alphanumeric characters that uniquely identifies the processing job associated with this file.
`last_modified`	date	The most recent modification date of this object (usually via `POST`).
`member_id`	int	Your member ID. Required on: `PUT`, `POST`
`num_inactive_segment`	int	The number of inactive segments in the file. Deduplicated.
`num_invalid_format`	int	The number of uploaded lines containing formatting errors. This depends upon your particular file format configuration. Duplicate lines will also be considered an invalid format.
`num_invalid_segment`	int	The number of invalid segments in the file. Deduplicated.
`num_invalid_timestamp`	int	The number of invalid timestamps in the file.
`num_invalid_user`	int	A count of unique input lines that have an invalid or non-existent user.
`num_other_error`	int	This is a placeholder value not currently in use.
`num_past_expiration`	int	The number of expired segments in the file. Deduplicated.
`num_unauth_segment`	int	The number of segments in the file which you are unauthorized to access. Deduplicated.
`num_valid`	int	The number of valid lines in the uploaded file. Each user/segment combination is considered 1 line.
`num_valid_user`	int	A count of unique input lines that have a valid user ID.
`percent_complete`	int	The percentage of the processing that has been completed, given the current `phase` at the time of the request.
`phase`	enum	The current processing status. Returns one of the following values: - `error` - `starting` - `uploading` - `validating` - `processing` - `completed`
`segment_log_lines`	string	A string containing newline-separated lines consisting of the segment ID and the number of users successfully added or removed. This field defaults to 200 lines. The format is `added: SEG_ID:COUNT SEG_ID:COUNT ... removed:` `SEG_ID:COUNT` ... where `SEG_ID` is the segment ID and `COUNT` is the number of users successfully added or removed. `SEG_ID:COUNT` pairs are sorted by `COUNT` (descending). Example: `added: 15889133:386221 15547290:186227 removed: 15889278:369734 15889206:255307 15889179:232831`
`start_time`	date	The time at which file upload began.
`time_to_process`	decimal	The time it took to process the segment file, in minutes.
`upload_url`	string	The URL where you'll upload your segment data file.
`uploaded_time`	date	The time at which the file associated with this job ID was uploaded.
`validated_time`	date	The time at which file validation was completed.

Share via

Bidders - Troubleshooting BSS uploads

Different Phases of Batch Segment Service

Possible upload errors

Possible processing errors

View your file upload history

JSON fields

Batch segment upload job

Additional resources

Share via

Bidders - Troubleshooting BSS uploads

Different Phases of Batch Segment Service

Possible upload errors

Possible processing errors

View your file upload history

JSON fields

Batch segment upload job

Related topics

Additional resources