Menu
Amazon Neptune
User Guide (API Version 2017-11-29)

Loader Get Status

Preview Release    Amazon Neptune Preview is available only to whitelisted customers. To request access to Neptune, see the information on the Amazon Neptune Preview page.

Gets the status of a loader job.

To get load status, you must send an HTTP GET request to the http://your-neptune-endpoint:8182/loader endpoint. To get the status for a particular load request, you must include the loadId as a URL parameter, or the loadId can be appended to the URL path.

Request Syntax

GET http://your-neptune-endpoint:8182/loader?loadId=loadId
GET http://your-neptune-endpoint:8182/loader/loadId
GET http://your-neptune-endpoint:8182/loader

Request Parameters

loadId

The ID of the load job. If you do not specify a loadId, a list of load IDs is returned.

details

Include details beyond overall status. Default: False

Allowed values: TRUE, FALSE

errors

Include the list of errors. The list of errors is paged. The page and errorsPerPage parameters allow you to page through all the errors. Default: False

Allowed values: TRUE, FALSE

page

The error page number. Only valid with the errors parameter set to TRUE. Default: 1

Allowed values: Positive integers

errorsPerPage

The number of errors per each page. Only valid with the errors parameter set to TRUE. Default: 10

Allowed values: Positive integers

limit

The number of load ids to list. Only valid when requesting a list of load IDs by sending a GET request with no loadId specified. Default: 100

Allowed values: Positive integers, 1 - 100

Response Syntax

{ "status" : "200 OK", "payload" : { "feedCount" : [ { "LOAD_FAILED" : number } ], "overallStatus" : { "fullUri" : "s3://bucket/key", "runNumber" : number, "retryNumber" : number, "status" : "string", "totalTimeSpent" : number, "totalRecords" : number, "totalDuplicates" : number, "parsingErrors" : number, "datatypeMismatchErrors" : number, "insertErrors" : number, }, "failedFeeds" : [ { "fullUri" : "s3://bucket/key", "runNumber" : number, "retryNumber" : number, "status" : "string", "totalTimeSpent" : number, "totalRecords" : number, "totalDuplicates" : number, "parsingErrors" : number, "datatypeMismatchErrors" : number, "insertErrors" : number, } ], "errors" : { "startIndex" : number, "endIndex" : number, "loadId" : "string, "errorLogs" : [ ] } } } }

200 OK

Successful status check returns a 200 code.

Errors

When an error occurs, a JSON object is returned in the BODY of the response. The message object contains a description of the error.

Error 400

An invalid loadId returns a 400 bad request error. The message describes the error.

Error 500

A valid request that cannot be processed returns a 500 internal server error. The message describes the error.

Response Fields

The following are the fields returned by the status command with descriptions.

These fields appear in the overallStatus field for all loads, and the failedFeeds for each failed feed.

fullUri

Type: string

The uri of the file or files to be loaded. In the format: s3://bucket/key

runNumber

Type: unsigned long

The run number of this load or feed. This is incremented when the load is restarted.

retryNumber

Type: unsigned long

The retry number of this load or feed. This is incremented when the loader retries a feed or load automatically.

status

Type: string

The returned status of the load or feed. LOAD_COMPLETED indicates a successful load with no problems. For a list of all load status values, see Overall and Feed Status Descriptions.

totalTimeSpent

Type: unsigned long

The time, in seconds, spent to parse and insert data for the load or feed. This does not include the time spent fetching the list of source files.

totalRecords

Type: unsigned long

Total records loaded or attempted to load.

totalDuplicates

Type: unsigned long

The number of duplicate records encountered.

parsingErrors

Type: unsigned long

The number of parsing errors encountered.

datatypeMismatchErrors

Type: unsigned long

The number of records with a data type that did not match the given data.

insertErrors

Type: unsigned long

The number of records that were unable to be inserted due to errors.

Error Fields

The following are the fields in the error field in the status response.

startIndex

Type: unsigned long

The index of the first included error.

endIndex

Type: unsigned long

The index of the last included error.

loadId

Type: string

The ID for the load. You can use this ID to print the errors for the load by setting the errors parameter to true.

errorLogs

Type: list

A list of error logs.

Overall and Feed Status Descriptions

The following are possible responses returned in the status field from the loader with a description of the error.

LOAD_NOT_STARTED

Load has been recorded but not started.

LOAD_IN_PROGRESS

Load has started and is in progress.

LOAD_COMPLETED

Load has completed without any errors or errors within an acceptable threshold.

LOAD_CANCELLED_BY_USER

Load has been cancelled by user.

LOAD_CANCELLED_DUE_TO_ERRORS

Load has been cancelled by the system due to errors.

LOAD_UNEXPECTED_ERROR

Load failed with an unexpected error.

LOAD_FAILED

Load was rolled back because the error threshold was breached.

LOAD_S3_READ_ERROR

Feed failed due to intermittent or transient Amazon S3 connectivity issues. If any of the feeds receive this error, overall load status is set to LOAD_FAILED.

LOAD_S3_ACCESS_DENIED_ERROR

Access was denied to the S3 bucket. If any of the feeds receive this error, overall load status is set to LOAD_FAILED.

LOAD_COMMITTED_W_WRITE_CONFLICTS

Loaded data committed with unresolved write conflicts.

The loader will try to resolve the write conflicts in separate transactions and update the feed status as the load progresses. If the final feed status is LOAD_COMMITTED_W_WRITE_CONFLICTS, then try resuming the load and it will likely succeed without write conflicts. A write conflict is not usually related to bad input data, but duplicates in data can increase the likelihood of write conflicts.

LOAD_DATA_DEADLOCK

Load was automatically rolled back due to deadlock.

Examples

Example Request for Load Status

The following is a request sent via HTTP GET using the curl command.

curl -X GET 'http://your-neptune-endpoint:8182/loader/0a237328-afd5-4574-a0bc-c29ce5f54802'

Example Response

{ "status" : "200 OK", "payload" : { "feedCount" : [ { "LOAD_FAILED" : 1 } ], "overallStatus" : { "datatypeMismatchErrors" : 0, "fullUri" : "s3://bucket/key", "insertErrors" : 0, "parsingErrors" : 5, "retryNumber" : 0, "runNumber" : 1, "status" : "LOAD_FAILED", "totalDuplicates" : 0, "totalRecords" : 5, "totalTimeSpent" : 3.0 } } }

Example Request for Load Ids

The following is a request sent via HTTP GET using the curl command.

curl -X GET 'http://your-neptune-endpoint:8182/loader?limit=3'

Example Response

{ "status" : "200 OK", "payload" : { "loadIds" : [ "a2c0ce44-a44b-4517-8cd4-1dc144a8e5b5", "09683a01-6f37-4774-bb1b-5620d87f1931", "58085eb8-ceb4-4029-a3dc-3840969826b9" ] } }

Example Request for Detailed Load Status

The following is a request sent via HTTP GET using the curl command.

curl -X GET 'http://your-neptune-endpoint:8182/loader/0a237328-afd5-4574-a0bc-c29ce5f54802?details=true'

Example Response

{ "status" : "200 OK", "payload" : { "failedFeeds" : [ { "datatypeMismatchErrors" : 0, "fullUri" : "s3://bucket/key", "insertErrors" : 0, "parsingErrors" : 5, "retryNumber" : 0, "runNumber" : 1, "status" : "LOAD_FAILED", "totalDuplicates" : 0, "totalRecords" : 5, "totalTimeSpent" : 3.0 } ], "feedCount" : [ { "LOAD_FAILED" : 1 } ], "overallStatus" : { "datatypeMismatchErrors" : 0, "fullUri" : "s3://bucket/key", "insertErrors" : 0, "parsingErrors" : 5, "retryNumber" : 0, "runNumber" : 1, "status" : "LOAD_FAILED", "totalDuplicates" : 0, "totalRecords" : 5, "totalTimeSpent" : 3.0 } } }

Example Request for Detailed Status with Load Errors

The following is a request sent via HTTP GET using the curl command.

curl -X GET 'http://your-neptune-endpoint:8182/loader/0a237328-afd5-4574-a0bc-c29ce5f54802?details=true&errors=true&page=1&errorsPerPage=3'

Example Response

{ "status" : "200 OK", "payload" : { "failedFeeds" : [ { "datatypeMismatchErrors" : 0, "fullUri" : "s3://bucket/key", "insertErrors" : 0, "parsingErrors" : 5, "retryNumber" : 0, "runNumber" : 1, "status" : "LOAD_FAILED", "totalDuplicates" : 0, "totalRecords" : 5, "totalTimeSpent" : 3.0 } ], "feedCount" : [ { "LOAD_FAILED" : 1 } ], "overallStatus" : { "datatypeMismatchErrors" : 0, "fullUri" : "s3://bucket/key", "insertErrors" : 0, "parsingErrors" : 5, "retryNumber" : 0, "runNumber" : 1, "status" : "LOAD_FAILED", "totalDuplicates" : 0, "totalRecords" : 5, "totalTimeSpent" : 3.0 }, "errors" : { "endIndex" : 3, "errorLogs" : [ { "errorCode" : "UNKNOWN_ERROR", "errorMessage" : "Expected '<', found: |", "fileName" : "s3://bucket/key", "recordNum" : 1 }, { "errorCode" : "UNKNOWN_ERROR", "errorMessage" : "Expected '<', found: |", "fileName" : "s3://bucket/key", "recordNum" : 2 }, { "errorCode" : "UNKNOWN_ERROR", "errorMessage" : "Expected '<', found: |", "fileName" : "s3://bucket/key", "recordNum" : 3 } ], "loadId" : "0a237328-afd5-4574-a0bc-c29ce5f54802", "startIndex" : 1 } } }