Amazon Neptune
User Guide (API Version 2017-11-29)

The AWS Documentation website is getting a new look!
Try it now and let us know what you think. Switch to the new look >>

You can return to the original look by selecting English in the language selector above.

Neptune Loader Get-Status API

Gets the status of a loader job.

To get load status, you must send an HTTP GET request to the https://your-neptune-endpoint:port/loader endpoint. To get the status for a particular load request, you must include the loadId as a URL parameter, or append the loadId to the URL path.

For the possible responses returned for each feed from the loader along with a description of the error, see the Neptune Loader Error and Feed Messages section in Neptune Service Errors.

Neptune Loader Get-Status Request Syntax

GET https://your-neptune-endpoint:port/loader?loadId=loadId
GET https://your-neptune-endpoint:port/loader/loadId
GET https://your-neptune-endpoint:port/loader

Neptune Loader Get-Status Request Parameters

loadId

The ID of the load job. If you do not specify a loadId, a list of load IDs is returned.

details

Include details beyond overall status. Default: False

Allowed values: TRUE, FALSE

errors

Include the list of errors. The list of errors is paged. The page and errorsPerPage parameters allow you to page through all the errors. Default: False

Allowed values: TRUE, FALSE

page

The error page number. Only valid with the errors parameter set to TRUE. Default: 1

Allowed values: Positive integers

errorsPerPage

The number of errors per each page. Only valid with the errors parameter set to TRUE. Default: 10

Allowed values: Positive integers

limit

The number of load ids to list. Only valid when requesting a list of load IDs by sending a GET request with no loadId specified. Default: 100

Allowed values: Positive integers, 1 - 100

Response Syntax

{ "status" : "200 OK", "payload" : { "feedCount" : [ { "LOAD_FAILED" : number } ], "overallStatus" : { "fullUri" : "s3://bucket/key", "runNumber" : number, "retryNumber" : number, "status" : "string", "totalTimeSpent" : number, "startTime" : number, "totalRecords" : number, "totalDuplicates" : number, "parsingErrors" : number, "datatypeMismatchErrors" : number, "insertErrors" : number, }, "failedFeeds" : [ { "fullUri" : "s3://bucket/key", "runNumber" : number, "retryNumber" : number, "status" : "string", "totalTimeSpent" : number, "startTime" : number, "totalRecords" : number, "totalDuplicates" : number, "parsingErrors" : number, "datatypeMismatchErrors" : number, "insertErrors" : number, } ], "errors" : { "startIndex" : number, "endIndex" : number, "loadId" : "string, "errorLogs" : [ ] } } } }

200 OK

Successful status check returns a 200 code.

startTime

This is a POSIX time epoch value, namely the number of seconds elapsed since 00:00 on 1 January 1970 in Coordinated Universal Time (UTC).

Neptune Loader Get-Status Parameters Errors

When an error occurs, a JSON object is returned in the BODY of the response. The message object contains a description of the error.

Error 400

An invalid loadId returns a 400 bad request error. The message describes the error.

Error 500

A valid request that cannot be processed returns a 500 internal server error. The message describes the error.

Neptune Loader Get-Status Response Fields

Response Fields

The following are the fields returned by the status command with descriptions.

These fields appear in the overallStatus field for all loads, and the failedFeeds for each failed feed.

fullUri

Type: string

The URI of the file or files to be loaded. In the format: s3://bucket/key

runNumber

Type: unsigned long

The run number of this load or feed. This is incremented when the load is restarted.

retryNumber

Type: unsigned long

The retry number of this load or feed. This is incremented when the loader retries a feed or load automatically.

status

Type: string

The returned status of the load or feed. LOAD_COMPLETED indicates a successful load with no problems. For a list of all load status values, see Overall and Feed Status Descriptions.

totalTimeSpent

Type: unsigned long

The time, in seconds, spent to parse and insert data for the load or feed. This does not include the time spent fetching the list of source files.

totalRecords

Type: unsigned long

Total records loaded or attempted to load.

totalDuplicates

Type: unsigned long

The number of duplicate records encountered.

parsingErrors

Type: unsigned long

The number of parsing errors encountered.

datatypeMismatchErrors

Type: unsigned long

The number of records with a data type that did not match the given data.

insertErrors

Type: unsigned long

The number of records that were unable to be inserted due to errors.

Neptune Loader Get-Status Error Fields

The following are the fields in the error field in the status response.

startIndex

Type: unsigned long

The index of the first included error.

endIndex

Type: unsigned long

The index of the last included error.

loadId

Type: string

The ID for the load. You can use this ID to print the errors for the load by setting the errors parameter to true.

errorLogs

Type: list

A list of error logs.

Overall and Feed Status Descriptions

The possible responses returned for each feed from the loader, with a description of the error, are in the Neptune Loader Error and Feed Messages section of the Neptune Service Errors topic.

Load-Status Examples

Example Request for Load Status

The following is a request sent via HTTP GET using the curl command.

curl -X GET 'https://your-neptune-endpoint:port/loader/0a237328-afd5-4574-a0bc-c29ce5f54802'

Example Response

{ "status" : "200 OK", "payload" : { "feedCount" : [ { "LOAD_FAILED" : 1 } ], "overallStatus" : { "datatypeMismatchErrors" : 0, "fullUri" : "s3://bucket/key", "insertErrors" : 0, "parsingErrors" : 5, "retryNumber" : 0, "runNumber" : 1, "status" : "LOAD_FAILED", "totalDuplicates" : 0, "totalRecords" : 5, "totalTimeSpent" : 3.0 } } }

Example Request for Load Ids

The following is a request sent via HTTP GET using the curl command.

curl -X GET 'https://your-neptune-endpoint:port/loader?limit=3'

Example Response

{ "status" : "200 OK", "payload" : { "loadIds" : [ "a2c0ce44-a44b-4517-8cd4-1dc144a8e5b5", "09683a01-6f37-4774-bb1b-5620d87f1931", "58085eb8-ceb4-4029-a3dc-3840969826b9" ] } }

Example Request for Detailed Load Status

The following is a request sent via HTTP GET using the curl command.

curl -X GET 'https://your-neptune-endpoint:port/loader/0a237328-afd5-4574-a0bc-c29ce5f54802?details=true'

Example Response

{ "status" : "200 OK", "payload" : { "failedFeeds" : [ { "datatypeMismatchErrors" : 0, "fullUri" : "s3://bucket/key", "insertErrors" : 0, "parsingErrors" : 5, "retryNumber" : 0, "runNumber" : 1, "status" : "LOAD_FAILED", "totalDuplicates" : 0, "totalRecords" : 5, "totalTimeSpent" : 3.0 } ], "feedCount" : [ { "LOAD_FAILED" : 1 } ], "overallStatus" : { "datatypeMismatchErrors" : 0, "fullUri" : "s3://bucket/key", "insertErrors" : 0, "parsingErrors" : 5, "retryNumber" : 0, "runNumber" : 1, "status" : "LOAD_FAILED", "totalDuplicates" : 0, "totalRecords" : 5, "totalTimeSpent" : 3.0 } } }

Example Request for Detailed Status with Load Errors

The following is a request sent via HTTP GET using the curl command.

curl -X GET 'https://your-neptune-endpoint:port/loader/0a237328-afd5-4574-a0bc-c29ce5f54802?details=true&errors=true&page=1&errorsPerPage=3'

Example Response

{ "status" : "200 OK", "payload" : { "failedFeeds" : [ { "datatypeMismatchErrors" : 0, "fullUri" : "s3://bucket/key", "insertErrors" : 0, "parsingErrors" : 5, "retryNumber" : 0, "runNumber" : 1, "status" : "LOAD_FAILED", "totalDuplicates" : 0, "totalRecords" : 5, "totalTimeSpent" : 3.0 } ], "feedCount" : [ { "LOAD_FAILED" : 1 } ], "overallStatus" : { "datatypeMismatchErrors" : 0, "fullUri" : "s3://bucket/key", "insertErrors" : 0, "parsingErrors" : 5, "retryNumber" : 0, "runNumber" : 1, "status" : "LOAD_FAILED", "totalDuplicates" : 0, "totalRecords" : 5, "totalTimeSpent" : 3.0 }, "errors" : { "endIndex" : 3, "errorLogs" : [ { "errorCode" : "UNKNOWN_ERROR", "errorMessage" : "Expected '<', found: |", "fileName" : "s3://bucket/key", "recordNum" : 1 }, { "errorCode" : "UNKNOWN_ERROR", "errorMessage" : "Expected '<', found: |", "fileName" : "s3://bucket/key", "recordNum" : 2 }, { "errorCode" : "UNKNOWN_ERROR", "errorMessage" : "Expected '<', found: |", "fileName" : "s3://bucket/key", "recordNum" : 3 } ], "loadId" : "0a237328-afd5-4574-a0bc-c29ce5f54802", "startIndex" : 1 } } }

Data Prefetch Task Interrupted Error

Occasionally when you get a LOAD_FAILED status and then request more detailed information, the error returned may be a PARSING_ERROR with a Data prefetch task interrupted message, like this:

"errorLogs" : [ { "errorCode" : "PARSING_ERROR", "errorMessage" : "Data prefetch task interrupted: Data prefetch task for 11467 failed", "fileName" : "s3://some-source-bucket/some-source-file", "recordNum" : 0 } ]

This error occurs when there was a temporary interruption in the data load process that was typically not caused by your request or your data. It can usually be resolved simply by running the bulk upload request again. If you are using default settings, namely "mode":"AUTO", and "failOnError":"TRUE", the loader skips the files that it already successfully loaded and resumes loading files it had not yet loaded when the interruption occurred.