Amazon Neptune
User Guide (API Version 2017-11-29)

Neptune Loader Get-Status API

Gets the status of a loader job.

To get load status, you must send an HTTP GET request to the http://your-neptune-endpoint:8182/loader endpoint. To get the status for a particular load request, you must include the loadId as a URL parameter, or the loadId can be appended to the URL path.

The possible responses returned for each feed from the loader, with a description of the error, are in the Neptune Loader Error and Feed Messages section of the Neptune Service Errors topic.

Neptune Loader Get-Status Request Syntax

GET http://your-neptune-endpoint:8182/loader?loadId=loadId
GET http://your-neptune-endpoint:8182/loader/loadId
GET http://your-neptune-endpoint:8182/loader

Neptune Loader Get-Status Request Parameters

loadId

The ID of the load job. If you do not specify a loadId, a list of load IDs is returned.

details

Include details beyond overall status. Default: False

Allowed values: TRUE, FALSE

errors

Include the list of errors. The list of errors is paged. The page and errorsPerPage parameters allow you to page through all the errors. Default: False

Allowed values: TRUE, FALSE

page

The error page number. Only valid with the errors parameter set to TRUE. Default: 1

Allowed values: Positive integers

errorsPerPage

The number of errors per each page. Only valid with the errors parameter set to TRUE. Default: 10

Allowed values: Positive integers

limit

The number of load ids to list. Only valid when requesting a list of load IDs by sending a GET request with no loadId specified. Default: 100

Allowed values: Positive integers, 1 - 100

Response Syntax

{ "status" : "200 OK", "payload" : { "feedCount" : [ { "LOAD_FAILED" : number } ], "overallStatus" : { "fullUri" : "s3://bucket/key", "runNumber" : number, "retryNumber" : number, "status" : "string", "totalTimeSpent" : number, "totalRecords" : number, "totalDuplicates" : number, "parsingErrors" : number, "datatypeMismatchErrors" : number, "insertErrors" : number, }, "failedFeeds" : [ { "fullUri" : "s3://bucket/key", "runNumber" : number, "retryNumber" : number, "status" : "string", "totalTimeSpent" : number, "totalRecords" : number, "totalDuplicates" : number, "parsingErrors" : number, "datatypeMismatchErrors" : number, "insertErrors" : number, } ], "errors" : { "startIndex" : number, "endIndex" : number, "loadId" : "string, "errorLogs" : [ ] } } } }

200 OK

Successful status check returns a 200 code.

Neptune Loader Get-Status Parameters Errors

When an error occurs, a JSON object is returned in the BODY of the response. The message object contains a description of the error.

Error 400

An invalid loadId returns a 400 bad request error. The message describes the error.

Error 500

A valid request that cannot be processed returns a 500 internal server error. The message describes the error.

Neptune Loader Get-Status Response Fields

Response Fields

The following are the fields returned by the status command with descriptions.

These fields appear in the overallStatus field for all loads, and the failedFeeds for each failed feed.

fullUri

Type: string

The URI of the file or files to be loaded. In the format: s3://bucket/key

runNumber

Type: unsigned long

The run number of this load or feed. This is incremented when the load is restarted.

retryNumber

Type: unsigned long

The retry number of this load or feed. This is incremented when the loader retries a feed or load automatically.

status

Type: string

The returned status of the load or feed. LOAD_COMPLETED indicates a successful load with no problems. For a list of all load status values, see Overall and Feed Status Descriptions.

totalTimeSpent

Type: unsigned long

The time, in seconds, spent to parse and insert data for the load or feed. This does not include the time spent fetching the list of source files.

totalRecords

Type: unsigned long

Total records loaded or attempted to load.

totalDuplicates

Type: unsigned long

The number of duplicate records encountered.

parsingErrors

Type: unsigned long

The number of parsing errors encountered.

datatypeMismatchErrors

Type: unsigned long

The number of records with a data type that did not match the given data.

insertErrors

Type: unsigned long

The number of records that were unable to be inserted due to errors.

Neptune Loader Get-Status Error Fields

The following are the fields in the error field in the status response.

startIndex

Type: unsigned long

The index of the first included error.

endIndex

Type: unsigned long

The index of the last included error.

loadId

Type: string

The ID for the load. You can use this ID to print the errors for the load by setting the errors parameter to true.

errorLogs

Type: list

A list of error logs.

Overall and Feed Status Descriptions

The possible responses returned for each feed from the loader, with a description of the error, are in the Neptune Loader Error and Feed Messages section of the Neptune Service Errors topic.

Load-Status Examples

Example Request for Load Status

The following is a request sent via HTTP GET using the curl command.

curl -X GET 'http://your-neptune-endpoint:8182/loader/0a237328-afd5-4574-a0bc-c29ce5f54802'

Example Response

{ "status" : "200 OK", "payload" : { "feedCount" : [ { "LOAD_FAILED" : 1 } ], "overallStatus" : { "datatypeMismatchErrors" : 0, "fullUri" : "s3://bucket/key", "insertErrors" : 0, "parsingErrors" : 5, "retryNumber" : 0, "runNumber" : 1, "status" : "LOAD_FAILED", "totalDuplicates" : 0, "totalRecords" : 5, "totalTimeSpent" : 3.0 } } }

Example Request for Load Ids

The following is a request sent via HTTP GET using the curl command.

curl -X GET 'http://your-neptune-endpoint:8182/loader?limit=3'

Example Response

{ "status" : "200 OK", "payload" : { "loadIds" : [ "a2c0ce44-a44b-4517-8cd4-1dc144a8e5b5", "09683a01-6f37-4774-bb1b-5620d87f1931", "58085eb8-ceb4-4029-a3dc-3840969826b9" ] } }

Example Request for Detailed Load Status

The following is a request sent via HTTP GET using the curl command.

curl -X GET 'http://your-neptune-endpoint:8182/loader/0a237328-afd5-4574-a0bc-c29ce5f54802?details=true'

Example Response

{ "status" : "200 OK", "payload" : { "failedFeeds" : [ { "datatypeMismatchErrors" : 0, "fullUri" : "s3://bucket/key", "insertErrors" : 0, "parsingErrors" : 5, "retryNumber" : 0, "runNumber" : 1, "status" : "LOAD_FAILED", "totalDuplicates" : 0, "totalRecords" : 5, "totalTimeSpent" : 3.0 } ], "feedCount" : [ { "LOAD_FAILED" : 1 } ], "overallStatus" : { "datatypeMismatchErrors" : 0, "fullUri" : "s3://bucket/key", "insertErrors" : 0, "parsingErrors" : 5, "retryNumber" : 0, "runNumber" : 1, "status" : "LOAD_FAILED", "totalDuplicates" : 0, "totalRecords" : 5, "totalTimeSpent" : 3.0 } } }

Example Request for Detailed Status with Load Errors

The following is a request sent via HTTP GET using the curl command.

curl -X GET 'http://your-neptune-endpoint:8182/loader/0a237328-afd5-4574-a0bc-c29ce5f54802?details=true&errors=true&page=1&errorsPerPage=3'

Example Response

{ "status" : "200 OK", "payload" : { "failedFeeds" : [ { "datatypeMismatchErrors" : 0, "fullUri" : "s3://bucket/key", "insertErrors" : 0, "parsingErrors" : 5, "retryNumber" : 0, "runNumber" : 1, "status" : "LOAD_FAILED", "totalDuplicates" : 0, "totalRecords" : 5, "totalTimeSpent" : 3.0 } ], "feedCount" : [ { "LOAD_FAILED" : 1 } ], "overallStatus" : { "datatypeMismatchErrors" : 0, "fullUri" : "s3://bucket/key", "insertErrors" : 0, "parsingErrors" : 5, "retryNumber" : 0, "runNumber" : 1, "status" : "LOAD_FAILED", "totalDuplicates" : 0, "totalRecords" : 5, "totalTimeSpent" : 3.0 }, "errors" : { "endIndex" : 3, "errorLogs" : [ { "errorCode" : "UNKNOWN_ERROR", "errorMessage" : "Expected '<', found: |", "fileName" : "s3://bucket/key", "recordNum" : 1 }, { "errorCode" : "UNKNOWN_ERROR", "errorMessage" : "Expected '<', found: |", "fileName" : "s3://bucket/key", "recordNum" : 2 }, { "errorCode" : "UNKNOWN_ERROR", "errorMessage" : "Expected '<', found: |", "fileName" : "s3://bucket/key", "recordNum" : 3 } ], "loadId" : "0a237328-afd5-4574-a0bc-c29ce5f54802", "startIndex" : 1 } } }