Neptune Loader Get-Status Responses

Neptune Loader Get-Status Response JSON layout

The general layout of a loader status response is as follows:

{
    "status" : "200 OK",
    "payload" : {
        "feedCount" : [
            {
                "LOAD_FAILED" : number
            }
        ],
        "overallStatus" : {
            "fullUri" : "s3://bucket/key",
            "runNumber" : number,
            "retryNumber" : number,
            "status" : "string",
            "totalTimeSpent" : number,
            "startTime" : number,
            "totalRecords" : number,
            "totalDuplicates" : number,
            "parsingErrors" : number,
            "datatypeMismatchErrors" : number,
            "insertErrors" : number,
        },
        "failedFeeds" : [
            {
                "fullUri" : "s3://bucket/key",
                "runNumber" : number,
                "retryNumber" : number,
                "status" : "string",
                "totalTimeSpent" : number,
                "startTime" : number,
                "totalRecords" : number,
                "totalDuplicates" : number,
                "parsingErrors" : number,
                "datatypeMismatchErrors" : number,
                "insertErrors" : number,
            }
        ],
        "errors" : {
            "startIndex" : number,
            "endIndex" : number,
            "loadId" : "string,
            "errorLogs" : [ ]
        }
    }
}

Neptune Loader Get-Status overallStatus and failedFeeds response objects

The possible responses returned for each failed feed, including the error descriptions, are the same as for the overallStatus object in a Get-Status response.

The following fields appear in the overallStatus object for all loads, and the failedFeeds object for each failed feed:

• fullUri   –   The URI of the file or files to be loaded.

  Type: string

  Format: s3://bucket/key.

• runNumber   –   The run number of this load or feed. This is incremented when the load is restarted.

  Type: unsigned long.

• retryNumber   –   The retry number of this load or feed. This is incremented when the loader automatically retries a feed or load.

  Type: unsigned long.

• status   –   The returned status of the load or feed. LOAD_COMPLETED indicates a successful load with no problems. For a list of all load status values, see Request parameters.

  Type: string.

• totalTimeSpent   –   The time, in seconds, spent to parse and insert data for the load or feed. This does not include the time spent fetching the list of source files.

  Type: unsigned long.

• totalRecords   –   Total records loaded or attempted to load.

  Type: unsigned long.

• totalDuplicates   –   The number of duplicate records encountered.

  Type: unsigned long.

• parsingErrors   –   The number of parsing errors encountered.

  Type: unsigned long.

• datatypeMismatchErrors   –   The number of records with a data type that did not match the given data.

  Type: unsigned long.

• insertErrors   –   The number of records that could not be inserted due to errors.

  Type: unsigned long.

Neptune Loader Get-Status errors response object

Errors fall into the following categories:

• Error 400   –   An invalid loadId returns an HTTP 400 bad request error. The message describes the error.

• Error 500   –   A valid request that cannot be processed returns an HTTP 500 internal server error. The message describes the error.

See Neptune Loader Error and Feed Messages for a list of the error and feed messages returned by the loader in case of errors.

When an error occurs, a JSON errors object is returned in the BODY of the response, with the following fields:

• startIndex   –   The index of the first included error.

  Type: unsigned long.

• endIndex   –   The index of the last included error.

  Type: unsigned long.

• loadId   –   The ID of the load. You can use this ID to print the errors for the load by setting the errors parameter to TRUE.

  Type: string.

• errorLogs   –   A list of the errors.

  Type: list.

Neptune Loader Get-Status errorLogs response object

The errorLogs object under errors in the loader Get-Status response contains an object describing each error using the following fields:

• errorCode   –   Identifies the nature of error.

  It can take one of the following values:

  • PARSING_ERROR

  • S3_ACCESS_DENIED_ERROR

  • FROM_OR_TO_VERTEX_ARE_MISSING

  • ID_ASSIGNED_TO_MULTIPLE_EDGES

  • SINGLE_CARDINALITY_VIOLATION

  • FILE_MODIFICATION_OR_DELETION_ERROR

  • OUT_OF_MEMORY_ERROR

  • INTERNAL_ERROR (returned when the bulk loader cannot determine the type of the error).

• errorMessage   –   A message describing the error.

  This can be a generic message associated with the error code or a specific message containing details, for example about a missing from/to vertex or about a parsing error.

• fileName   –   The name of the feed.

• recordNum   –   In the case of a parsing error, this is the record number in the file of the record that could not be parsed. It is set to zero if the record number is not applicable to the error, or if it could not be determined.

For example, the bulk loader would generate a parsing error if it encountered a faulty row such as the following in an RDF nquads file:

<http://base#subject> |http://base#predicate> <http://base#true> .

As you can see, the second http in the row above should be preceded by  <  rather than  | . The resulting error object under errorLogs in a status response would look like this:

{
    "errorCode" : "PARSING_ERROR",
    "errorMessage" : "Expected '<', found: |",
    "fileName" : "s3://bucket/key",
    "recordNum" : 12345
},