抽象 DataSink 类 - AWS Glue

抽象 DataSink 类

程序包:com.amazonaws.services.glue

abstract class DataSink

DataSource 的写入器模拟。DataSink 封装可将 DynamicFrame 写入到的目标和格式。

Def writeDynamicFrame

def writeDynamicFrame( frame : DynamicFrame, callSite : CallSite = CallSite("Not provided", "") ) : DynamicFrame

Def pyWriteDynamicFrame

def pyWriteDynamicFrame( frame : DynamicFrame, site : String = "Not provided", info : String = "" )

Def writeDataFrame

def writeDataFrame(frame: DataFrame, glueContext: GlueContext, callSite: CallSite = CallSite("Not provided", "") ): DataFrame

Def pyWriteDataFrame

def pyWriteDataFrame(frame: DataFrame, glueContext: GlueContext, site: String = "Not provided", info: String = "" ): DataFrame

Def setCatalogInfo

def setCatalogInfo(catalogDatabase: String, catalogTableName : String, catalogId : String = "")

Def supportsFormat

def supportsFormat( format : String ) : Boolean

Def setFormat

def setFormat( format : String, options : JsonOptions ) : Unit

Def withFormat

def withFormat( format : String, options : JsonOptions = JsonOptions.empty ) : DataSink

Def setAccumulableSize

def setAccumulableSize( size : Int ) : Unit

Def getOutputErrorRecordsAccumulable

def getOutputErrorRecordsAccumulable : Accumulable[List[OutputError], OutputError]

Def errorsAsDynamicFrame

def errorsAsDynamicFrame : DynamicFrame

DataSink 对象

object DataSink

Def recordMetrics

def recordMetrics( frame : DynamicFrame, ctxt : String ) : DynamicFrame