Interface IContinuousLoggingProps

(experimental) Properties for enabling Continuous Logging for Glue Jobs.

Namespace: Amazon.CDK.AWS.Glue.Alpha

Assembly: Amazon.CDK.AWS.Glue.Alpha.dll

Syntax (csharp)

public interface IContinuousLoggingProps

Syntax (vb)

Public Interface IContinuousLoggingProps

Remarks

Stability: Experimental

See: https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html

ExampleMetadata: infused

Examples

using Amazon.CDK;
             using Amazon.CDK.AWS.IAM;
             Stack stack;
             IRole role;
             Code script;

             new PySparkEtlJob(stack, "PySparkETLJob", new PySparkEtlJobProps {
                 JobName = "PySparkETLJobCustomName",
                 Description = "This is a description",
                 Role = role,
                 Script = script,
                 GlueVersion = GlueVersion.V3_0,
                 ContinuousLogging = new ContinuousLoggingProps { Enabled = false },
                 WorkerType = WorkerType.G_2X,
                 MaxConcurrentRuns = 100,
                 Timeout = Duration.Hours(2),
                 Connections = new [] { Connection.FromConnectionName(stack, "Connection", "connectionName") },
                 SecurityConfiguration = SecurityConfiguration.FromSecurityConfigurationName(stack, "SecurityConfig", "securityConfigName"),
                 Tags = new Dictionary<string, string> {
                     { "FirstTagName", "FirstTagValue" },
                     { "SecondTagName", "SecondTagValue" },
                     { "XTagName", "XTagValue" }
                 },
                 NumberOfWorkers = 2,
                 MaxRetries = 2
             });

Synopsis

Properties

ConversionPattern	(experimental) Apply the provided conversion pattern.
Enabled	(experimental) Enable continuous logging.
LogGroup	(experimental) Specify a custom CloudWatch log group name.
LogStreamPrefix	(experimental) Specify a custom CloudWatch log stream prefix.
Quiet	(experimental) Filter out non-useful Apache Spark driver/executor and Apache Hadoop YARN heartbeat log messages.

Properties

ConversionPattern

(experimental) Apply the provided conversion pattern.

string? ConversionPattern { get; }

Property Value

string

Remarks

This is a Log4j Conversion Pattern to customize driver and executor logs.

Default: %d{yy/MM/dd HH:mm:ss} %p %c{1}: %m%n

Stability: Experimental

Enabled

(experimental) Enable continuous logging.

bool Enabled { get; }

Property Value

bool

Remarks

Stability: Experimental

LogGroup

(experimental) Specify a custom CloudWatch log group name.

ILogGroup? LogGroup { get; }

Property Value

ILogGroup

Remarks

Default: - a log group is created with name /aws-glue/jobs/logs-v2/.

Stability: Experimental

LogStreamPrefix

(experimental) Specify a custom CloudWatch log stream prefix.

string? LogStreamPrefix { get; }

Property Value

string

Remarks

Default: - the job run ID.

Stability: Experimental

Quiet

(experimental) Filter out non-useful Apache Spark driver/executor and Apache Hadoop YARN heartbeat log messages.

bool? Quiet { get; }

Property Value

bool?

Remarks

Default: true

Stability: Experimental