適用於視窗設定範例的 Kinesis 代理程式 - Microsoft Windows 的 Amazon Kinesis Kinesis 代理程式

本文為英文版的機器翻譯版本,如內容有任何歧義或不一致之處,概以英文版為準。

適用於視窗設定範例的 Kinesis 代理程式

所以此appsettings.json組態檔案是一種 JSON 文件 Amazon Kinesis 可控制如何收集日誌、事件和指標。它也可以控制 Windows 專用 Kinesis Agent 如何轉換這些資料並將其串流到各種 AWS 服務。如需組態檔案中來源、目的地和管道宣告的詳細資訊,請參閱來源宣告目的地宣告以及管道宣告

下列各節包含各種不同類型案例的組態檔案範例。

從各種來源串流到 Kinesis Data Streams

以下為範例示範:appsettings.json組態檔案示範如何將日誌和事件從各種來源串流到 Kinesis Data Streams,以及從 Windows 效能計數器串流到 Amazon CloudWatch 指標。

DirectorySourceSysLog 記錄剖析器

以下檔案會將 syslog 格式日誌記錄從所有檔案串流到,其中包含.log副檔名為C:\LogSource\目錄中的SyslogKinesisDataStreamKinesis Data Streams 會將 us-east-1 區域中的 Kinesis Data stream 串流。您可以建立書籤,以確保即使代理程式關閉並於稍後重新啟動時,仍會傳送日誌檔的所有資料。自訂應用程式可以讀取及處理來自 SyslogKinesisDataStream 串流的記錄。

{ "Sources": [ { "Id": "SyslogDirectorySource", "SourceType": "DirectorySource", "Directory": "C:\\LogSource\\", "FileNameFilter": "*.log", "RecordParser": "SysLog", "TimeZoneKind": "UTC", "InitialPosition": "Bookmark" } ], "Sinks": [ { "Id": "KinesisStreamSink", "SinkType": "KinesisStream", "StreamName": "SyslogKinesisDataStream", "Region": "us-east-1" } ], "Pipes": [ { "Id": "SyslogDS2KSSink", "SourceRef": "SyslogDirectorySource", "SinkRef": "KinesisStreamSink" } ] }

DirectorySourceSingleLineJson 記錄剖析器

以下檔案會將 JSON 格式日誌記錄從具有.log副檔名為C:\LogSource\目錄中的JsonKinesisDataStreamKinesis Data Streams 會將 us-east-1 區域中的 Kinesis Data stream 串流。串流之前,系統會將 ComputerNameDT 索引鍵的鍵/值對新增到每個 JSON 物件,包括電腦名稱和處理記錄的日期與時間值。自訂應用程式可以讀取及處理來自 JsonKinesisDataStream 串流的記錄。

{ "Sources": [ { "Id": "JsonLogSource", "SourceType": "DirectorySource", "RecordParser": "SingleLineJson", "Directory": "C:\\LogSource\\", "FileNameFilter": "*.log", "InitialPosition": 0 } ], "Sinks": [ { "Id": "KinesisStreamSink", "SinkType": "KinesisStream", "StreamName": "JsonKinesisDataStream", "Region": "us-east-1", "Format": "json", "ObjectDecoration": "ComputerName={ComputerName};DT={timestamp:yyyy-MM-dd HH:mm:ss}" } ], "Pipes": [ { "Id": "JsonLogSourceToKinesisStreamSink", "SourceRef": "JsonLogSource", "SinkRef": "KinesisStreamSink" } ] }

ExchangeLogSource

以下檔案會將 Microsoft Exchange 產生的日誌記錄和存放在具有.log擴充功能C:\temp\ExchangeLog\目錄中的ExchangeKinesisDataStream在 us-east-1 區域中的 Kinesis Data stream,以 JSON 格式串流。雖然 Exchange 日誌不是 JSON 格式,但 Windows 適用 Kinesis 代理程式可以剖析這些日誌並轉換為 JSON。串流之前,系統會將 ComputerNameDT 索引鍵的鍵/值對新增到每個 JSON 物件,其中包含電腦名稱和處理記錄的日期與時間值。自訂應用程式可以讀取及處理來自 ExchangeKinesisDataStream 串流的記錄。

{ "Sources": [ { "Id": "ExchangeSource", "SourceType": "ExchangeLogSource", "Directory": "C:\\temp\\ExchangeLog\", "FileNameFilter": "*.log" } ], "Sinks": [ { "Id": "KinesisStreamSink", "SinkType": "KinesisStream", "StreamName": "ExchangeKinesisDataStream", "Region": "us-east-1", "Format": "json", "ObjectDecoration": "ComputerName={ComputerName};DT={timestamp:yyyy-MM-dd HH:mm:ss}" } ], "Pipes": [ { "Id": "ExchangeSourceToKinesisStreamSink", "SourceRef": "ExchangeSource", "SinkRef": "KinesisStreamSink" } ] }

W3SVCLogSource

以下檔案會將 Internet Information Services (IIS) (存放在這些檔案的標準位置中) 串流到IISKinesisDataStreamKinesis Data Streams 會將 us-east-1 區域中的 Kinesis Data stream 串流。自訂應用程式可以讀取及處理來自 IISKinesisDataStream 串流的記錄。IIS 是一種 Windows web 伺服器。

{ "Sources": [ { "Id": "IISLogSource", "SourceType": "W3SVCLogSource", "Directory": "C:\\inetpub\\logs\\LogFiles\\W3SVC1", "FileNameFilter": "*.log" } ], "Sinks": [ { "Id": "KinesisStreamSink", "SinkType": "KinesisStream", "StreamName": "IISKinesisDataStream", "Region": "us-east-1" } ], "Pipes": [ { "Id": "IISLogSourceToKinesisStreamSink", "SourceRef": "IISLogSource", "SinkRef": "KinesisStreamSink" } ] }

WindowsEventLogSource 與查詢

下列檔案會從 Windows 系統事件記錄檔資料流記錄事件,其層級為CriticalError(小於或等於 2) 串流到SystemKinesisDataStream在 us-east-1 區域中的 Kinesis Data stream,以 JSON 格式串流。自訂應用程式可以讀取及處理來自 SystemKinesisDataStream 串流的記錄。

{ "Sources": [ { "Id": "SystemLogSource", "SourceType": "WindowsEventLogSource", "LogName": "System", "Query": "*[System/Level<=2]" } ], "Sinks": [ { "Id": "KinesisStreamSink", "SinkType": "KinesisStream", "StreamName": "SystemKinesisDataStream", "Region": "us-east-1", "Format": "json" } ], "Pipes": [ { "Id": "SLSourceToKSSink", "SourceRef": "SystemLogSource", "SinkRef": "KinesisStreamSink" } ] }

WindowsETWEventSource

以下檔案會將 Microsoft 通用語言執行平台 (CLR) 的例外狀況和安全事件串流到ClrKinesisDataStream在 us-east-1 區域中的 Kinesis Data stream,以 JSON 格式串流。自訂應用程式可以讀取及處理來自 ClrKinesisDataStream 串流的記錄。

{ "Sources": [ { "Id": "ClrETWEventSource", "SourceType": "WindowsETWEventSource", "ProviderName": "Microsoft-Windows-DotNETRuntime", "TraceLevel": "Verbose", "MatchAnyKeyword": "0x00008000, 0x00000400" } ], "Sinks": [ { "Id": "KinesisStreamSink", "SinkType": "KinesisStream", "StreamName": "ClrKinesisDataStream", "Region": "us-east-1", "Format": "json" } ], "Pipes": [ { "Id": "ETWSourceToKSSink", "SourceRef": "ClrETWEventSource", "SinkRef": "KinesisStreamSink" } ] }

WindowsPerformanceCounterSource

以下檔案會 CloudWatch 開啟檔案總數、重新啟動後嘗試登入總數、磁碟每秒讀取數,以及可用磁碟空間百分比的效能計數器串流至 us-east-1 區域中的區域。您可以在 CloudWatch 中繪製這些指標的圖表、從圖表建置儀表板,以及設定警示以在超過閾值時傳送通知。

{ "Sources": [ { "Id": "PerformanceCounter", "SourceType": "WindowsPerformanceCounterSource", "Categories": [ { "Category": "Server", "Counters": [ "Files Open", "Logon Total" ] }, { "Category": "LogicalDisk", "Instances": "*", "Counters": [ "% Free Space", { "Counter": "Disk Reads/sec", "Unit": "Count/Second" } ] } ], } ], "Sinks": [ { "Namespace": "MyServiceMetrics", "Region": "us-east-1", "Id": "CloudWatchSink", "SinkType": "CloudWatch" } ], "Pipes": [ { "Id": "PerformanceCounterToCloudWatch", "SourceRef": "PerformanceCounter", "SinkRef": "CloudWatchSink" } ] }

從 Windows 應用程式事件日誌串流到目的地

以下為範例示範:appsettings.json組態檔案示範將 Windows 應用程式事件日誌串流到適用於微軟視窗的 Amazon Kinesis 代理程式中的各種目的地。如需使用 KinesisStreamCloudWatch 目的地類型的範例,請參閱從各種來源串流到 Kinesis Data Streams

KinesisFirehose

下列檔案串流CriticalErrorWindows 應用程式記錄檔事件到WindowsLogFirehoseDeliveryStreamKinesis Data Firehose 交付串流位於 us-east-1 區域中。如果與 Kinesis Data Firehose 的連線中斷,系統會先將事件排入記憶體佇列。若有必要,系統會接著將它們排入磁碟檔案上的佇列,直到恢復連線。然後,事件即可解除佇列狀態,並後接任何新事件一起傳送。

您可以根據資料管道要求,設定 Kinesis Data Firehose,將串流資料存放到多種不同類型的儲存體與分析服務。

{ "Sources": [ { "Id": "ApplicationLogSource", "SourceType": "WindowsEventLogSource", "LogName": "Application", "Query": "*[System/Level<=2]" } ], "Sinks": [ { "Id": "WindowsLogKinesisFirehoseSink", "SinkType": "KinesisFirehose", "StreamName": "WindowsLogFirehoseDeliveryStream", "Region": "us-east-1", "QueueType": "file" } ], "Pipes": [ { "Id": "ALSource2ALKFSink", "SourceRef": "ApplicationLogSource", "SinkRef": "WindowsLogKinesisFirehoseSink" } ] }

CloudWatchLogs

下列檔案串流CriticalErrorWindows 應用程式 CloudWatch Logs 事件串流到MyServiceApplicationLog-Group日誌群組。每個串流名稱開頭為 Stream-。結尾為串流建立時的四位數年份、二位數月份和二位數日期,全部串連在一起 (例如,Stream-20180501 是 2018 年 5 月 1 日建立的串流)。

{ "Sources": [ { "Id": "ApplicationLogSource", "SourceType": "WindowsEventLogSource", "LogName": "Application", "Query": "*[System/Level<=2]" } ], "Sinks": [ { "Id": "CloudWatchLogsSink", "SinkType": "CloudWatchLogs", "LogGroup": "MyServiceApplicationLog-Group", "LogStream": "Stream-{timestamp:yyyyMMdd}", "Region": "us-east-1", "Format": "json" } ], "Pipes": [ { "Id": "ALSource2CWLSink", "SourceRef": "ApplicationLogSource", "SinkRef": "CloudWatchLogsSink" } ] }

使用管道

以下範例 appsettings.json 組態檔案示範如何使用管道相關的功能。

此範例會將日誌項目從c:\LogSource\ApplicationLogFirehoseDeliveryStreamKinesis Data Firehose 交付串流。它只包含符合 FilterPattern 鍵/值對所指定規則表達式的字行。具體來說,日誌檔中只有以1011會將其串流到 Kinesis Data Firehose。

{ "Sources": [ { "Id": "ApplicationLogSource", "SourceType": "DirectorySource", "Directory": "C:\\LogSource\\", "FileNameFilter": "*.log", "RecordParser": "SingleLine" } ], "Sinks": [ { "Id": "ApplicationLogKinesisFirehoseSink", "SinkType": "KinesisFirehose", "StreamName": "ApplicationLogFirehoseDeliveryStream", "Region": "us-east-1" } ], "Pipes": [ { "Id": "ALSourceToALKFSink", "Type": "RegexFilterPipe", "SourceRef": "ApplicationLogSource", "SinkRef": "ApplicationLogKinesisFirehoseSink", "FilterPattern": "^(10|11),.*" } ] }

使用多個來源和管道

以下範例 appsettings.json 組態檔案示範如何使用多個來源和管道。

此範例會將應用程式、安全性和系統 Windows 事件日誌串流到EventLogStreamKinesis Data Firehose 交付串流使用三個來源、三個管道和單一目的地。

{ "Sources": [ { "Id": "ApplicationLog", "SourceType": "WindowsEventLogSource", "LogName": "Application" }, { "Id": "SecurityLog", "SourceType": "WindowsEventLogSource", "LogName": "Security" }, { "Id": "SystemLog", "SourceType": "WindowsEventLogSource", "LogName": "System" } ], "Sinks": [ { "Id": "EventLogSink", "SinkType": "KinesisFirehose", "StreamName": "EventLogStream", "Format": "json" }, ], "Pipes": [ { "Id": "ApplicationLogToFirehose", "SourceRef": "ApplicationLog", "SinkRef": "EventLogSink" }, { "Id": "SecurityLogToFirehose", "SourceRef": "SecurityLog", "SinkRef": "EventLogSink" }, { "Id": "SystemLogToFirehose", "SourceRef": "SystemLog", "SinkRef": "EventLogSink" } ] }