AWS Cloudwatch - Windows Logs

AWS Cloudwatch - Windows Logs

References:

• http://docs.aws.amazon.com/AWSEC2/latest/WindowsGuide/send_logs_to_cwl_gs.html
• http://docs.aws.amazon.com/AWSEC2/latest/WindowsGuide/UsingConfig_Install.htm

Basic Steps to get it running (this was tested with EC2 service version 3.19.1153)

1. Update EC2Config to the latest version
2. Open EC2ConfigService Settings
3. Under General Tab, Enable CloudWatch Logs
4. Copy the sample JSON file to your EC2 install location's settings folder
1. Download Sample JSON File
2. c:\program files\amazon\ec2configservice\settings
5. Edit the JSON file 
1. "Id":"CloudWatchLogs" section should have your information. I leave AccessKey and SecretKey blank because I prefer to use IAM Role that has access to write to CloudWatch. Also, I prefer {hostname} to default, {instance_id} because hostname means something without cross referencing. 
2. "Id":"CloudWatch" section should have your region and NameSpace. NameSpace is the name that you give to your CustomMetrics.
6. Go to Services, restart "Ec2Config" service. 
7. You should see Application and System Event Logs in your CloudWatch Logs

Configuring Logs in AWS.EC2.Windows.CloudWatch.json file

Windows Logs

Fullname: 

AWS.EC2.Windows.CloudWatch.EventLog.EventLogInputComponent,

AWS.EC2.Windows.CloudWatch

Id: Update the Id to something unique.

Edit the LogName and Levels to your desired Event and Type of messages. Below are possible values for them. 

Possible LogNames (not a complete list). These can be obtained from Windows Event Viewer.

• Security
• System
• Application
• Setup
• EC2ConfigService
• Microsoft-Windows-TerminalServices-LocalSessionManager/Operational

Possible Levels:

• 1: Error Only
• 2: Warning Only
• 4. Information Only
• 3: Error and Warning
• 5: Error and Information
• 6: Warning and Information
• 7: Error, Warning, and Information

Performance Counters

Fullname: 

AWS.EC2.Windows.CloudWatch.PerformanceCounterComponent.PerformanceCounterInputComponent,

AWS.EC2.Windows.CloudWatch

Id: Update the Id to something unique. Example JSON file has "PerformanceCounter." You can use "MemoryCounter" instead. Do not use special characters or spaces in the ID. 

CategoryName: These can be obtained from Performance Monitor: Add Counter. Categories are first level values shown on the box on top left. They are shown in blue.

CounterName: These can be obtained by expanding the CategoryName. 

InstanceName: These can be obtained from Bottom Left of the Add Counter dialog box. For most this is blank.

MetricName: Some custom metric name that defines this metric

Unit. Possible Values:

Seconds | Microseconds | Milliseconds | Bytes | Kilobytes | Megabytes | Gigabytes | Terabytes | Bits | Kilobits | Megabits | Gigabits | Terabits | Percent | Count | Bytes/Second | Kilobytes/Second | Megabytes/Second | Gigabytes/Second | Terabytes/Second | Bits/Second | Kilobits/Second | Megabits/Second | Gigabits/Second | Terabits/Second | Count/Second | None

DimensionName: Name of the dimension that uniquely identifies this data value. For my situation, I used "ServerName"

DimensionValue: The value for the dimension. For my situation, I used the system variable called,     "{hostname}" Other possible values are {instance_id} and {ip_address}, or combination of these three.

Here's my setting:

Custom logs: 

Custom logs can be uploaded to cloudwatch provided that it meets certain criteria:

• Each entry must begin with the date format following by a space
• Log must be one of .NET framework supported text encoding: https://msdn.microsoft.com/en-us/library/system.text.encoding.aspx

Fullname: 

AWS.EC2.Windows.CloudWatch.CustomLog.CustomLogInputComponent,

AWS.EC2.Windows.CloudWatch"

LogDirectoryPath: Location of the logs

TimestampFormat: See https://msdn.microsoft.com/en-us/library/8kb3ddd4.aspx

Filter: See https://msdn.microsoft.com/en-us/library/system.io.filesystemwatcher.filter.aspx

CultureName: Leave it blank to use local locality settings

TimeZoneKind: Local to use local timezone

LineCount: Number of lines in the header to identity the log file

Custom Metrics not found elsewhere:

If there are metrics that are not mentioned elsewhere, you can push the metrics up via cli (or SDK)

1 2 3 4 5 6

aws cloudwatch put-metric-data --namespace "MyOwnNameSpace" --metric-name "Memory_Usage" --dimensions "Metric=MegabytesFree,OS=Win,ServerName=MyOwn" --unit "Megabytes" --value "3000"

Unless timestamp is used, it will upload the metric using current data/time. 

Other Logs not mentioned above

• IIS Logs: Didn't use it, but it seems pretty simple enough. Just enable it in the Flows to use it as is.
• ETW (Event Tracing for Windows): Also didn't use it. 

Configuring Flow to post the date in AWS.EC2.Windows.CloudWatch.json file

Find the "Flows" section. Each flow consists of Data ID followed by Destination ID. Using the example JSON file, the two destinations are CloudWatchLogs and CloudWatch. If there are more than one Data ID, then enclose them in (). Here's the example from AWS documentation.