AWS Cloudwatch - Windows Logs

AWS Cloudwatch - Windows Logs

References:

• http://docs.aws.amazon.com/AWSEC2/latest/WindowsGuide/send_logs_to_cwl_gs.html
• http://docs.aws.amazon.com/AWSEC2/latest/WindowsGuide/UsingConfig_Install.htm

Basic Steps to get it running (this was tested with EC2 service version 3.19.1153)

1. Update EC2Config to the latest version
2. Open EC2ConfigService Settings
3. Under General Tab, Enable CloudWatch Logs
4. Copy the sample JSON file to your EC2 install location's settings folder
1. Download Sample JSON File
2. c:\program files\amazon\ec2configservice\settings
5. Edit the JSON file 
1. "Id":"CloudWatchLogs" section should have your information. I leave AccessKey and SecretKey blank because I prefer to use IAM Role that has access to write to CloudWatch. Also, I prefer {hostname} to default, {instance_id} because hostname means something without cross referencing. 
2. "Id":"CloudWatch" section should have your region and NameSpace. NameSpace is the name that you give to your CustomMetrics.
6. Go to Services, restart "Ec2Config" service. 
7. You should see Application and System Event Logs in your CloudWatch Logs

Configuring Logs in AWS.EC2.Windows.CloudWatch.json file

Windows Logs

Fullname: 

AWS.EC2.Windows.CloudWatch.EventLog.EventLogInputComponent,

AWS.EC2.Windows.CloudWatch

Id: Update the Id to something unique.

Edit the LogName and Levels to your desired Event and Type of messages. Below are possible values for them. 

Possible LogNames (not a complete list). These can be obtained from Windows Event Viewer.

• Security
• System
• Application
• Setup
• EC2ConfigService
• Microsoft-Windows-TerminalServices-LocalSessionManager/Operational

Possible Levels:

• 1: Error Only
• 2: Warning Only
• 4. Information Only
• 3: Error and Warning
• 5: Error and Information
• 6: Warning and Information
• 7: Error, Warning, and Information

Performance Counters

Fullname: 

AWS.EC2.Windows.CloudWatch.PerformanceCounterComponent.PerformanceCounterInputComponent,

AWS.EC2.Windows.CloudWatch

Id: Update the Id to something unique. Example JSON file has "PerformanceCounter." You can use "MemoryCounter" instead. Do not use special characters or spaces in the ID. 

CategoryName: These can be obtained from Performance Monitor: Add Counter. Categories are first level values shown on the box on top left. They are shown in blue.

CounterName: These can be obtained by expanding the CategoryName. 

InstanceName: These can be obtained from Bottom Left of the Add Counter dialog box. For most this is blank.

MetricName: Some custom metric name that defines this metric

Unit. Possible Values:

Seconds | Microseconds | Milliseconds | Bytes | Kilobytes | Megabytes | Gigabytes | Terabytes | Bits | Kilobits | Megabits | Gigabits | Terabits | Percent | Count | Bytes/Second | Kilobytes/Second | Megabytes/Second | Gigabytes/Second | Terabytes/Second | Bits/Second | Kilobits/Second | Megabits/Second | Gigabits/Second | Terabits/Second | Count/Second | None

DimensionName: Name of the dimension that uniquely identifies this data value. For my situation, I used "ServerName"

DimensionValue: The value for the dimension. For my situation, I used the system variable called,     "{hostname}" Other possible values are {instance_id} and {ip_address}, or combination of these three.

Here's my setting:

Custom logs: 

Custom logs can be uploaded to cloudwatch provided that it meets certain criteria:

• Each entry must begin with the date format following by a space
• Log must be one of .NET framework supported text encoding: https://msdn.microsoft.com/en-us/library/system.text.encoding.aspx

Fullname: 

AWS.EC2.Windows.CloudWatch.CustomLog.CustomLogInputComponent,

AWS.EC2.Windows.CloudWatch"

LogDirectoryPath: Location of the logs

TimestampFormat: See https://msdn.microsoft.com/en-us/library/8kb3ddd4.aspx

Filter: See https://msdn.microsoft.com/en-us/library/system.io.filesystemwatcher.filter.aspx

CultureName: Leave it blank to use local locality settings

TimeZoneKind: Local to use local timezone

LineCount: Number of lines in the header to identity the log file

Custom Metrics not found elsewhere:

If there are metrics that are not mentioned elsewhere, you can push the metrics up via cli (or SDK)

1 2 3 4 5 6

aws cloudwatch put-metric-data --namespace "MyOwnNameSpace" --metric-name "Memory_Usage" --dimensions "Metric=MegabytesFree,OS=Win,ServerName=MyOwn" --unit "Megabytes" --value "3000"

Unless timestamp is used, it will upload the metric using current data/time. 

Other Logs not mentioned above

• IIS Logs: Didn't use it, but it seems pretty simple enough. Just enable it in the Flows to use it as is.
• ETW (Event Tracing for Windows): Also didn't use it. 

Configuring Flow to post the date in AWS.EC2.Windows.CloudWatch.json file

Find the "Flows" section. Each flow consists of Data ID followed by Destination ID. Using the example JSON file, the two destinations are CloudWatchLogs and CloudWatch. If there are more than one Data ID, then enclose them in (). Here's the example from AWS documentation. 

AWS DynamoDB

How to use DynamoDB from AWS CLI. No extra SDK or API. Examples.

These examples assumes you've followed AWS tutorial on setting up your own table. But so we're on the same page, here's my table details:

• Primary partition key: thisDate (String)
• That's it...

Why would you ever want to use DynamoDB from command line when you can use PHP, Java, .Net, etc? For me, I wanted to use DynamoDB for storage of metrics, logs, and flags that my PowerShell scripts may use to do (or not do) some activity for my servers. Hey, it's Tuesday, run this script unless there is an entry in the database that tells you otherwise. Until now, I've been using Oracle DB for this purpose which is way overkill. I'm doing this on Powershell, but I'm not using AWS CLI for Powershell. I could have ran these from Command Prompt or Bash. Be sure to format properly for JSON whatever environment you may try these on. Also these JSONs were created using AWS Web Console's DynamoDB Create Item button.

Putting an entry into the table

1 2 3 4 5

$item = '{"thisDate":{"S":"20170131"},"Enabled":{"S":"Yes"}}' $jsonItem = $item | convertTo-json aws dynamodb put-item --table-name myTest --item $jsonItem --condition-expression "attribute_not_exist(thisDate)"

The --condition-expression used here makes sure the thisDate column is unique and not overwrite existing value

Retrieving an entry from the table

1 2 3 4

$item = '{"Date":{"S":"20170131"}}' $jsonItem = $item | convertTo-json aws dynamodb get-item --table-name myTest --key $jsonItem

This will return the content in JSON format.

Retrieving content that matches certain criteria from the table

1 2 3 4 5 6

$attrib_names = '{"#d":"thisDate"}' | convertTo-json $attrib_values = '{":year":{"S","2017"}}' | convertTo-json aws dynamodb scan --table-name myTest --expression-attribute-names $attrib_names --expression-attribute-values $attrib_values --filter-expression "begins_with(#d,:year)"

Attribute names are just substitution of the real column to a short form. These must begin with the "#" symbol. And these are mandatory if your column is a reserved word. If you don't want to take a chance, then just create a substitution attribute name for all your columns that you want to use in the filter expression. But here are list of all reserved words, http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/ReservedWords.html

Attribute values are variable name, definition, and the value that will be used in the filter expression. These variables must begin with the ":" symbol.

Filter-expression can also be your regular comparison operators (<,>,=,>=,<=). For example,
--filter-expression "#d >= :year" would also be a valid comparison. Here's a more detailed (and really complicated) explanation.
http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/QueryAndScan.html#FilteringResults

Note that when doing scanning, the entire table is read, so you're not saving any read-times or reservations. And I haven't found a simple way to delete based on some "where" clause, so I just loop through the keys returned from the table scan.

ColdFusion Oddities

I started working [again] with ColdFusion at work recently. We've also migrated from version 9 to 10. And I'm frustrated a lot from certain 'features' that isn't working as expected. This is probably due to legacy settings when I took over the system. But it's annoying nonetheless and I thought I should document it for future reference. If you know of a better workaround, feel free to leave me a comment.

1. "The specific CFC [name] could not be found."

In an attempt to bind a CFC to a CFGRID, you may have seen its inability to find the said CFC file.
<cfgrid name="books" format="html" bind="cfc:books.getRS({cfgridpage},{...},{...},{...})"...
This is because, most likely, the ColdFusion is looking under installation directory's wwwroot (ie. c:\coldfusion10\cfusion\wwwroot). But why? In the same page, I can instantiate an object as follows:
<cfobject name = "book_object" component="books"/>
Whatever the reason, adding a mapping in my Application.cfc didn't do the trick. Instead, I ended up adding a mapping "/mycfc" in the CF Admin site. As a result, my working code looks like this:
<cfgrid name="books" format="html" bind="cfc:mycfc.books.getRS({cfgridpage},{...},{...},{...})"...

2. "Import for tag CFFORM are missing. Use CFAJAXIMPORT..."

So it may be a violation of some Web Programmer's ethics, but I have a single page, "Index.cfm" that uses CFLAYOUT and in it is how I display all my contents. Hence, no need to put header and footer onto each of my pages.
<cflayout type="vbox" name="mylayout">
  <cflayoutarea>header...</cflayoutarea>
  <cflayoutarea name = "body">
    <cfdiv id="contentHere" bind="url:home.cfm"/>
  </cflayoutarea>
  <cflayoutarea>footer...</cflayoutarea>
</cflayout>
So all the content goes in the highlighted portion. But if I try to put a page with CFFORM or CFGRID, it refuses with a something is missing error. If the same page is requested from outside of the cfdiv, it works without any problem. A way around this problem was to put the following line above the <cfdiv.../> line.
<cfajaximport tags="cfform,cfgrid">

3. A cfinput checkbox cannot be used alone.

So with the previously mentioned cfgrid, I had added a filter checkbox, and I passed this to the bind argument. But it kept seeing the value of the checkbox as "yes", even when it wasn't checked.
Admin Only?: <cfinput type="checkbox" name="adminOnly" value="yes" id="adminOnly" onClick="refreshGrid()">
For clarification, refreshGrid() is a javascript function that calls the ColdFusion.grid.refresh command.
The only way around this that I could tell was to add a second checkbox that had the "no" value and hide it.
<cfinput type="checkbox" name="adminOnly" value="no" disabled="disabled" style="opacity:0">


More to come...

Kindle Fire HD 8.9

How to install custom ROM on your Kindle Fire HD 8.9 from Windows 7.

If you had any other version of Kindle Fire, you wouldn't be reading this post because you would have already found one that worked for you on XDA developers site. If you're at a point where you are getting the following errors, you've come to the right place. This instruction assumes you've already followed these instructions:

1. Downloaded all the necessary files on Installing Cyanogenmod on Kindle 8.9
2. Downloaded Google App package here
3. You've downloaded Android SDK and followed the instruction in enabling ADB driver for your Kindle device here
4. You've successfully followed Installing Cyanogenmod instruction up to,
   Installing CyanogenMod from Recovery

Now you're getting the following errors:

• When issuing "adb devices" command you get nothing, hence you can't push files onto your device.
• Your Kindle device in your Device Manager still shows up with a question make (despite what the Amazon developer told you in his instruction).
• You try to mound from TRWP, but you get some funky error in your console. This just means that it couldn't mount it for R/W access like a flash drive for easy drag and drop access. 

Take the following steps to fix your issue.

1. Enable ADB driver for your Kindle device
1. Right Click on your Unknown Kindle device from Device Manager
2. Click Update Driver Software...
3. Browse for driver and Click "Let me pick from a list of devices on my computer"
4. Select Android Composite ADB Interface (latest version)
5. Now your ADB connection should work and you should have a new Device icon minus the question mark
2. Go back to your terminal, and type, "adb devices"
3. Now you should see a device in a recovery mode (if you left it in TWRP)
4. Transfer files as instructed by the Installing Cyanogenmod provided above. 
5. Enjoy.
   

AFCAP and Common Criteria

Common Criteria

The formal name, Common Criteria for Information Technology Security Evaluation (CC), is an international standards for certification of computer technology. CC establishes (by its users) certain security functions and assurance requirements. The vendor in return can implement these requirements. Then the independent laboratories can evaluate the products to validate the claim made by the vendors. (https://www.commoncriteriaportal.org/)

 

EAL

 

When a product receive CC certification, it will also have associated level of assurance. For example, commonly used assurance level seen will be Evaluation Assurance Level (EAL). EAL ranges from EAL 1 (most basic) to EAL 7 (most stringent). Higher EAL does not mean more security, it reflect the degree to which this product has been verified. (https://en.wikipedia.org/wiki/Evaluation_Assurance_Level)

• EAL1: Functionally Tested
• EAL2: Structurally Tested
• EAL3: Methodically Tested and Checked
• EAL4: Methodically Designed, Tested, and Reviewed
• EAL5: Semiformally Designed and Tested
• EAL6: Semiformally Verified Design and Tested
• EAL7: Formally Verified Design and Tested

CC Products
Simplest way to determine if your product has received a CC certification, go to this link https://www.commoncriteriaportal.org/products/, expand all categories, then press F3 to search.

AF C&A Process: How does having CC Certification affect my ability to use the product on the AF network?
That depends on what type of product it is. If it is IA or IA-Enabled Product, then it must have CC certification in order for you to use it. This will be documented in IA Control: ESCS, DCAS, DCCS. This is also true of Multi-Function Devices (MFDs). If your products is neither, then having CC certificate can provide you (IAM) the assurance that the product has been tested. (https://afkm.wpafb.af.mil/forum/default.aspx?g=rsstopic&pg=posts&t=119258&Filter=OO-SC-IA-01)