AWS Cloudwatch Cross-Account Cross-Region

AWS Notes

Cloudwatch Cross-Account Cross-Region

By enabling this feature you can create Master-Member relationship to share Cloudwatch data. Any accounts can be set up as master. And in AWS Organization, you can allow any account to have access to list of accounts in your Org. With this enabled, you can switch to view another account's Cloudwatch data and you can also create a Dashboard that contains dataset from any of its member accounts. 

References:

Cross-Account-Cross-Region

Cloudwatch_xaxr_dashboard

Setup Cloudwatch's Master account(s)

To allow these accounts to obtain list of all accounts in Org. Create this role and policy in the AWS Org's Master account. The name must be exactly as shown. Add the account number of your selected Cloudwatch Master account. 

Policy
Name: CloudWatch-CrossAccountSharing-ListAccounts-Policy
Body:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "organizations:ListAccountsForParent",
                "organizations:ListAccounts"
            ],
            "Resource": "*"
        }
    ]
}

Role

Name: CloudWatch-CrossAccountSharing-ListAccountsRole
Body:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::999999999999:root"
      },
      "Action": "sts:AssumeRole",
      "Condition": {}
    }
  ]
}

Setup Member account(s)

To expose an account's data, do this for every member account. Add the account number of your selected Cloudwatch Master account.

Role
Name: CloudWatch-CrossAccountSharingRole
Body:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::99999999999:root"
      },
      "Action": "sts:AssumeRole",
      "Condition": {}
    }
  ]
}

Choose from these collection of AWS Managed Policies based on your needs

1) Full Read Only:
- arn:aws:iam::aws:policy/CloudWatchReadOnlyAccess
- arn:aws:iam::aws:policy/CloudWatchAutomaticDashboardsAccess
- arn:aws:iam::aws:policy/job-function/ViewOnlyAccess
- arn:aws:iam::aws:policy/AWSXrayReadOnlyAccess
2) Both Dashboard and X-Ray
- arn:aws:iam::aws:policy/CloudWatchReadOnlyAccess
- arn:aws:iam::aws:policy/CloudWatchAutomaticDashboardsAccess
- arn:aws:iam::aws:policy/AWSXrayReadOnlyAccess
- arn:aws:iam::aws:policy/CloudWatchReadOnlyAccess
3) Dashboard only
- arn:aws:iam::aws:policy/CloudWatchReadOnlyAccess
- arn:aws:iam::aws:policy/CloudWatchAutomaticDashboardsAccess
4) X-ray only
- arn:aws:iam::aws:policy/CloudWatchReadOnlyAccess
- arn:aws:iam::aws:policy/AWSXrayReadOnlyAccess

Finally, you do this once in every Cloudwatch's Master account. I could not figure out a way to do this programmatically 

• Cloudwatch
• Settings
• Configure Cross-account cross-region
• Edit cross-account cross-region
• Select AWS Org selector and Save

AWS VPC Data Call

Terraform Notes

VPC Data call

Click Here for TF Reference.

In Terraform you can make Data Call to get information about VPC. Few catches, though...

• TF limits you to return only exactly 1 matching VPC
• TF will fail miserable if 0 matching VPC is returned

There are existing arguments, but if you just want the VPC that has a Tag Key, you can do a sub-block of filter.

provider "aws" {
  region   = "us-east-1"
  profile  = "dx"
  insecure = "true"
  alias    = "dx"
}

data "aws_vpc" "this" {
  filter {
    name="tag-key"
    values=["SGCatalog"]
  }
  provider = aws.dx
}

locals {
    VPC_ID = data.aws_vpc.this.id
    SGCatalogItems = split(",",data.aws_vpc.this.tags.SGCatalog)  
}

module "security_groups" {
  source = "make_my_security_groups"
  vpc_id = local.VPC_ID
  sg = local.SGCatalogItems
  providers = {
    aws = aws.dx
  }
}

Inviting AWS SecurityHub Members

Terraform Notes

Security Hub Members

Want to automate inviting members to join the Master account for Security Hub? You can simplify this using AWS Organization.

Because the data call to "aws_organizations_organization" returns a list of map of non_master_accounts, we must first convert this to a map of map.

List of Map                     
Map of Map          
 {

    "arn" = ""
    "email" = ""
    "id" = ""
}
 "id" = {

                "arn" = ""
                "email" = ""
                "id" = ""
            }

provider "aws" {
  version = "~> 2.0"
  region  = "us-east-1"
  profile = "master"
}

provider "aws" {
  alias    = "master_east"
  region  = "us-east-1"
  insecure = "true"
  profile = "master"
}

data "aws_organizations_organization" "current" {}
locals {
    all_accounts = {
        for x in data.aws_organizations_organization.current.non_master_accounts: x.id => x
    }  
}
  
resource "aws_securityhub_member" "this" {
    for_each = local.all_accounts
    account_id = each.value["id"]
    email      = each.value["email"]
    invite     = true}

Want to be have another account (not Master of Org) to be the Master of the SecurityHub collection?

provider "aws" {
  version = "~> 2.0"
  region  = "us-east-1"
  profile = "master"
}

provider "aws" {
  alias    = "master_east"
  region  = "us-east-1"
  insecure = "true"
  profile = "master"
}

provider "aws" {
  region   = "us-east-1"
  profile  = "secaudit"
  insecure = "true"
  alias    = "secaudit"
}

##Use the default provider so that we can get ID of ALL Account in the ORG
data "aws_organizations_organization" "master" {}

##Use the SecAudit provider to get the ID of that Account
data "aws_caller_identity" "secaudit" {
    provider  = aws.secaudit

}

##Make a map of all the Account IDs in the Org
locals {
    all_accounts = {
        for x in data.aws_organizations_organization.master.non_master_accounts : x.id => x
    }  
}

##Create invite for SecurityHub Member from SecAudit account to all other Accounts (excluding itself)
resource "aws_securityhub_member" "this" {
    provider  = aws.secaudit
    for_each = {
      for key, value in local.all_accounts:
      key => value
      if key != data.aws_caller_identity.secaudit.account_id
    }
    account_id = each.value["id"]
    email      = each.value["email"]
    invite     = true
} 

Get started with Jupyter

Jupyter

Doing some Python coding and want a nicer environment to try some code outside of py? Try JupyterLab.

Installation Link

First thing you want to do after you install JupyterLab is configure starting directory. The tutorial already exists here. 

In case that link is no longer available:

1. Run this command: jupyter notebook --generate-config
2. Edit the resulting file so that c.NotebookApp.notebook_dir = 'c:/mypath/blah/'
1. Be sure to use forward slash
2. Remove any # from beginning of the line

Start Jupyter

You can create a link for this execution file on your desktop or add it to start with your machine

Few notes:

• You have to do SHIFT+Enter to execute a line
• You have to hold CTRL+Right Mouse click to execute browser context actions (copy/paste/etc)

If you want a Jupyter for a large group on an auto-scaling cluster, try JupyterHub

There is also a JupyterHub for non-auto-scaling single server setup. 

Using Boto3 Paginator

Using Boto3 Paginator

If you are retrieving any data from AWS using Boto3, you should make a habit of using Paginator for the day when your data-set gets too large for a single call to handle.

Here's a simple illustration on how to lookup paginator arguments and returns and how to use it in your code.

Go to Boto3 Docs
Every service has a paginators. Write your code based on the class definition as shown above.
Here's the code:

thisClient = boto3.client("iam", aws_access_key_id=credentials['AccessKeyId'],
                                 aws_secret_access_key=credentials['SecretAccessKey'],
                                 aws_session_token=credentials['SessionToken'])
paginator = thisClient.get_paginator('list_users')
    response_iterator = paginator.paginate()
    for response in response_iterator:
        for key in response.get('Users'):
            thisUserId = key.get('UserId')
            thisUserName = key.get('UserName')

Managing Terraform State file

Terraform Notes

Moving resources and backing up tf state

This will illustrate process of migrating resources from one state to another as well as backing up and reverting to different states. 

In your working directory, you should have at least terraform.tfstate. If you have defined an external backend, then this file may be cached under .terraform directory (and your actual changes are happening on remote location). 

if your main.tf is this:

variable "timestamps" {
  type    = bool
  default = false
}

resource "null_resource" "exampleA" {
  triggers = {
    key = formatdate("YYYY-MM-DD hh:mm:ssZZZZZ", timestamp())
  }
}

Then your intial terraform.tfstate would look like this:

{
  "version": 4,
  "terraform_version": "0.12.24",
  "serial": 1,
  "lineage": "d0427b0f-6517-25bb-f708-6666666666",
  "outputs": {},
  "resources": [
    {
      "mode": "managed",
      "type": "null_resource",
      "name": "exampleA",
      "provider": "provider.null",
      "instances": [
        {
          "schema_version": 0,
          "attributes": {
            "id": "8139684392621207633",
            "triggers": {
              "key": "2020-04-23 17:41:12+00:00"
            }
          },
          "private": "bnVsbA=="
        }
      ]
    }
  ]
}

To get list of all your resource, you can run "terraform state list"

From the output from above, you can use the mv command to move resources to be managed by different terraform state. You can also use this command to refactor the name of the resource.

terraform state mv 

-backup='terraform.backup.tfstate' 

-state-out='terraform.new.tfstate' 

 null_resource.exampleA 

 null_resource.exampl

The above command will create two additional terraform state files.

terraform.tfstate

{
  "version": 4,
  "terraform_version": "0.12.24",
  "serial": 2,
  "lineage": "d0427b0f-6517-25bb-f708-6666666666",
  "outputs": {},
  "resources": []
}

terraform.new.tfstate

{
  "version": 4,
  "terraform_version": "0.12.24",
  "serial": 1,
  "lineage": "c7e1be5e-5363-39be-7078-6666666666",  "outputs": {},
  "resources": [
    {
      "mode": "managed",
      "type": "null_resource",
      "name": "exampleB",
      "provider": "provider.null",
      "instances": [
        {
          "schema_version": 0,
          "attributes": {
            "id": "8139684392621207633",
            "triggers": {
              "key": "2020-04-23 17:41:12+00:00"
            }
          },
          "private": "bnVsbA=="
        }
      ]
    }
  ]
}

terraform.backup.tfstate

{
  "version": 4,
  "terraform_version": "0.12.24",
  "serial": 1,
  "lineage": "d0427b0f-6517-25bb-f708-6666666666",  "outputs": {},
  "resources": [
    {
      "mode": "managed",
      "type": "null_resource",
      "name": "exampleA",
      "provider": "provider.null",
      "instances": [
        {
          "schema_version": 0,
          "attributes": {
            "id": "8139684392621207633",
            "triggers": {
              "key": "2020-04-23 17:41:12+00:00"
            }
          },
          "private": "bnVsbA=="
        }
      ]
    }
  ]
}

Note that lineage of terraform.tfstate and terraform.backup.tfstate is going to be identical.

If you run terraform state list now you'll get nothing.

To go back to previous tf state, you can run the command in reverse where source (-state) is the new terraform state file and destination (-state-out) is the original state file.

terraform state mv 
-backup='terraform.backup2.tfstate' 
-state-out='terraform.tfstate' 
-state='terraform.new.tfstate'
 null_resource.exampleB 
 null_resource.exampleA

If this is just a local state, you can also just rename terraform.backup.tfstate to terraform.tfstate.

Now if you run terraform state list again you'll get your resource back.

Terraform for_each caution

Terraform Notes

For_Each Loop Caution

When using for_each, be sure to be mindful of what you want these new resources to be indexed as. Because if you ever try to change the index name, the resources must be destroyed and recreated. 

Example map variable

variable "account_map"{
    default = {
        "TEST-A-Key" = {
            "name" = "TEST-A-Name"
            "email" = "TEST-A@me.com"
        },
        "TEST-B-Key" = {
            "name" = "TEST-B-Name"
            "email" = "TEST-B@me.com"
        }
    }
}

When you create a resource based on the above map, you get this:

resourceType.name["TEST-A-Key"]
resourceType.name["TEST-B-Key"]

Unfortunately, in the above example (and in my real world case), I've tied the index name to be same as the name of the account. So when I went to updated the account name (the above variable is created from a file), it attempted to delete all my account resources. So now my only choice is to delete state of these resources and re-import OR live with mis-matched account name and index string of the resources.

So take caution and use some index string that describes the variable, not the specific content. Or maybe you don't mind destroy and create whenever you want to change a name.