AWS Adding Security Group Ingress rule

Bulk editing your Security Group Ingress rules

Need to add/delete rules to all the Security Group in your account? Here's how to do it in PowerShell. 

1 2 3 4 5 6 7 8 9 10 11

#Get list of all security group under this account #we assume AWS configure has been ran and output has been set to JSON $sgs = aws ec2 describe-security-groups --query 'SecurityGroup[]{Name:GroupName,Id:GroupId}' #convert this json output to $ps_sgs = $sgs | out-string | convertfrom-json foreach($sg in $ps_sgs){ #below line adds a rule to each SG aws ec2 authorize-security-group-ingress --group-id $sg.id --protocol -1 --port 0-65535 --cidr x.x.x.x/32 #Uncomment below to revoke the rules #aws ec2 revoke-security-group-ingress --group-id $sg.id --protocol -1 --port 0-65535 --cidr x.x.x.x/32 }

Explanation: 

Line 3: describe-security-groups command returns list of all security groups and their attributes of the account where this command is ran. To limit the amount of output, we query the command to only want GroupName and GroupID. We really just need GroupID, but in case we want to do any friendly output or log as we go along. We know these two attributes by looking at the output without the --query flag:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25

{ "SecurityGroups": [ { "IpPermissionsEgress": [], "Description": "My security group", "IpPermissions": [ { "PrefixListIds": [], "FromPort": 22, "IpRanges": [ { "CidrIp": "203.0.113.0/24" } ], "ToPort": 22, "IpProtocol": "tcp", "UserIdGroupPairs": [] } ], "GroupName": "MySecurityGroup", "OwnerId": "123456789012", "GroupId": "sg-903004f8", } ] }

Line 5: Convert the JSON output to PowerShell object.
Line 6: We can iterate through this PowerShell object applying next AWS command to each Security Group.
Line 8: This is how we add ALL ALL allow from this IP Address. The result would look as follows:

Line 9: This is how to remove ingress rules from this security group

References:

AWS EC2 describe-security-groups
PowerShell convertfrom-json

AWS Serverless Example

AWS Lambda and Node.js example

Let's build a page that will calculate the square of the number you provide. This is just a simple static HTML page that will make a POST call to AWS' API Gateway that calls a Lambda function that you have created. Data will be passed using JSON format. 

This example will utilize following AWS Features:

• Lambda
• API Gateway
• S3 Bucket
• Static Website Configuration
• Bucket Policy

Other Stateless server example:

• AWS WildRydes tutorial (AWS Cognito too)

References:

• https://nodejs.org/en/
• https://aws.amazon.com/lambda/
• https://aws.amazon.com/api-gateway/
• https://aws.amazon.com/s3/
• https://jquery.com/

Here are the steps:

1. Go to AWS Lambda
2. Create new function
   
3. Create as follows:
   
4. You can either create a new Role or use existing, but it should be able to at least write to log
   
5.  Runtime and Handler should set properly for you. If you want to know more about handler, press info link
   
6. Overwrite the "Hello World" sample with the below code:
   

   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

   exports.handler = (event, context, callback) => { const requestBody = JSON.parse(event.body); const argument = requestBody.argument; const square = argument * argument; const answer = `${argument}^2 = ${square}`; console.log(answer); callback(null, { statusCode:201, body: JSON.stringify({ Response: answer, }), headers: { 'Access-Control-Allow-Origin': '*', }, }); };

• Line 1: This is mandatory, see notes from step 5
• Line 2: reads the input JSON value from website
• Line 3: extracts "argument" variable 
• Line 4 - 5: squares the value received from website
• Line 6: posts this value to Cloudwatch Log
• Line 7: callback function has two arguments, the first if this function encounters an error and second if it succeeds. 
• Line 8 - 15: This is the response that will be sent back to website when it succeeds
• Line 12: This is mandatory, otherwise you will get "no access-control-allow-origin" error
   
7. Configure AWS API Gateway: go to API Gateway
8. Create a new API
   
9. Create Resource
   
10. Fill the form as follows
    
11. Create Method
    
12. Select POST (be sure to press the check box)
    
13. Fill out the form as follows (substitute for your region of choice)
    
14. Enable CORS (Accept all defaults and warnings)
    
15. Deploy API
    
16. Fill out the form as follows
    
17. Make note of the Invoke URL
    
18. Create config.js file and populate with the above URL
    

    1 2 3 4 5

    window._config = { api: { invokeUrl: 'https://xxxxxxxxxx.execute-api.us-east-1.amazonaws.com/prod' } };

19. Create mycalc.js and populate as follows
    

    1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41

    (function myScopeWrapper($) { $(function onDocReady() { alert('Welcome'); $('#enterBtn').click(handleRequestClick); if (!_config.api.invokeUrl) { $('#noApiMessage').show(); } }); function handleRequestClick(){ var argument = $('#argument').val(); calc(argument); } function calc(argument_var){ $.ajax({ method: 'POST', url: _config.api.invokeUrl + '/mycalc', headers: { }, data: JSON.stringify({ argument: argument_var }), contentType: 'application/json', success: completeRequest, error: completeRequest }); } function completeRequest(result) { console.log('Response received from API: ', result); displayUpdate(result.Response); } function displayUpdate(text) { $('#updates').append($('<li>' + text + '</li>')); } }(jQuery));

• Line 2: onDocReady is mandatory, the page will load this upon start. It prepares actions.
• Line 11: This will take the argument variable and call the calc function
• Line 17 - 29: this is the format you should use to pass HTTP content to the API
• Line 19: this should point to the API name you defined in AWS API Gateway
• Line 24: these are your JSON formatted list of variables
• Line 27: this is the action to take on success
• Line 28: this is the action to take on failure (I know this is lazy; I'm using the same function as success)
20. Create myCalc.html
    

    1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35

    <!DOCTYPE html> <html> <head> <meta charset="utf-8"> <meta name="viewport" content="initial-scale=1,maximum-scale=1,user-scalable=no"> <title>My Calc</title> <meta charset="utf-8"> <meta http-equiv="X-UA-Compatible" content="IE=edge"> <meta name="viewport" content="width=device-width, initial-scale=1"> <meta name="description" content="Serverless web application example"> <meta name="author" content=""> </head> <body> <div class="info panel panel-default"> <p>Welcome. Enter a number and press enter to get the square value</p> <div class="panel-heading"> <input id="argument"></input> <button id="enterBtn" class="btn btn-primary">Enter</button> </div> <div class="panel-body"> <ul id="updates"> <li>2^2 = 4</li> </ul> </div> </div> <script src="js/vendor/jquery-3.1.0.js"></script> <script src="js/config.js"></script> <script src="js/mycalc.js"></script> </body> </html>

• Line 20 of mycalc.html is called by Line 12 of mycalc.js
• Line 21 of mycalc.html is called by Line 4 of mycalc.js
• Line 25 of mycalc.html is updated by Line 38 of mycalc.js
• Line 30 imports jquery library (mandatory), you can either use the file or one of many CDNs available such as https://developers.google.com/speed/libraries/#jquery
• Line 31 imports the API Gateway location
• Line 32 imports the JavaScript used in this page
21. Import three files from above into your S3 bucket, be sure to drop all the JS files into appropriate sub directories
22. Configure bucket for Static Website
    
23. Edit Bucket Policy
    

    1 2 3 4 5 6 7 8 9 10 11

    { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": "*", "Action": "s3:GetObject", "Resource": "arn:aws:s3:::BUCKET-NAME/*" } ] }

24. Be aware that anyone can use this API call, so if you want to be cautious about it, you can restrict access to just your IP address:
    

    1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

    { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": "*", "Action": "s3:GetObject", "Resource": "arn:aws:s3:::BUCKET-NAME/*", "Condition": { "IpAddress": { "aws:SourceIp": "X.X.X.X/32" } } } ] }

25. Go to mycalc.html and open the link, it will look like this:
    https://s3.amazonaws.com/[bucket-name]/mycalc.html
26. You can also go to the Static URL that was defined from step 22:
    http://[bucket-name].s3-website-us-east-1.amazonaws.com/

Powershell Start-Job example

Example Start-Job usage

Example of using Start-Job in your script.

Job file (\\path\job-example2.ps1)

This is the script that will be called multiple times. Wanted to illustrate calling a file with parameters. It's a trival example, but wanted to show different output in the job results.

param(
    $runTask = $false,
    $action
)
if($runTask){
    $message = $action
    switch($action){
        "0"{
            $message = $message + ":zero"
        }
        "1"{
            $message = $message + ":one"
        }
        "2"{
            $message = $message + ":two"
        }
        "3"{
            $message = $message + ":three"
        }
        "4"{
            $message = $message + ":four"
        }
        "5"{
            $message = $message + ":five"
        }
        default{
            $message = $message + ":don't know"
        }
    }
}else{
    write-output "did not run"
}
Write-Output $message

Tasker file (\\path\job-example1.ps1)

This will create multiple jobs. Each job will be assigned a numeric iteration of the loop value as the name. We'll wait for all the jobs to finish and then get the output of each job using the name we assigned.

$scriptfile = $MyInvocation.MyCommand.Path
$scriptPath = split-path -parent $scriptfile

for($i=0;$i -le 5;$i++){
    $arguments = @($true,"$i")
    start-job -filepath "$scriptPath\job-example2.ps1" -ArgumentList $arguments -name $i
    sleep -Seconds 1
}

do{
    $state = get-job | ?{$_.state -eq 'Running'}
    sleep -second 1
}while($state -ne $null)

for($i=0;$i -le 5;$i++){
    receive-job -name $i
}

Expected output:

Id     Name            PSJobTypeName   State         HasMoreData     Location             Command                  
--     ----            -------------   -----         -----------     --------             -------                  
1      0               BackgroundJob   Running       True            localhost            param(...                
3      1               BackgroundJob   Running       True            localhost            param(...                
5      2               BackgroundJob   Running       True            localhost            param(...                
7      3               BackgroundJob   Running       True            localhost            param(...                
9      4               BackgroundJob   Running       True            localhost            param(...                
11     5               BackgroundJob   Running       True            localhost            param(...                
0:zero
1:one
2:two
3:three
4:four
5:five

ColdFusion and AWS SNS

How to tie ColdFusion 11 server with AWS SNS.
Let's say you have an AWS SNS topic that sends a lot of information. And you're overwhelmed and you want a more streamlined solution such as a dashboard with database backend. And for the sake of this scenario, we'll leave AWS Gateway API and Lambda out of it. Because you already have a ColdFusion just sitting around.

Step 1: Write the CFC files

1. Go to your website's physical directory and create a new folder called "rest". You don't have to call it that, but I do just for example sake. 
2. Create two files:
1. application.cfc
2. sns.cfc
3. application.cfc
   

   component output="false"
   {
        this.name = "my-rest";
        this.applicationTimeout = createTimespan(0,1,0,0);
        this.datasource = "myDB";
   
        this.restSettings.skipCFCWithError = true;
   
        public boolean function onRequestStart()
        {
             request._body = toString(getHttpRequestData().content);
             return true;
        }
   
        public boolean function onRequest()
        {
             return true;
        }
   }
   

1. It is important that you set "request._body" here. Otherwise, you won't see the content. 
2. You can put any other functions here to override the one from parent's Application.cfc
4. sns.cfc
   

   <cfcomponent rest="true" restpath="sns">
    <cffunction name="postSNSTopic" access="remote" httpMethod="POST" returntype="string" produces="application/json">
     <cfargument name="body" type="string" restArgumentSource="body"/>
     <cfscript>
      response = "y";
   
      messageType = getHttpRequestData().headers["x-amz-sns-message-type"];
      messageID   = getHttpRequestData().headers["x-amz-sns-message-id"];
      topicARN    = getHttpRequestData().headers["x-amz-sns-topic-arn"];
      subject     = "None";
      messageJson = "";
      content     = "";
   
      switch(messageType){
       case "SubscriptionConfirmation":
        //Enter subscription action here
        break;
       case "Notification":
        //Enter notification action here
        break;
       default:
        response = "n";
      }
      
      if(response IS "y"){
       content = getHttpRequestData().content;
       contentLen = len(content);
       if(contentLen GT 0){
        contentStruct = deserializeJson(toString(content));
        subject = contentStruct["Subject"];
        if(len(subject) EQ 0){
         subject = "None";
        }
        messageJson = contentStruct["Message"];
        messageLen  = len(messageJson);
        if(messageLen EQ 0){
         messageJson = "No Content";
        }
       }else{
        content = "No Content";
        //TroubleShooting
        //headers = getHttpRequestData().headers;
        //for(item in headers){
         //content = content & "." & item & ":" & getHttpRequestData().headers[item];
        //}
       }
      }
     </cfscript>
     <cfif response EQ "y">
      <cfquery name = "sns_insert" datasource="mydb">
       insert into sns_listener(messageType, messageID, topicARN, subject, messageJSON, content, timestamp) values (
        <cfqueryParam value=#messageType# cfSqlType="cf_sql_varchar" maxlength="100"/>,
        <cfqueryParam value=#messageID# cfSqlType="cf_sql_varchar" maxlength="100"/>,
        <cfqueryParam value=#topicARN# cfSqlType="cf_sql_varchar" maxlength="100"/>,
        <cfqueryParam value=#subject# cfSqlType="cf_sql_varchar" maxlength="100"/>,
        <cfqueryParam value=#messageJson# cfSqlType="cf_sql_clob" maxlength="100"/>,
        <cfqueryParam value=#content# cfSqlType="cf_sql_clob" maxlength="100"/>,
        sysdate)
      </cfquery>
     <cfelse>
      <cfquery name = "sns_insert" datasource="mydb">
       insert into sns_listener(messageType, timestamp) values ('error',sysdate)
      </cfquery>
     </cfif>
    </cffunction>
   </cfcomponent>
   

5. Details about HTTP Header and JSON content from SNS message can be found here.

Step 2: Configuring REST Service in ColdFusion 11

1. Log into ColdFusion Administrator
2. Under Server Settings, click Mappings
3. Add Logical Path as "/rest" 
4. Add Directory Path as the actual path of your files, "E:/website/rest"
5. Under Data & Services, click REST services
6. Add the Logical Path you used above, "/rest"
7. Leave Host as blank
8. You can leave service mapping blank, it will pick this up from your Application.cfc
9. Select "Set as default application"
10. Add service

Step 3: Go to AWS SNS and subscribe your endpoint

1. Your REST end point will be https://www.mywebsite.com/rest/sns
2. "rest" says this is a REST call
3. "sns" is the name of your cfc file that contains the POST function
4. Once you subscribe, you can manually confirm your subscription by pulling out the subscribeURL in the message or you can write a function in the body of sns.cfc above. 

References: 

ColdFusion 10 Rest Developer Guide

GetHttprequest function
cfqueryparam function
HTTPS SNS Issue with CA
SNS JSON format