Saturday, April 13, 2019

LAMP for beginner


Linux, Apache, MySQL, PHP (LAMP) for beginners...

Enough to get you started... (this was done in AWS).

  1. Launch a new EC2 instance from AWS' Red Hat 7 AMI (free tier)
  2. Log into it using "ec2-user"
  3. Elevate privilege
    sudo su - 
  4. Do a fresh update
    yum update
  5. Add repositories
    1. REMI:
      rpm -Uvh http://rpms.famillecollet.com/enterprise/remi-release-7.rpm
    2. EPEL
      rpm -Uvh http://dl.fedoraproject.org/pub/epel/7/x86_64/Packages/e/epel-release-7-11.noarch.rpm
    3. MySQL
      rpm -Uvh https://repo.mysql.com/mysql80-community-release-el7.rpm
  6. Installing Apache
    1. Yum install command
      yum install httpd
    2. Set it to start on boot:
      systemctl enable httpd.service 
    3. Start it now:
      systemctl start httpd.service
    4. Default log location (access_log and error_log)
      /var/log/httpd
    5. Default configuration
      /etc/httpd/conf/httpd.conf
  7. Installing mysql
    1. Yum install command
      yum install mysql-server
    2. Set this to start on boot:
      systemctl enable mysqld.service
    3. Start it now
      systemctl start mysqld.service
    4. Get the mysql temporary password for root
      grep "A temporary password" /var/log/mysqld.log | tail -nl
    5. You should get something like this...
       [Server] A temporary password is generated for root@localhost: V!cedo*iP0iW
    6. Secure your mysql
      mysql_secure_installation
    7. Accept all secure measures. Don't forget your new password!
  8. Install php
    1. Do it from REMI repo (RedHat only comes with php 5.3)
      yum --enablerepo=epel,remi-php73 install php
    2. Install Modules of your choice (these are what I installed)
      yum --enablerepo=remi-php73 install php-mysql php-xml php-xmlrpc php-soap php-gd php-fpm
    3. Restart Apache for the php install to take effect
      systemctl restart httpd.service
  9. You should be able to browse to it now
    1. you can do it locally:
      curl locahost
    2. You can do it remotely
      http://publicIP
    3. If you don't see it from remote, try turning off IPTABLES
      systemctl status iptables
  10. Write your first HTML page
    1. Go to /var/www/html/
    2. Edit a new file
      vi index.html
    3. Paste the following
      <html>
<head>
</head>
<body>
<p>Hello World</p>
</body>
</html>
  11. Write your first PHP page
    1. Still at /var/www/html/
    2. Edit a new file
      vi index.php
    3. Anything inside <?php and ?>  will be interpreted as PHP code
    4. Paste the following
      <html>
<head>
</head>
<body>
<?php
print("Hello World");
phpinfo();
?>
</body>
</html>
  12.  Let's secure your site
    1. Install mod_ssl module
      yum install mod_ssl
    2. Get self-signed cert via this command(or look up how to purchase one or get a free one from cacert.org)
      openssl req -newkey rsa:2048 -nodes -keyout /etc/ssl/private/myserver.key -x509 -days 365 -out /etc/ssl/private/myserver.crt
    3. Go to /etc/httpd/conf.d
    4. Create a new file ssl.conf and paste the following into it
      LoadModule ssl_module modules/mod_ssl.so

Listen 443
<VirtualHost *:443>
    ServerName myserver
    SSLEngine on
    SSLCertificateFile "/etc/ssl/private/myserver.crt"
    SSLCertificateKeyFile "/etc/ssl/private/myserver.key"
</VirtualHost>
    5. Restart httpd
      systemctl restart httpd.service
    6. Check log if you run into issue, you may have a syntax error
    7. Go to your browser and use https instead
  13. Bonus Round! Let's accept client certs. Go here for more info.
    1. Update the previous step's ssl.conf with this new
      LoadModule ssl_module modules/mod_ssl.so

Listen 443
<VirtualHost *:443>
    ServerName myserver
    SSLEngine on
    SSLCertificateFile "/etc/ssl/private/myserver.crt"
    SSLCertificateKeyFile "/etc/ssl/private/myserver.key"
          SSLCACertificateFile "/etc/ssl/private/myserver.crt" 
    ## Your choice here is required, optional, and optional_no_ca
    SSLVerifyClient optional_no_ca
    ## this is number of depths of CA to traverse, use 1
    SSLVerifyDepth 1
    ## this send the cert info to PHP
    SSLOptions +StdEnvVars +ExportCertData
    ## this will allow it to accept your own CA unknown to browser
    SSLCADNRequestPath /etc/ssl/private/ 
</VirtualHost>
    2. Restart httpd so that the new ssl.conf is accepted
    3. At this point, you need a cert loaded to your browser to test this
      1. You can use your company cert
      2. You can also use self-signed cert and loaded onto your browser (use openssl to self-sign a key pair then use "openssl pkcs12" command to put it together as PKCS12 so that you can import it into your browser)
    4. Now browse to your sample PHP page you created before and you should get prompted for your cert, give it and let's take a look at the Apache Environment variables
    5.   Creating a table to hold visitor credential. Here's additional how-to mysql.
      1. Log into mysql
        mysql -u root -p
      2. Create a new database
        mysql> CREATE DATABASE mylamp;
      3. Select this database for use
        mysql> USE mylamp;
      4. Let's create our table
        CREATE TABLE users (
    id INT NOT NULL PRIMARY KEY AUTO_INCREMENT,
    username VARCHAR(50) NOT NULL UNIQUE,
    created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
    lastvisit_at DATETIME DEFAULT CURRENT_TIMESTAMP 
);
      5. Let's check it
        mysql> SHOW TABLES;
      6. You can also do this:
        mysql> DESCRIBE users;
      7. Now let's create a new user to access this table (I'm disabling password validation for this user). More info here.
        SET GLOBAL validate_password.policy=LOW;
CREATE USER 'mylampuser'@'localhost'
  IDENTIFIED WITH mysql_native_password BY 'password';
GRANT ALL
  ON mylamp.*
  TO 'mylampuser'@'localhost'
  WITH GRANT OPTION;
      8. You can exit now
        mysql> exit
    6. Let's create some PHP files to write user information to the table. Here's additional info on how to do this.
      1. Create a new file, config.php (at /var/www/html)
        vi config.php
      2. Paste the following
        <?php
/* Database credentials. */
define('DB_SERVER', 'localhost');
define('DB_USERNAME', 'mylampuser');
define('DB_PASSWORD', 'password');
define('DB_NAME', 'mylamp');
 
/* Attempt to connect to MySQL database */
$link = mysqli_connect(DB_SERVER, DB_USERNAME, DB_PASSWORD, DB_NAME);
 
// Check connection
if($link === false){
    die("ERROR: Could not connect. " . mysqli_connect_error());
}
?>
      3. Create a new file, login.php (also at /var/www/html)
        vi login.php
      4. Paste the following 
        <?php
// Include config file
session_start();
require_once "config.php";
if(isset($_SERVER['SSL_CLIENT_S_DN_Email'])){
 $_SESSION['username'] = $_SERVER['SSL_CLIENT_S_DN_Email'];
 $username = $_SERVER['SSL_CLIENT_S_DN_Email'];
 $sql = "SELECT id FROM users WHERE username = ?";
 if($stmt = mysqli_prepare($link, $sql)){
  // Bind variables to the prepared statement as parameters
  mysqli_stmt_bind_param($stmt, "s", $username);
  // Attempt to execute the prepared statement
  if(mysqli_stmt_execute($stmt)){
   /* store result */
   mysqli_stmt_store_result($stmt);

   if(mysqli_stmt_num_rows($stmt) == 1){
    //update user
    $sql = "UPDATE users SET lastvisit_at = now() where username = ?";
   } else{
    //add a new user
    $sql = "Insert into users (username) values (?)";
   }
   if($stmt = mysqli_prepare($link, $sql)){
    mysqli_stmt_bind_param($stmt, "s", $username);
    if(mysqli_stmt_execute($stmt)){
     mysqli_stmt_store_result($stmt);
     mysqli_stmt_free_result($stmt);
     mysqli_stmt_close($stmt);
    }
   }else{
    echo "Oops! Something went wrong. Please try again later.";
   }
  } else{
   echo "Oops! Something went wrong. Please try again later.";
  }
}
}else{
 $_SESSION['username'] = 'unknown';
}
echo $_SESSION['username'];
mysqli_close($link);
?>

      5. Also create users.php to view all the entries in the table
        vi users.php
      6. Paste the following (more info here)
        <?php
require_once "config.php";

$sql = "SELECT * from users";
if($stmt = mysqli_prepare($link, $sql)){
        if(mysqli_stmt_execute($stmt)){
                mysqli_stmt_store_result($stmt);
                printf("Number of rows: %d.<br>", mysqli_stmt_num_rows($stmt));
                mysqli_stmt_bind_result($stmt, $id, $username, $created, $lastuse);
                while(mysqli_stmt_fetch($stmt)){
                        printf("%s -- %s -- %s -- %s. <br>",$id,$username,$created,$lastuse);
                }
                mysqli_stmt_free_result($stmt);
                mysqli_stmt_close($stmt);
        }
}else{
        echo "Oops! Something went wrong. Please try again later.";
}
/* close connection */
mysqli_close($link);
?>
      7. Browse to users.php, you'll see 0 entries
      8. In a new tab, open login.php, you should get success
      9. Now, go back to users.php tab, and refresh the page, you should see 1 entry with your information.
      10. For extra flexibility, you can include this in your other page to process the login action (index.php)
        <html>
<head>
</head>
<body>
<?php
include 'login.php';
print("Hello World");
?>
</body>
</html>























    at No comments:

    Thursday, March 21, 2019

    Lambda write & read DynamoDB

    Writing and Reading from DynamoDB using Lambda Python

    ...using API Gateway as trigger.

    Below is a simple example of writing to and reading from a DynamoDB table. This code would be triggered by a webpage that sends a JSON package via configured API Gateway. And it would return a JSON package back to the website. Be sure the Role has the necessary permission to read and write to DynamoDB table being used.
    import json
import decimal
import boto3
from boto3.dynamodb.conditions import Key, Attr

# Helper class to convert a DynamoDB item to JSON.
class DecimalEncoder(json.JSONEncoder):
    def default(self, o):
        if isinstance(o, decimal.Decimal):
            if o % 1 > 0:
                return float(o)
            else:
                return int(o)
        return super(DecimalEncoder, self).default(o)

def lambda_handler(event, context):
    # TODO implement
    #define DynamoDB object
    dynamodb = boto3.resource('dynamodb')
    #define which table we want to use
    table = dynamodb.Table("yourOwnTable")
    #print the status of the table
    print(table.table_status)
    
    #Get the body string from the event input json
    #we convert this to json so we can use it easier
    body = json.loads(event['body'])
    #we enter this into the DB
    respose1 = table.put_item(
        Item={
            'RequestTime': body['myFormVariables']['RequestTime'],
            'User': body['myFormVariables']['User'],
            'entryID': body['myFormVariables']['entryID'],
            'Field1': body['myFormVariables']['Field2'],
            'Field2': body['myFormVariables']['Field1']
        }
    )

    print("Table Output")
    ##get it all
    response2 = table.scan()
    ##Get only the objects that match the given key
    #response = table.query(KeyConditionExpression=Key('entryID').eq("L3dVGMDS7yqzdoxCxOm4fg"))
    
    #Print all values in the response
    for i in response2['Items']:
        print(i['entryID'], ":", i['Field1'], ":", i['Field2'], ":", i['RequestTime'])
    
    #Just to illustrate how I handled cognito user information that was passed in
    cognito = event['requestContext']['authorizer']['claims']['cognito:username']
    
    return {
        'statusCode': 200,
        'body': json.dumps(response2, cls=DecimalEncoder)
    }

    Test Sample

    Here is a test event you can use to test your code. 

    {
  "path": "/entry",
  "httpMethod": "POST",
  "headers": {
    "Accept": "*/*",
    "Authorization": "eyJraWQiOiLTzRVMWZs",
    "content-type": "application/json; charset=UTF-8"
  },
  "queryStringParameters": null,
  "pathParameters": null,
  "requestContext": {
    "authorizer": {
      "claims": {
        "cognito:username": "the_username"
      }
    }
  },
  "body": "{\"myFormVariables\":{
  \"RequestTime\":"2019-01-11T05:56:22.517Z\",
  \"User\":\"the_username\",
  \"entryID\":\"L3dVGMDS7yqzdoxCxOm4f1\",
  \"Field1\":47,
  \"Field2\":122}}"
}

    Output


    at No comments:

    Sunday, March 10, 2019

    Lambda read S3 object with S3 trigger

    Configuring your Lambda function to read S3 object with S3 upload trigger


    Are you new to Lambda and haven't quite got the test case figured out for S3 upload trigger? In the steps below, you'll build a new Python Lambda function from a blueprint and use the CloudWatch log to populate a new test case. This way you don't have to upload just to test your new function.
    1. Create a new Lambda Function
    2. Select "Use a blueprint"
    3. Search for "S3"
    4. Select "s3-get-object-python3"
    5.  Configure it using mostly default settings
      1. Function Name: of your choice
      2. Create a new role from AWS policy template
      3. Role Name: of your choice
      4. Select your own trigger bucket
      5. Select all object create event
      6. Leave prefix and Suffix blank
      7. Enable trigger
      8. Select Create Function
    6.  Uncomment line 12 that begins print('Received event: '
    7.  Upload a file to the bucket you select in the previous step
    8.  From the Lambda function console, go to Monitoring
    9. Go to View logs in CloudWatch - this will open a new tab into CloudWatch
    10. Click the latest and probably ONLY log stream
    11. View in TEXT (not rows)
    12. Copy the text like below as shown in your log screen into your clipboard
    13. Now go back to your Lambda function console
    14. Up by the top between Action and Test, click on the drop down and select Configure Test Event
    15. Create new test event (it does not matter what template you choose)
    16. Give it a name
    17. In the body, paste the string from previous step that you put in your clipboard (overwrite existing text)
    18. Save and then click Test
    19. Your Lambda function will behave as if you've just uploaded that same file again
    20. Now go edit the function to do what you want and test easily!
    at No comments:

    Saturday, March 9, 2019

    Powershell Update ENI's DNS Tag

    Update AWS network interface's Tag with its DNS entry (PowerShell 3.0)

    Want to keep a tag that contains the DNS entry of an IP address of your ENIs?

    This PowerShell function calls nslookup (Windows native) and adds a tag to ENI with the returned value.

    You can also find me here.


    ################################################
#
# Get all the reserved ENI in the VPC and
#  create a new tag based on the 
#  responding nslookup on the Private IP
#
#
#  (\_/)
#  (>.<)
# (")_(")
#
#################################################
# Get ENIs with ID, Status, PrivateIP, and DNS Tag
$ip_raw = aws ec2 describe-network-interfaces --filter "Name=vpc-id,Values=vpc-xxxxxxx" --query "NetworkInterfaces[*].{ID:NetworkInterfaceId,Status:Status,IP:PrivateIpAddress,DNS:TagSet[?Key=='DNS'].Value}"
# Convert this to PS Object
$ip = $ip_raw | out-string | convertfrom-json
foreach($item in $ip){
# Proceed only if DNS Tag does NOT exist
  if($item.DNS.length -eq 0){
    $error.clear()
    $dns = nslookup $item.ip
    if($error.count -eq 0){
      $name_line = $dns | ?{$_ -like 'Name*'}[0]
      $dns_name = ($name_line.split(":")[1].trim()
    }else{
      $dns_name = "None"
    }
    aws ec2 create-tags --resources $item.ID --tags "Key=DNS,Value=$dns_name"
  }
}
    at No comments:

    Lambda Update EC2 Tags

    Update EC2 Tags using Python (Lambda function)


    Simple example demonstrating Python's ability to lookup and update Tags.
    You can also find me here.  


    import json
import boto3
from datetime import date

# I don't like the dictionary returned by AWS, so I convert it to
# Key:Value pairs
def parse_tags(tag_dict):
    my_dict = {}
    for tag in tag_dict:
        for item in tag:
            if item == 'Key':
                key = tag[item]
            else:
                value = tag[item]
        my_dict[key]= value
    return my_dict

def lambda_handler(event, context):
    today = date.today()
    year = today.strftime("%Y")
    month = today.strftime("%m")
    day = today.strftime("%d")
    #Declare object for all of our ec2 objects in this region
    ec2 = boto3.resource('ec2', region_name='us-east-2')
    #give me all the instances
    instances = ec2.instances.all()
    print('Instances')
    for ins in instances:
        print("Instance Id: ", ins.id)
        ins_tag_dict = {}
        if(ins.tags != None):
            ins_tag_dict = parse_tags(ins.tags)
        #get function of dictionary return None if not found
        name_tag = ins_tag_dict.get('Name')
        if(name_tag == None):
            #Create a name tag for this object
            name_tag = 'Sam'
            ins.create_tags(Tags=[{'Key':'Name','Value': name_tag}])
        #Give me all the volumes for this instance
        volumes = ins.volumes.all()
        for vol in volumes:
            vol_tag_dict = {}
            if(vol.tags != None):
                vol_tag_dict = parse_tags(vol.tags)
            print("Volume Id: ",vol.id)
            # attachment (LIST) has following values:
            ## [{'AttachTime': datetime.datetime(2018, 12, 3, 5, 11, 5, tzinfo=tzlocal()), 'Device': '/dev/xvda', 'InstanceId': 'i-XXXXXXXX', 'State': 'attached', 'VolumeId': 'vol-XXXXXXX', 'DeleteOnTermination': True}]
            # Convert this LIST to DICT
            vol_att_dict = vol.attachments[0]
            vol_device = vol_att_dict.get('Device')
            vol_name_tag = vol_tag_dict.get('Name')
            if(vol_name_tag == None):
                #Create a name tag for this object
                vol_name_tag = name_tag + '_' + vol_device
                vol.create_tags(Tags=[{'Key':'Name','Value': vol_name_tag}])
        #Give ENI names of the EC2 Instance if they are missing name tag
        net_interfaces = ins.network_interfaces
        for eni in net_interfaces:
            print("ENI Id: ",eni.id)
            eni_tag_dict = {}
            if(eni.tag_set != None):
                eni_tag_dict = parse_tags(eni.tag_set)
            eni_name_tag = eni_tag_dict.get('Name')
            if(eni_name_tag == None):
                eni.create_tags(Tags=[{'Key':'Name','Value':name_tag}])
    print('Volumes not in use')
    #give me all the volumes that are not in use
    volumes = ec2.volumes.filter(Filters=[{'Name': 'status', 'Values': ['available']}])
    for vol in volumes:
        print("Volume Id: ",vol.id)
        vol_tag_dict = {}
        if(vol.tags != None):
            vol_tag_dict = parse_tags(vol.tags)
        vol_mode_tag = vol_tag_dict.get('Mode')
        if(vol_mode_tag == None):
            #If Mode tag does not exist, make it Auto mode
            vol.create_tags(Tags=[{'Key':'Mode','Value':'Auto'}])
        vol_expire_tag = vol_tag_dict.get('Expire')
        if(vol_expire_tag == None):
            #If Expire tag does not exist then set it to 7 day from now
            ## we'd have a different function to do the actual cleanup
            new_day = str(int(day) + 7).zfill(2)
            expireDate = year + month + new_day
            vol.create_tags(Tags=[{'Key':'Expire','Value': expireDate}])
    return {
        'statusCode': 200,
        'body': json.dumps('Finished!')
    }
    at No comments:
    Subscribe to: Posts (Atom)

    AWS WAF log4j query

    How to query AWS WAF log for log4j attacks 1. Setup your Athena table using this instruction https://docs.aws.amazon.com/athena/latest/ug/wa...

    • AWS WAF log4j query
      How to query AWS WAF log for log4j attacks 1. Setup your Athena table using this instruction https://docs.aws.amazon.com/athena/latest/ug/wa...
    • Python Hints
      Python hints Untrusted target If you get untrusted cert error when trying to download library because your firewall, you can do this (only...
    • Working with Cloudwatch Event Rules in Terraform
      Terraform Notes Working with Cloudwatch Event Rules References: Loading from Files in Terraform Using Loops in Terraform Terrafo...