SHIFT

--- Sjoerd Hooft's InFormation Technology ---

User Tools

Site Tools


Sidebar

Sponsor:

Would you like to sponsor this site?
Or buy me a beer?:


Recently Changed Pages:

View All Pages
View All Q Pages


View All Tags


Sign up for Q to post comments.





WIKI Disclaimer: As with most other things on the Internet, the content on this wiki is not supported. It was contributed by me and is published “as is”. It has worked for me, and might work for you.
Also note that any view or statement expressed anywhere on this site are strictly mine and not the opinions or views of my employer.


Terms And Conditions for Q users


Pages with comments

PageDateDiscussionTags
2019/06/22 10:36 1 Comment
2019/03/15 16:02 1 Comment
2019/03/15 16:02 1 Comment
2019/03/15 16:02 3 Comments
2017/04/20 15:28 1 Comment
2017/04/20 15:23 1 Comment
2017/04/19 14:44 1 Comment
2017/04/17 20:10 1 Comment
2017/04/17 20:07 1 Comment
2017/04/17 19:58 1 Comment
2017/04/17 19:52 1 Comment

View All Comments

webappaws

Creating a Serverless WebApp with AWS

In this article I will describe how I created a Web App using only serverless AWS technlogy. If you're looking on hosting a static S3 website or how to deploy code files using azure DevOps see Getting Started With AWS, Transfer Domain to AWS and Getting Started with Azure DevOps for AWS. In this article I'll describe the following technologies:

  • AWS SES (Simple Email Service)
  • Route 53
  • Cognito
  • Dynamo DB
  • IAM
  • Lambda
  • API Gateway
  • CloudWatch
  • Frontend Javascript

The Use Case

The Use Case for which I created the WebApp is a simple but secure website to maintain the balance volunteers have to get snacks. They use a simple system in which they (translate turven) every time they take something. At the end of the day/week/month someone had an enormous excel list to update the balance of all volunteers. This was error prone, the excel file kept breaking and for the volunteers it was unclear what their balance was at any given time.

I was asked (as being one of the volunteers) if I could create a new excel file… Instead I created a WebApp, as described here on this page.

AWS SES

The WebApp will send an email to a volunteer when their balance is updated. This means that the userlist must contain an email address and Lambda will have to be able to send an email using SES.

Verify Domain

Follow these steps to verify a email domain:

  • Go to the Simple Email Service console
  • Go to domains → Verify a new domain
  • Fill in domain name → Verify this domain
  • Use route53 to verify and notice to not select the receive email record sets (off by default, so don't change anything)
Note that we will later use IAM to add an inline policy: ses:sendemail
Note that we also did not enable DKIM or set SPF records yet. More information later.

Sandbox

By default SES is always enabled as a sandbox, meaning that you can use SES only for sending email from and to verified domains. There are also restrictions on the amount of email you can sent. Because we will sent to all volunteers that will be using the WebApp we must make sure that SES is removed from sandbox modus. This procedure can take up a few days as it includes creating a support ticket. See here for more information on both the restrictions as a detailed description on how to remove SES from the sandbox. In my case it took somewhere about a day, and I honestly told them that I did not have a policy on handling bounces and that kind of stuff.

Check Sandbox Status

As far as I know, the easiest way to check if you're still in sandbox mode is to go to the SES console, and then go to sending statistics under Email Sending, and if you see a blue warning indicating your account is in sandbox mode… you're still in sandbox mode.

Set SPF Record

At the very least, when using another domain to send email from your domain you should set a SPF record. To do so, go to route 53 and change or add a TXT record:

Add Amazon SES to an existing SPF record:

"v=spf1 include:spf.protection.outlook.com include:amazonses.com -all"

Create a new SPF record:

"v=spf1 include:amazonses.com -all"

Amazon Cognito

We'll be using Cognito so users can authenticate to the WebApp.

Create Amazon Cognito Pool

  • From the AWS Console click Services then select Cognito under Mobile Services.
  • Choose Manage User Pools.
  • Choose Create a User Pool
  • Provide a name for your user pool and then select Review Defaults
    • If you'll be having different user pools be sure to create a distinctive name like the domainname
  • On the review page, click Create pool.
  • Note the Pool Id on the Pool details page of your newly created user pool.

Add App Client to User Pool

From the Amazon Cognito console select your user pool and then select the App clients section. Add a new app client and make sure the Generate client secret option is deselected. Client secrets aren't currently supported with the JavaScript SDK. If you do create an app with a generated secret, delete it and create a new one with the correct configuration.

  • From the Pool Details page for your user pool, select App clients from the left General Settings section in the navigation bar.
  • Choose Add an app client.
  • Give the app client a name such as DomainNameWebApp.
  • Uncheck the Generate client secret option. Client secrets aren't currently supported for use with browser-based applications.
  • Choose Create app client.
  • Note the App client id for the newly created application.

DynamoDB

We'll use DynamoDB to store both the transactions (like a logfile) and the current balance of the volunteers. In DynamoDB we'll only need to create the table and define the primarykey. When filling the database, as long as we'll provide a value for the primarykey, anything we throw at it will be accepted. That means we won't to define all the required keys up front.

Create the Tables

Repeat the steps below for these tables:

TableName LogTable SaldoTable
Partition Key LogID Naam
  • From the AWS Management Console, choose Services then select DynamoDB under Databases.
  • Choose Create table.
  • Enter the Table name. This field is case sensitive.
  • Enter the Partition key and select String for the key type. This field is case sensitive.
  • Check the Use default settings box and choose Create.
  • Scroll to the bottom of the Overview section of your new table and note the ARN. We'll need this later on.

IAM

Every Lambda function has an IAM role associated with it. This role defines what other AWS services the function is allowed to interact with. We will create an IAM role that grants your Lambda function permission to write logs to Amazon CloudWatch Logs and access to write and read (scan) items to your DynamoDB SaldoTable, and write items to LogTable.

Attach the managed policy called AWSLambdaBasicExecutionRole to this role to grant the necessary CloudWatch Logs permissions. Also, create a custom inline policy for your role that allows the required DynamoDB and SES permissions.

Create IAM Role for Lambda

  • Go to the IAM console
  • Go to Roles → Create Role
  • Select Lambda for the role type from the AWS service group, then click Next: Permissions.
  • Search & Select the AWSLambdaBasicExecutionRole role
  • Click Next: Tags → Next: Review
  • Enter domainnameLambda for the Role Name.
  • Choose Create Role.

See The Policy in JSON

If you would check the policy in JSON you'd see that only CloudWatch related permissions were assigned:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "logs:CreateLogGroup",
                "logs:CreateLogStream",
                "logs:PutLogEvents"
            ],
            "Resource": "*"
        }
    ]
}

Add DynomoDB Permissions

Now we add the DynamoDB permissions:

  • Type domainnameLambda into the filter box on the Roles page and choose the role you just created.
  • On the Permissions tab, choose the Add inline policy link in the lower right corner to create a new inline policy.
  • Select Choose a service and search and select DynamoDB
  • Choose Select actions.
  • Begin typing PutItem into the search box labeled Filter actions and check the box next to PutItem when it appears. Repeat this voor UpdateItem en Scan
  • Select the Resources section.
  • With the Specific option selected, choose the Add ARN link in the table section.
  • Paste the ARN of the table you created in the previous section in the Specify ARN for table field, and choose Add.
  • Choose Review Policy.
  • Enter domainnameDynamoDB for the policy name and choose Create policy.
Note that you might see warnings that you need resources of the table type. As long as you put in the correct ARNs you can ignore these warnings.

See The Policy in JSON

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "dynamodb:PutItem",
                "dynamodb:Scan",
                "dynamodb:UpdateItem"
            ],
            "Resource": [
                "arn:aws:dynamodb:eu-west-1:**accountnr**:table/SaldoTable",
                "arn:aws:dynamodb:eu-west-1:**accountnr**:table/LogTable"
            ]
        }
    ]
}

Add SES Permissions

  • Type domainnameLambda into the filter box on the Roles page and choose the role you just created.
  • On the Permissions tab, choose the Add inline policy link in the lower right corner to create a new inline policy.
  • Select Choose a service and search and select SES
  • Choose Select actions.
  • Begin typing SendEmail into the search box labeled Filter actions and check the box next to SendEmail when it appears.
  • Select the Resources section.
  • Select Any
  • Choose Review Policy.
  • Enter domainnameSES for the policy name and choose Create policy.

Lambda

AWS Lambda will run the code required to actually put data into DynamoDB and send the email. We'll need to create three functions:

  • domainnameStreepLijst - This will update the SaldoTable, LogTable and send the email
  • domainnameGetLog - This will read the LogTable
  • domainnameGetSaldo - This will read the SaldoTable

Create a Function

Follow these steps to create a function:

  • Choose Services then select Lambda in the Compute section.
  • Click Create function.
  • Keep the default Author from scratch card selected.
  • Enter domainnameStreepLijst in the Name field.
  • Select Node.js 8.10 for the Runtime.
  • Ensure Choose an existing role is selected from the Role dropdown.
  • Select domainnameLambda from the Existing Role dropdown.
  • Click on Create function.
  • Scroll down to the Function code section and replace the exiting code in the index.js code editor with the contents below
  • Click “Save” in the upper right corner of the page.

You can open the function after creating it to update the code. The index.js page is opened by default so you can edit it. After editing you can click save.

Repeat these steps for all functions.

domainnameStreepLijst

This will update the SaldoTable, LogTable and send the email:

const randomBytes = require('crypto').randomBytes;
 
const AWS = require('aws-sdk');
 
const ddb = new AWS.DynamoDB.DocumentClient();
 
const ses = new AWS.SES();
 
var emailfrom = 'sjoerd_getshifting.com';
 
 
exports.handler = (event, context, callback) => {
    //Disabled checking for auth - enabled op 17-3
    if (!event.requestContext.authorizer) {
      errorResponse('Authorization not configured', context.awsRequestId, callback);
      return;
    }
 
    // LogID should be the exact same case as in the database
    const LogID = toUrlString(randomBytes(16));
    console.log('Received event (', LogID, '): ', event);
 
    // Because we're using a Cognito User Pools authorizer, all of the claims
    // included in the authentication token are provided in the request context.
    // This includes the username as well as other attributes.
    const username = event.requestContext.authorizer.claims['cognito:username'];
    //const username = 'tst';
 
    // The body field of the event in a proxy integration is a raw string.
    // In order to extract meaningful values, we need to first parse this string
    // into an object. A more robust implementation might inspect the Content-Type
    // header first and use a different parsing strategy based on that value.
 
    const requestBody = JSON.parse(event.body);
 
    //Email optie returns emaillaag/emailaltijd
    var emailoption = requestBody.Emailoptie;
    var saldo = requestBody.Saldo;
 
	recordSaldo(requestBody, username).then(() => {    
    // You can use the callback function to provide a return value from your Node.js
        // Lambda functions. The first parameter is used for failed invocations. The
        // second parameter specifies the result data of the invocation.
 
        // Because this Lambda function is called by an API Gateway proxy integration
        // the result object must use the following structure.
        callback(null, {
            statusCode: 201,
            body: JSON.stringify({
                Naam: requestBody.Naam,
				Saldo: requestBody.Saldo,
				Poco: username,
            }),
            headers: {
                'Access-Control-Allow-Origin': '*',
            },
        });
    }).catch((err) => {
        console.error(err);
 
        // If there is an error during processing, catch it and return
        // from the Lambda function successfully. Specify a 500 HTTP status
        // code and provide an error message in the body. This will provide a
        // more meaningful error response to the end client.
        errorResponse(err.message, context.awsRequestId, callback);
    });
 
	//  	Europe/Amsterdam
    // Setting timezone
    var dutchTime = new Date().toLocaleString('en-US', {timeZone: "Europe/Amsterdam"});
    var requestTime = new Date(dutchTime).toISOString();
    //console.log('Time now: ', requestTime);
 
 
	recordTransaction(LogID, username, requestBody, requestTime).then(() => {    
 
    }).catch((err) => {
        console.error(err);
 
        // If there is an error during processing, catch it and return
        // from the Lambda function successfully. Specify a 500 HTTP status
        // code and provide an error message in the body. This will provide a
        // more meaningful error response to the end client.
        // so we'll disable this one as well.
        //errorResponse(err.message, context.awsRequestId, callback);
    });
 
 
    if (emailoption == "emailaltijd"){
        console.log('Emailoption is emailaltijd so we will send the email ');
        sendEmail(requestBody, username).then(() => {
        })
        .catch(err => {
            console.error(err);
        });
    }else if (saldo < 5){
        console.log('Emailoption is not emailaltijd but saldo is below 5 so we\'ll send the email anyway ');
        sendEmail(requestBody, username).then(() => {
        })
        .catch(err => {
            console.error(err);
        });
    }else {
        console.log('Emailoption is not emailaltijd and saldo is above 5 so we\'ll do nothing ');
    }
 
 
};
 
 
 
 
function recordTransaction(LogID, username, requestBody, requestTime) {
    return ddb.put({
        TableName: 'domainnameLogTable',
        Item: {
            LogID: LogID,
            Poco: username,
			Consumed: requestBody,
            RequestTime: requestTime,
        }
    }).promise();
}
 
 
 
//function recordSaldo(naam, saldo) {
function recordSaldo(requestBody, username) {
    return ddb.update({
        TableName: 'domainnameSaldoTable',
        //Key: {"Naam": naam},
        Key: {"Naam": requestBody.Naam},
        UpdateExpression: "SET Saldo = :saldo, Email = :email",
        ExpressionAttributeValues: {
            ":saldo": requestBody.Saldo,
            ":email": requestBody.Email
        },
        ReturnValues:"UPDATED_NEW"
    }).promise();
}
 
function sendEmail (requestBody, username) {
    console.log('Send email from: ', username);
    var params = {
        Destination: {
            ToAddresses: [
                requestBody.Email
            ]
        },
        Message: {
            Body: {
                Text: {
                    Data: 'Beste ' + requestBody.Naam + ', \nJe saldo is nu: ' + requestBody.Saldo + '. \nDit is bijgewerkt door: '+ username + '. \nOpmerkingen: ' + requestBody.Opmerkingen + '. \nSaldo gestort: ' + requestBody.SaldoBij + '. \n\nDit heb je deze keer gestreept: \nTotaal Koek: ' + requestBody.Koek + '\nTotaal Bier: ' + requestBody.Bier + '\nTotaal Fris: ' + requestBody.Fris + '\nTotaal Reep en M&Ms: ' + requestBody.ReepMenM+ '\nTotaal Chips: ' + requestBody.Chips + '\nTotaal Snoep: ' + requestBody.Snoep + '\nTotaal Maaltijd Zaterdag: ' + requestBody.MaaltijdZa+ '\nTotaal Maaltijd Zondag: ' + requestBody.MaaltijdZo + '\n\nSta je tekort? Wil je dan zo snel mogelijk geld overmaken?',
                    Charset: 'UTF-8'
                }
            },
            Subject: {
                Data: 'Saldo aanpassing: ' + requestBody.Saldo,
                Charset: 'UTF-8'
            }
        },
        Source: emailfrom
    };
    return ses.sendEmail(params).promise();
}
 
 
// randomizer
function toUrlString(buffer) {
    return buffer.toString('base64')
        .replace(/\+/g, '-')
        .replace(/\//g, '_')
        .replace(/=/g, '');
}
 
function errorResponse(errorMessage, awsRequestId, callback) {
  callback(null, {
    statusCode: 500,
    body: JSON.stringify({
      Error: errorMessage,
      Reference: awsRequestId,
    }),
    headers: {
      'Access-Control-Allow-Origin': '*',
    },
  });
}

domainnameGetLog

This will read the LogTable, just change the tablename if you want the other:

var aws = require('aws-sdk');
var dynamodb = new aws.DynamoDB();
 
exports.handler = (event, context, callback) => {
    dynamodb.scan({TableName: 'domainnameLogTable'}, (err, data) => {
        callback(null, data['Items']);
    });
};

API Gateway

Now we'll create an Amazon API Gateway to expose the Lambda function we'be build as a RESTful API. This API will be accessible on the public Internet. It will be secured using the Amazon Cognito user pool we've created.

Create the API Gateway

  • In the AWS Management Console, click Services then select API Gateway under Application Services.
  • Choose Create API.
  • Select a REST API and a New API and enter DomainName for the API Name.
  • Endpoint Type: Edge optimized
  • Choose Create API
Note: Edge optimized are best for public services being accessed from the Internet. Regional endpoints are typically used for APIs that are accessed primarily from within the same AWS Region.

Create a Cognito User Pools Authorizer

  • Under your newly created API, choose Authorizers.
  • Chose Create New Authorizer.
  • Enter DomainName for the Authorizer name.
  • Select Cognito for the type.

5. In the Region drop-down under Cognito User Pool, select the Region where you created your Cognito user pool (Ireland) 6. Enter DomainName in the Cognito User Pool input. 7. Enter Authorization for the Token Source. 8. Choose Create.

Create a New Put Method

Create a new resource called /streeplijst within your API. Then create a POST method for that resource and configure it to use a Lambda proxy integration backed by the FillLogTable function you created

  • In the left nav, click on Resources under your DomainName API.
  • From the Actions dropdown select Create Resource.
  • Enter streeplijst as the Resource Name.
    • Note that this the name to identify it later with using ajax calls
  • Ensure the Resource Path is set to streeplijst.
  • Select Enable API Gateway CORS for the resource.
  • Click Create Resource.
  • With the newly created /streeplijst resource selected, from the Action dropdown select Create Method.
  • Select POST from the new dropdown that appears, then click the checkmark.
  • Select Lambda Function for the integration type.
  • Check the box for Use Lambda Proxy integration.
  • Select the Region you are using for Lambda Region.
  • Enter the name of the function you created in the previous module, domainnameStreepLijst, for Lambda Function.
  • Choose Save. Please note, if you get an error that you function does not exist, check that the region you selected matches the one you used in the previous module.
  • When prompted to give Amazon API Gateway permission to invoke your function, choose OK.
  • Click on the Method Request card, on the Method Request name.
  • Choose the pencil icon next to Authorization.
  • Select the domainname Cognito user pool authorizer from the drop-down list, and click the checkmark icon.

Enable CORS

Enable CORS to prevent errors like:

Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at https://xxxxxxxxxx.execute-api.eu-west-1.amazonaws.com/prod/streeplijst. (Reason: CORS header ‘Access-Control-Allow-Origin’ missing).

For both the streeplijst resource and the options field:

  • Select method
  • Actions → Enable Cors
  • Select all checkboxes: DEEFAULT 4XX DEFAULT 5XX POST OPTIONS
  • Click enable CORS and replace existing CORS headers

Create a New GET Method

To create the methods for the two GET methods, use the same approach as for the PUT methos except:

  • Do NOT enable “use lambda proxy integration”
Do not forget to enable CORS for the GET methods as well

Deploy Your API

From the Amazon API Gateway console, choose Actions, Deploy API. You'll be prompted to create a new stage. You can use prod for the stage name.

  • In the Actions drop-down list select Deploy API.
  • Select [New Stage] in the Deployment stage drop-down list.
  • Enter prod for the Stage Name.
  • Choose Deploy.
  • Note the Invoke URL. We'll use in the frontend javascript.

Enable CloudWatch Logging

To enable CloudWatch logs for the API Gateway we first need to create an IAM role for it so it is allowed to log to CloudWatch:

  • Navigate to Services → IAM.
  • Click Roles.
  • Click Create Role Button.
  • Under Select Role Type choose API Gateway“.
  • Next permissions
  • Check “AmazonAPIGatewayPushToCloudWatchLogs”, click Next Step.
  • Click Create Role.
  • Click your new role in the roles listing.
  • Make a note of the Role from the Role ARN field.

Now configure the API Gateway:

  • API Gateway service console → Settings → Fill IN ARN
  • Select API Gateway → Stages → Prod → Logs/Tracing → Enable Cloudwatch log → LOG Level: INFO (or error after troubleshooting)

FrontEnd JavaScript

Ok, there might be some jquery there as well but I hardly know the difference.

Requirements

You need these two files, which you can download using this tutorial or get them using npm.

  • amazon-cognito-identity.min.js
  • aws-cognito-sdk.min.js

Config.js

In the config.js you define the Cognito Pool, Client APP en API gateway. You noted all the IDs and urls along the way:

window._config = {
    cognito: {
        userPoolId: 'eu-west-1_XXXXXX', // e.g. us-east-2_uXboG5pAb
        userPoolClientId: 'XXXXXXXXXXXXXXXXXXXXXX', // e.g. 25ddkmj4v6hfsfvruhpfi7n4hv
        region: 'eu-west-1' // e.g. us-east-2
    },
    api: {
        invokeUrl: 'https://xxxxxxxx.execute-api.eu-west-1.amazonaws.com/prod' 
    }
};

JQuery Ajax Call

This is based upon this tutorial:

Put

    function submitToAPI(input) {
        //event.preventDefault();
 
        console.log('Test 46 - token ' + authToken);
 
        $.ajax({
            method: 'POST',
 
            url: _config.api.invokeUrl + '/streeplijst',
            headers: {
                Authorization: authToken
            },
            dataType: "JSON",
            crossDomain: "true",
 
            data: JSON.stringify(input),
 
            contentType: 'application/json',
            success: completeRequest,
            // success: function () {
            //     // clear form and show a success message
            //     alert("Successfull");
            //     document.getElementById("streeplijstform").reset();
            //     location.reload();
            // },
            error: function ajaxError(jqXHR, textStatus, errorThrown) {
                console.error('Error requesting streeplijstupdate: ', textStatus, ', Details: ', errorThrown);
                console.error('Response: ', jqXHR.responseText);
                alert('An error occured when requesting the streeplijst update:\n' + jqXHR.responseText);
            }
        });
    }

Get

var WildRydes = window.WildRydes || {};
 
(function rideScopeWrapper($) {
    var authToken;
    //console.log('Test 8 - userpool ' + userPool);
    WildRydes.authToken.then(function setAuthToken(token) {
        if (token) {
            authToken = token;
        } else {
            window.location.href = '/signin.html';
        }
    }).catch(function handleTokenError(error) {
        alert(error);
        window.location.href = '/signin.html';
    });
 
 
    $(function onDocReady() {
 
        $('#logtablebutton').click(getlog);
 
    });       
 
 
 
function getlog(e) {
    e.preventDefault();
 
    console.log('Test 24 - token ' + authToken);
 
        //var api_gateway_url = _config.api.invokeUrl + '/getlog';
 
        var rows = [];
 
        //$.get(api_gateway_url, function(data) {
        $.ajax({
            method: 'GET',
 
            url: _config.api.invokeUrl + '/getlog',
            headers: {
                Authorization: authToken
            },
            dataType: "JSON",
            crossDomain: "true",
 
            contentType: 'application/json',
            success: function (data) {
            console.log('Get Response received from API: ', data);
            // eerst op volgorde krijgen
            function sortFunction() {
                data.sort(function(a, b){
                    var x = a.RequestTime['S'].toLowerCase();
                    var y = b.RequestTime['S'].toLowerCase();
                    //console.log('Processing sort: ', x + y);
 
                    if (x < y) {return 1;}
                    if (x > y) {return -1;}
                    return 0;
                }); 
            };
            sortFunction();
            data.forEach(function(item) {
 
                var consumed = item['Consumed']['M'];
                //console.log('Get Response received from API Consumed: ', consumed);
 
                //console.log('Get Response received from API Naam: ', naam);
                var result = "Streeplijst: ";
                for(var key in consumed){
 
                    value = consumed[key]['S'];
                    var streep = " - ";
                    var istekst = " is ";
                    var result = result + key + istekst + value + streep;
                }
                //console.log('Test with resultaat: ', result);
 
                rows.push(`<tr> \
                    <td>${item['LogID']['S']}</td> \
                    <td>${result}</td> \
                    <td>${item['Poco']['S']}</td> \
                    <td>${item['RequestTime']['S']}</td> \
                </tr>`);
 
            });
 
            $('#logtable').append(rows.join()).show();
 
            //console.log('Function clickgetsaldo number of rows: ',  child);
            }, // added for ajax call
        });
 
    };
 
}(jQuery));

Resources

You could leave a comment if you were logged in.
webappaws.txt · Last modified: 2019/06/03 12:42 by sjoerd