Let's say that you have a RESTful API deployed to some reserved compute resources in AWS. The API's surface is well-designed and is perfectly capable of meeting all of the needs of any clients that consume it. But then, you're told that there is some small internal feature or action that must be implemented in your cloud infrastructure.
For example, say you need to set up a service that sends an email containing data from the same data sources accessed by your API when triggered by a HTTP request. The API's codebase could very easily be extended to provide this functionality, but doing so can lead to several types of tech debt.
- Design - The API surface should abstract the complexities of data retrieval/syncing, so adding endpoints to manage this process goes against the design pattern.
- Security - If this is an external-facing API, you might run into serious security issues with allowing processes that consume your API to trigger such an action!
- Cost optimization - This is probably not a concern if your code just needs to send an email, but if you need to perform a more compute or memory intensive action, like a sync between two data sources. If this data sync is a very bursty workload, then running it on the same resources provisioned for nominal API usage may lead to surge pricing.
So, looks like you need a scheduled process within your cloud infrastructure that runs this operation separately from your API. This now seems like a lot more of an undertaking. The source code for this tiny task should be very quick to implement, but the overhead of provisioning, testing, deploying, and maintaining new resources dedicated for that task seems too high.
Serverless Functions to the Rescue!
Serverless functions like AWS Lambda (which is what we'll focus on) are perfect for these kinds of actions. They're quick and easy to develop and can cost almost nothing compared to reserved compute like EC2 for small tasks like sending an email or two.
The code below is an example of an entire Lambda function implementation, minus any package dependencies. See how quickly these can be to develop?
var AWS = require('aws-sdk');
AWS.config.update({region: 'us-west-2'});
var ses = new AWS.SES({apiVersion: '2010-12-01'});
exports.handler = async (event) => {
var params = // email params
try {
var data = await ses.sendEmail(params).promise();
return {
statusCode: 200,
body: JSON.stringify("Email sent! Message ID: " + data.MessageId),
};
} catch (error) {
console.error(error);
return {
statusCode: 500,
body: JSON.stringify("Error sending email: " + error.message),
};
}
};
But wait! What if a second data sync action needs to be added? A third? Serverless functions are great for implementing one-off features like this in a vacuum, but managing a whole flock of serverless functions with their own source code, their own VPC configurations, their own deployment processes, etc., can become an unmanageable mess very quickly.
Serverless Framework
The Serverless Framework package provides a suite of Infrastructure-as-Code capabilities built to allow you to deploy and configure one or more serverless functions defined within a single directory. This means that you can easily keep your functions' implementations and configurations all tracked within one repository!
Let's walk through the setup of a project using serverless framework to show how simple it is.
Create a Node Project, and Install the Serverless Framework Package
In your project directory, with your unix shell of choice, run the following.
npm init
npm install serverless
Use the Serverless Framework CLI to Pick the Project Template
The CLI will provide a list of templates to choose from. For this post, we're going to create a Lambda application targeting Node.js with an Amazon API Gateway integration for triggering our functions.
? What do you want to make?
AWS - Node.js - Starter
> AWS - Node.js - HTTP API
AWS - Node.js - Scheduled Task
AWS - Node.js - SQS Worker
AWS - Node.js - Express API
AWS - Node.js - Express API with DynamoDB
AWS - Python - Starter
AWS - Python - HTTP API
AWS - Python - Scheduled Task
AWS - Python - SQS Worker
AWS - Python - Flask API
AWS - Python - Flask API with DynamoDB
Other
Name the project once you've selected the template.
? What do you want to make? AWS - Node.js - HTTP API
? What do you want to call this project? example-project
This should generate the following files in your Node project.
The configuration of each Lambda function and this gateway are all specified in the generated serverless.yaml file.
service: example-project
frameworkVersion: '3'
provider:
name: aws
runtime: nodejs18.x
functions:
api:
handler: index.handler
events:
- httpApi:
path: /
method: get
This config tells the API gateway to invoke our Lambda (referenced here as the auto-generated function name of index.handler, which matches the function code's file name and export field) in a Node 18 runtime, whenever a GET request is sent to the API gateway. The auto-generated function (in ./index.js) looks like this.
module.exports.handler = async (event) => {
return {
statusCode: 200,
body: JSON.stringify(
{
message: "Go Serverless v3.0! Your function executed successfully!",
input: event,
},
null,
2
),
};
};
Creating the Functions
Let's update index.js to include both of our emailing functions.
var AWS = require('aws-sdk');
AWS.config.update({region: 'us-west-2'});
var ses = new AWS.SES({apiVersion: '2010-12-01'});
const sendEmailAndReturnResponse = async (params) => {
try {
var data = await ses.sendEmail(params).promise();
return {
statusCode: 200,
body:
JSON.stringify("Email sent! Message ID: " + data.MessageId),
};
} catch (error) {
console.error(error);
return {
statusCode: 500,
body: JSON.stringify("Error sending email: " + error.message),
};
}
};
exports.mail1 = async (event) => {
var params = getEmailParams1(event); // getEmailParams1 is a function that you'll define for your own system
return sendEmailAndReturnResponse(params);
};
exports.mail2 = async (event) => {
var params = getEmailParams2(event); // getEmailParams2 is a function that you'll define for your own system
return sendEmailAndReturnResponse(params);
};
Configuring the Functions
Updates the serverless.yaml file to map both functions to POST requests sent to their endpoints, and to route the API gateway endpoints to each function correctly.
service: example-project
frameworkVersion: '3'
provider:
name: aws
runtime: nodejs18.x
functions:
api:
handler: index.mail1
events:
- httpApi:
path: /
method: post
api:
handler: index.mail2
events:
- httpApi:
path: /
method: post
Enter your AWS account credentials, and both of your functions will be deployed to a new lambda application for you!
Update Your Functions
Updates to the serverless.yaml or the function source code can be easily applied to the deployments by simply re-running the command above. Serverless Framework tracks the configuratin of your functions with AWS CloudFormation, so it knows exactly what to update when changes have been pushed.
Learn more about DMC's application development expertise and contact us for your next project.