Schedule a Consultation
Search
left arrowBack
Pavel Rykov

Pavel Rykov

July 26, 2023 ・ Infrastructure

Migrating your Kubernetes app to AWS Lambda and the other way

Introduction

Migrating from Kubernetes to AWS Lambda and the other way around can be a confusing, or even difficult task. If you’re a Kubernetes user who is considering moving to AWS Lambda, or vice versa, it is important to understand the differences between the two technologies and the tools required to complete the job. This guide will provide an overview of the key differences between Kubernetes and AWS Lambda, as well as their respective benefits and drawbacks.

Kubernetes and AWS Lambda are two very different technologies, each with its own set of advantages and disadvantages. Kubernetes is a relatively easy and cost-effective way to set up and manage containerized applications and services, while AWS Lambda is a serverless computing technology that runs code on demand and is highly scalable. Kubernetes offers more flexible deployment options and automated management features, such as pod scaling and updates, but also requires more setup and maintenance. Alternatively, AWS Lambda provides an easier learning curve but needs more architecture work to ensure application scalability.

In order to successfully migrate from one platform to the other, you need to understand the differences, capabilities, and best practices for both platforms. This guide will cover the key steps and considerations you need to take when migrating your Kubernetes application to AWS Lambda, or the other way around, starting with an overview of the differences between the two. We will also go over the steps to follow in order to migrate your application successfully and with minimal disruption.

Kubernetes Architecture

Kubernetes is a cloud platform for creating and managing applications. It is composed of numerous components that work together to provide a powerful and efficient environment for applications to run. Kubernetes allows for efficient management of application lifecycles, deployments, and scaling.

Kubernetes consists of several major components that form the architecture. These components include nodes, services, and Pods. Nodes are physical or virtual machines that run the Kubernetes master component and host application containers. Services are abstractions that define network endpoints and public IP addresses. Pods are containers that are created to run applications.

Kubernetes also provides an API that allows developers to interact with the cluster and manage application deployments. This API consists of several key components such as the Node Controller, Service Controller, Replication Controller, and Scheduler. The Node Controller is responsible for managing and responding to requests from nodes. The Service Controller is responsible for discovering and maintaining network and public IP address mappings. The Replication Controller is responsible for ensuring application versions are correct and running and managing replicas. The Scheduler is responsible for deciding where and which containers should run.

Kubernetes also uses an overlay network for communication between nodes and services. This overlay network provides a secure, reliable, and efficient communication mechanism for applications and services to communicate across a network. This allows for microservices and applications to be communicated with easily and flexibly.

Lastly, Kubernetes provides a platform for monitoring and managing applications. This includes the ability to view deployment status, identify and troubleshoot errors, and audit application performance. This makes it easier for teams to manage and develop applications. This can help reduce the amount of time that developers spend troubleshooting and debugging applications.

Kubernetes provides a powerful and efficient environment for developing, deploying and managing applications. It is composed of several key components that all together form the architecture. It utilizes an API, overlay network, and monitoring platform to provide a secure and reliable system. This makes it easier for developers to manage and develop applications, resulting in increased productivity.

AWS Lambda Architecture

AWS Lambda architecture is an event-driven computing architecture designed to provide scalability and reliability to applications built on Amazon Web Services (AWS). Lambda is an ideal architecture for applications requiring real-time processing and high throughput. It allows developers to quickly deploy and manage their applications without the need for provisioning or managing servers.

The Lambda architecture is based on the concept of decoupling the workload of an application into smaller “Lambda” functions. Each of these functions is given their own resources and workload, and can be triggered independently based on triggers or events. This allows for more flexibility and scalability as each Lambda function can scale according to its own needs.

In addition to its distributed workload, Lambda provides a number of services that make application development easier. These services include Amazon CloudFront, Amazon API Gateway, Amazon Kinesis Streams and Amazon SNS. These services provide a unified experience enabling developers to quickly create, deploy and manage their applications with minimal effort.

The Lambda architecture also offers security benefits by isolating each Lambda function in its own container. This improves system security because each container has no access to any other resources on the system. Additionally, Lambda functions also have access to AWS services such as IAM roles, which enables developers to create granular-level access control for their application.

Finally, Lambda allows developers to quickly and easily deploy their applications to the cloud using the AWS CLI and Amazon CloudFormation. With CloudFormation, developers can easily define their application architecture and deploy it to the cloud in a matter of minutes. This speeds up the deployment process and reduces the time needed for development, allowing for quicker time to market and increased productivity.

Migrating your Application from Kubernetes to AWS Lambda

Migrating an application from Kubernetes to AWS Lambda can be a daunting task. AWS Lambda provides an event-driven, serverless platform, while Kubernetes provides a traditional container-based platform. Each platform offers a different way of managing and running your application, and migrating your application between the two can be a complex process.

The first step in the migration process is to understand the differences between the two platforms. AWS Lambda is based on the idea of functions-as-a-service (FaaS) which allows developers to focus on their application logic rather than managing infrastructure or dealing with scalability issues. Kubernetes, on the other hand, is a container-orchestration platform that allows for more flexibility and control over how the application is deployed.

The next step is to map the application’s code to AWS Lambda’s functions. This requires breaking the code into separate functions or “microservices” that can be mapped to various lambdas. The code must also be adapted to run within the Lambda environment, as AWS Lambda has different memory and execution limits than Kubernetes.

Once the code has been adapted and mapped to the lambdas, the third step is to create the necessary infrastructure to support the application. This typically involves using the AWS Serverless Application Model (SAM) to define the necessary resources such as databases, queues, and other services.

Finally, the application must be tested and deployed to the Lambda environment. This process involves executing integration tests to ensure that the code is functioning correctly and that the application is able to scale as needed. Once the tests are successful, the application can then be deployed to Lambda.

However, if your application is built on the OpenFaaS platform or is already using the amazon/aws-lambda-* containers, then the process will be a lot quicker. These projects provide preconfigured and containerized Lambda functions that can be deployed with minimal changes. The OpenFaaS and aws-lambda-* container images allow developers to package, test, and deploy their Lambda functions without having to rewrite any code.

Moreover, the AWS Lambda service provides extensive integration with other Amazon services such as AWS CloudFormation and AWS CodeDeploy, which facilitate the deployment of Lambda functions with minimal effort. Additionally, AWS provides tutorials and libraries that help developers quickly integrate their functions into the Lambda service.

Simple example

Let's consider the following example: suppose you have a Python application using the Flask framework that implements an API doing some operations with a database. For example, the standard CRUD for working with a users table.

from flask import Flask, request, render_template
import pymysql

app = Flask(__name__)


@app.route('/showUsers', methods=['GET'])
def showUsers():
   connection = pymysql.connect(
       host='localhost', user='testuser', password='test', db='testdb', charset='utf8mb4',
       cursorclass=pymysql.cursors.DictCursor)
   cursor = connection.cursor()
   sql = 'SELECT * FROM users'
   cursor.execute(sql)
   users = cursor.fetchall()
   connection.close()
   return users


@app.route('/addUser', methods=['POST'])
def addUser():
   connection = pymysql.connect(
       host='localhost', user='testuser', password='test', db='testdb', charset='utf8mb4',
       cursorclass=pymysql.cursors.DictCursor)
   cursor = connection.cursor()
   userName = request.form['userName']
   email = request.form['email']
   sql = f'INSERT INTO users (name, email) VALUES ("{userName}", "{email}")'
   cursor.execute(sql)
   connection.commit()
   connection.close()
   return 'User Added Successfully!'


@app.route('/editUser/<string:id>', methods=['POST'])
def editUser(id):
   connection = pymysql.connect(
       host='localhost', user='testuser', password='test', db='testdb', charset='utf8mb4',
       cursorclass=pymysql.cursors.DictCursor)
   cursor = connection.cursor()
   userName = request.form['userName']
   email = request.form['email']
   sql = f'UPDATE users SET name="{userName}", email="{email}" WHERE id={id}'
   cursor.execute(sql)
   connection.commit()
   connection.close()
   return 'User Updated Successfully!'


@app.route('/deleteUser/<string:id>', methods=['POST'])
def deleteUser(id):
   connection = pymysql.connect(
       host='localhost', user='testuser', password='test', db='testdb', charset='utf8mb4',
       cursorclass=pymysql.cursors.DictCursor)
   cursor = connection.cursor()
   sql = f'DELETE FROM users WHERE id={id}'
   cursor.execute(sql)
   connection.commit()
   connection.close()
   return 'User Deleted Successfully!'


if __name__ == '__main__':
   app.run()

And you need to move this test application from Kubernetes to AWS Lambda, for this you need to create separate functions that performs the necessary work in accordance with the AWS Lambda architecture standards:

import json
import pymysql
import os


def showUsers(event, context):
   connection = pymysql.connect(
       host=os.environ['RDS_HOSTNAME'], user=os.environ['RDS_USERNAME'],
       password=os.environ['RDS_PASSWORD'], db=os.environ['RDS_DB_NAME'],
       charset='utf8mb4', cursorclass=pymysql.cursors.DictCursor)
   cursor = connection.cursor()
   sql = 'SELECT * FROM users'
   cursor.execute(sql)
   users = cursor.fetchall()
   connection.close()
   return {
       'statusCode': 200,
       'body': json.dumps(users)
   }


def addUser(event, context):
   connection = pymysql.connect(
       host=os.environ['RDS_HOSTNAME'], user=os.environ['RDS_USERNAME'],
       password=os.environ['RDS_PASSWORD'], db=os.environ['RDS_DB_NAME'],
       charset='utf8mb4', cursorclass=pymysql.cursors.DictCursor)
   cursor = connection.cursor()
   userName = event['userName']
   email = event['email']
   sql = f'INSERT INTO users (name, email) VALUES ("{userName}", "{email}")'
   cursor.execute(sql)
   connection.commit()
   connection.close()
   return {
       'statusCode': 200,
       'body': 'User Added Successfully!'
   }


def editUser(event, context):
   connection = pymysql.connect(
       host=os.environ['RDS_HOSTNAME'], user=os.environ['RDS_USERNAME'],
       password=os.environ['RDS_PASSWORD'], db=os.environ['RDS_DB_NAME'],
       charset='utf8mb4', cursorclass=pymysql.cursors.DictCursor)
   cursor = connection.cursor()
   id = event['userId']
   userName = event['userName']
   email = event['email']
   sql = f'UPDATE users SET name="{userName}", email="{email}" WHERE id={id}'
   cursor.execute(sql)
   connection.commit()
   connection.close()
   return {
       'statusCode': 200,
       'body': 'User Updated Successfully!'
   }


def deleteUser(event, context):
   connection = pymysql.connect(
       host=os.environ['RDS_HOSTNAME'], user=os.environ['RDS_USERNAME'],
       password=os.environ['RDS_PASSWORD'], db=os.environ['RDS_DB_NAME'],
       charset='utf8mb4', cursorclass=pymysql.cursors.DictCursor)
   cursor = connection.cursor()
   id = event['userId']
   sql = f'DELETE FROM users WHERE id={id}'
   cursor.execute(sql)
   connection.commit()
   connection.close()
   return {
       'statusCode': 200,
       'body': 'User Deleted Successfully!'
   }

Then you can deploy this application through the AWS console or use the serverless project functionality.

service: users-management

provider:
  name: aws
  runtime: python3.7
  region: ${env:AWS_REGION}
  environment:
    RDS_HOSTNAME: ${env:RDS_HOSTNAME}
    RDS_USERNAME: ${env:RDS_USERNAME}
    RDS_PASSWORD: ${env:RDS_PASSWORD}
    RDS_DB_NAME: ${env:RDS_DB_NAME}

functions:
  showUsers:
    name: show-users
    handler: app.showUsers
    events:
      - http:
          path: users/show
          method: get
  addUser:
    name: add-user
    handler: app.addUser
    events:
      - http:
          path: users/add
          method: post
  edituser:
    name: edit-user
    handler: app.editUser
    events:
      - http:
          path: users/edit
          method: post
  deleteUser:
    name: delete-user
    handler: app.deleteUser
    events:
      - http:
          path: users/delete
          method: post

Next, you must make sure that the ~/.aws directory has the correct settings in the credentials and config files.

After that, you can deploy the application on AWS Lambda:

sls deploy --env AWS_REGION,RDS_HOSTNAME,RDS_USERNAME,RDS_PASSWORD,RDS_DB_NAME

Note: you may also use the .env file to list the given environment variables in it, which will be read at the time of running the sls command.

For more detailed information on how to install and use the serverless CLI application, please refer to the following link.

Migrating your Application from AWS Lambda to Kubernetes

Migrating your Application from AWS Lambda to Kubernetes can be a daunting task - but it is possible. To achieve a successful migration, there are a few key steps you should take.

The first step is to assess your application's dependencies and resource requirements. It is important to understand the resource needs of your application in order to determine which Kubernetes cluster size best suits your application. After assessing the requirements, the next step would be to set up the Kubernetes cluster and deploy your application onto it. This will require you to configure Kubernetes and create the appropriate Kubernetes objects such as Deployments, Services, and Ingress. The next step is to integrate the namespace, pod and deployment lifecycles into your continuous integration/deployment pipeline. There are a lot of tools for integration with Kubernetes, such as Helm and Kustomize, that allow us to deploy applications easier and faster.

The last step is to test your application on the Kubernetes cluster and monitor it to identify any potential issues. If any issues are identified, you should resolve them before completely switching over to the Kubernetes cluster. By following these steps, you should be able to migrate your application from AWS Lambda to Kubernetes successfully.

Simple example

We can use the original code we previously discussed, which we moved to AWS Lambda, to run it on a Kubernetes cluster. To do that, there are several options available:

  1. Rewriting the application with a monolithic architecture format;

  2. Using the OpenFaaS platform to adapt the application to your Kubernetes Cluster;

  3. Utilizing the official AWS Lambda Docker images to run the application in Kubernetes with changes made only to the configuration and deployment procedure.

Let's take a closer look at the third alternative, as it is the simplest and fastest one and does not require any changes in the original application code.

First of all, we will prepare a Dockerfile describing the procedure of preparing an application for running in Kubernetes. The example Dockerfile below uses amazon/aws-lambda-python:3.7 as a base:

FROM amazon/aws-lambda-python:3.7
ENV APP_FUNCTION="app.handler"

# Install the dependencies
COPY requirements.txt  .
RUN pip install -r requirements.txt --target "${LAMBDA_TASK_ROOT}"

# Copy function code
COPY app.py ${LAMBDA_TASK_ROOT}

# Set the CMD to your handler
CMD [ "${APP_FUNCTION}" ]

Assuming the Docker Registry is already set up and available at registry.example.com, this instruction guide will help you build the Docker Image. To proceed, execute the following command:

docker build -t registry.example.com/demo/lambda-python-app:1.0.0 ./lambda-python-app

And afterwards, push the image to the registry with the following command:

docker push registry.example.com/demo/lambda-python-app:1.0.0

Having accomplished that, the application can now be deployed on the Kubernetes Cluster.

Now we need to prepare configuration files for deploying the application to Kubernetes, which requires us to create namespaces, deployments, services and ingress rules. Let's put all files below to the lambda-k8s folder.

Example namespace.yml

---
apiVersion: v1
kind: Namespace
metadata:
  name: lambda-python-app

Example deployment.yml

---
apiVersion: apps/v1beta1
kind: Deployment
metadata:
  name: lambda-python-app-deployment
  namespace: lambda-python-app
  labels:
    app.kubernetes.io/name: lambda-python-app
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app.kubernetes.io/name: lambda-python-app
    annotations:
      app.kubernetes.io/name: lambda-python-app
    spec:
      containers:
      - name: lambda-python-app-showUsers
        image: registry.example.com/lamp-demo/lambda-python-app:1.0.0
        ports:
        - containerPort: 3000
        env: 
        - name: APP_FUNCTION
          value: "app.showUsers"
        - name: AWS_REGION
          value: "us-east-1"
        - name: RDS_HOSTNAME
          value: "hostname"
        - name: RDS_USERNAME
          value: "username"
        - name: RDS_PASSWORD
          value: "password"
        - name: RDS_DB_NAME
          value: "database name"
...
...same for all other lambda functions...
...

Example service.yml

---
apiVersion: v1
kind: Service
metadata:
  name: lambda-python-app-service
  namespace: lambda-python-app
  labels:
    app.kubernetes.io/name: lambda-python-app
spec:
 type: ClusterIP
 ports:
   - port: 80
     targetPort: 3000
     protocol: TCP
     name: http
 selector:
  app.kubernetes.io/name: lambda-python-app

Example ingress.yml

---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
 name: lambda-python-app-ingress
 namespace: lambda-python-app
 labels:
   app.kubernetes.io/name: lambda-python-app
spec:
 rules:
 - host: lambda-python-app.example.com
   http:
     paths:
       - path: /users/showUsers
         pathType: Prefix
         backend:
           service:
             name: lambda-python-app-service
             port:
               number: 80
       - path: /users/addUser
         pathType: Prefix
         backend:
           service:
             name: lambda-python-app-service
             port:
               number: 80
...
...same for all other lambda functions...
...

Now let’s deploy Lambda functions to Kubernetes Cluster:

kubectl apply -f ./lambda-k8s

Summary and Conclusion

In conclusion, migrating applications from Kubernetes to AWS Lambda and back again is a great way to ensure that your cloud applications are running optimally in both environments. By taking advantage of the agility and scalability of AWS Lambda and the scalability, extensibility and flexibility of Kubernetes, you can find the right solution for your application. Each platform offers its own distinct benefits and capabilities and understanding how to migrate applications between them can give you an edge when developing cloud applications.

  • Infrastructure
  • Value
twitterTweetlinkedinSharefacebookShare