lambda_sqs_light

template-serverless-lambda-python - Lambda SQS Light

Template for build flexible SQS processor with AWS Lambda.

Service Architecture

Diagrams with application usage of this architecture.

More details here.

Cloud Architecture

Example of the architecture running on AWS Cloud.

Docker Architecture

Exemple of the architecture running with docker.

General Service Routes Architecture

There are no routes for this project.

Prerequisites

• Python >=3.6
• docker
• docker-compose
• python-dotenv
• jsonformatter
• requests
• pytz
• redis
• pyyaml
• apispec
• marshmallow
• Flask

Features

• Docker-compose
• Localstack
• SQS Integration
• Flask
• MySQL
• Redis

Details about requirements files

requirements.txt

Collection of common application modules, light modules.

requirements-vendor.txt

Collection of specific application modules, heavy modules that can be converted to layers if necessary.

requirements-tests.txt

Collection of specific test application modules.

Installation

Installing AWS CLI

Documentation: https://docs.aws.amazon.com/pt_br/cli/latest/userguide/install-cliv2.html

Execute the follow command:

apt install python38-env
apt install awscli
apt install zip
app install pip

Execute the follow command:

aws configure

Installing python venv support

Execute the follow command:

apt install python38-env

Creating the Docker Network

To execute the creation of the Docker network, execute the follow command:

./scripts/docker/create-network.sh

Running

Running Locally

To create the venv and install the modules execute:

./scripts/venv.sh

Running the app (only for APIs)

This mode provide an execution of Flask without the docker, but you will need to start any dependencies of the project by yourself.

Execute the follow command:

./scripts/flask/run-local.sh

Running via docker (Fast Development mode - without Lambda creation)

To do de fast development execute de follow commands.

To execute the build:

./scripts/dev.sh --build

Execute the follow command:

./scripts/dev.sh

Running via docker (Full Development mode - with Lambda creation)

To simulate the lambda creating in the development environment execute the commands bellow.

To execute the build:

./scripts/runenv.sh --build

Execute the follow command:

./scripts/runenv.sh

Recovering the environment in error cases

Execute the follow command:

./scripts/fixenv.sh

Automation scripts information

Bellow we describe the usage of the automation scripts. These kebab case scripts helps the developer in general tasks.

General scripts

Kebab case script to help the developer in general tasks.

Script	Description	Context
autopep8.sh	Execute the code-lint for pep8	Codelint
boot.sh	Boot the application during de container execution	Local boot
boot-db.sh	Boot the data for the database	Local boot
boot-queues.sh	Boot the queues of the application in the localstack	Local boot
boot-validate-connection.sh	Check if localstack is ready to connect	Local boot
clean-env.sh	Clean the ./vendor folder	Local install
fixenv.sh	In some cases where the network are deleted, you can fix the container references	Local install
install.sh	Script to install the dependencies	Local install
install-local.sh	Script to install the dependencies in the ./vendor folder	Local install
openapi.sh	Script to generate the openapi.yaml	CI/CD pipeline
preenv.sh	Script to execute the pre build commands	Local boot
pylint.sh	Script to execute the pylint analysis	Local development
dev.sh	Script to start the project locally for development	Local development
runenv.sh	Script to start the project locally	Local development
testenv.sh	Script to run the environment with focus in the component tests	Local development
venv.sh	Script to install the dependencies in the venv folder	Local install
venv-exec.sh	Script to execute scripts to install content inside the venv	Local install
zip.sh	Generate a zip file with the application content	Other

Docker scripts

Helper scripts to do tasks for docker context;

Flask scripts

Helper scripts to run flask locally, not inside a docker container;

Localstack scripts

Helper scripts to run commands over Localstack resources like S3, SQS, Lambda, etc;

Migrations scripts

Helper scripts to execute migrations;

OpenApi scripts

Helper scripts to generate openapi schemas and specifications;

Tests scripts

Helper scripts to execute tests and generate reports;

Samples

See the project samples in this folder here.

Running tests

To run the unit tests of the project you can execute the follow command:

First you need install the tests requirements:

./scripts/venv-exec.sh ./scripts/tests/install-tests.sh 

Unit tests:

Executing the tests:

./scripts/venv-exec.sh ./scripts/tests/unit-tests.sh

Executing a specific file:

./scripts/venv-exec.sh ./scripts/tests/unit-tests.sh /tests/unit/test_app.py

Components tests:

Start the docker containers:

./scripts/testenv.sh

Executing the tests:

./scripts/venv-exec.sh ./scripts/tests/component-tests.sh

Executing a specific file:

./scripts/venv-exec.sh ./scripts/tests/component-tests.sh /tests/component/test_app.py

Integration tests:

Copy the file env/integration.env.example to env/integration.env and edit it with de staging parameters.

Executing the tests:

./scripts/venv-exec.sh ./scripts/tests/integration-tests.sh

Executing a specific file:

./scripts/venv-exec.sh ./scripts/tests/integration-tests.sh /tests/integration/test_app.py

All tests:

Executing the tests:

 ./scripts/venv-exec.sh ./scripts/tests/tests.sh 

Generating coverage reports

To execute coverage tests you can execute the follow commands:

Unit test coverage:

Execute the follow command:

./scripts/venv-exec.sh ./scripts/tests/unit-coverage.sh

Component test coverage:

Start the docker containers:

./scripts/testenv.sh

Execute the follow command:

./scripts/venv-exec.sh ./scripts/tests/component-coverage.sh

Integration test coverage:

Copy the file env/integration.env.example to env/integration.env and edit it with de staging parameters.

Execute the follow command:

./scripts/venv-exec.sh ./scripts/tests/integration-coverage.sh

Observation:

The result can be found in the folder target/*.

License

See the license: LICENSE.md.

Contributions

• Anderson de Oliveira Contreira andersoncontreira

IDE configuration

• For docstring syntax please use the reStructuredText
• For line limit use 100 chars as defined by PEP8

Pylint

To execute the pylint in the sourcecode of the project, execute the follow command:

./scripts/pylint.sh

Or:

./scripts/pylint.sh ./app.py

AutoPEP8

To execute the autopep8 in the sourcecode of the project, execute the follow command:

./scripts/autopep8.sh

Or:

./scripts/autopep8.sh ./app.py

Sending an item to Localstack SQS

To send a default event to SQS execute the follow command:

./scripts/localstack/sqs/send-message.sh

To send a default payload to a specific queue, execute the follow command:

./scripts/localstack/sqs/send-message.sh test-queue

To send a specific payload, execute the follow command:

./scripts/localstack/sqs/send-message.sh test-queue '{ 
   "chavenfe": "32210206107255000134550010001712551245826554", 
   "ocor": "MOTIVO DO CANCELAMENTO", 
   "origem": "SAC/EAGLE", 
   "pedido": "Z1223321" 
}'

To send a specific file payload, execute the follow command:

./scripts/localstack/sqs/send-message.sh test-queue ./samples/ocorens/cancelamento.json