[FIXED] Hash AWS secret value with Argon Algorithm on EC2

Issue

[UPDATE]

I don't know why this question has been marked for closing, it seems a common scenario to me, almost all of the passwords I have seen and I know of are stored in secret storages in plain text (AWS Secret Manager, Hashicorp Vault...ecc) for being used by applications in various scenario such as database connection, and things like that. Encrypting the password locally would force me to keep track of the plain text password in other storage (in my mind, in 1Password...wherever) and breaking other common practice such as automated password rotation.

[The question]

I have a personal and password protected Jupyter Notebook running on an AWS Linux EC2 instance built with Terraform.

At instance creation I am creating and storing a secret password in AWS secrets manager for later retrieving within the user_data and saving it in jupyter_notebook_config.py:

echo "c.ServerApp.password = u'$(aws secretsmanager get-secret-value --secret-id ${aws_secretsmanager_secret.jupyter.arn} --query SecretString --output text)'" >> /home/ec2-user/.jupyter/jupyter_notebook_config.py

It works, with the disadvantage that I have to argon-encrypt the secret before storing it in AWS Secrets Manager.

I am therefore looking for a way to store it in plain text in AWS secret manager and put some local encryption mechanism in between before saving in Jupyter config file. The new script would ideally be:

PLAIN_PWD=$(aws secretsmanager get-secret-value --secret-id ${aws_secretsmanager_secret.jupyter.arn} --query SecretString --output text)
ARGON_HASHED_PWD=some-existing-linux-function($PLAIN_PWD)
echo "c.ServerApp.password = u'$(ARGON_HASHED_PWD)'" >> /home/ec2-user/.jupyter/jupyter_notebook_config.py

Where some-existing-linux-function is already existing or may be installed with package manager (i.e. yum).

In particular an utility for hashing a password is already provided in Jupiter but unfortunately it requires manual intervention such as password confirmation.

This is a fragment of such utility:

import hashlib
from random import random
import argon2

# ......

if algorithm == "argon2":
    ph = argon2.PasswordHasher(
        memory_cost=10240,
        time_cost=10,
        parallelism=8,
    )
    h_ph = ph.hash(passphrase)
    return ":".join((algorithm, h_ph))

Solution

In the end, I decided for the moment to not mess up with EC2 and various dependencies to have argon2 installed as it is something beyond my knowledge.

Instead I ended up having a Python lambda function which does the encryption for me. Moreover I can re-use the function in other scenarios. For example I have an EKS and an ECS Jupyter installation with the same problem.

These are the steps:

Prerequisites

An AWS secret and an IAM role with a policy granting permissions on such secret.

1. Lambda Layer

Prepare a Lambda layer containing argon2 libraries. After several tentatives with different OS (Mac, Windows, Linux...) I run these commands in an EC2 instance because I had some compatibility issues:

python3 -m venv argon2
source argon2/bin/activate
mkdir python # <- AWS requires everything inside a folder named 'python'
pip3 install cffi argon2-cffi argon2-cffi-bindings -t python
zip -r argon2-layer.zip python

2.Create Layer

In Lambda console I created a layer uploading the zip file created in step 1. Important: because the EC2 instance has Python 3.9, I chose Python 3.9 as Lambda runtime as there must be a match between the package version created in step one and the lambda runtime. Other option will cause runtime error.

3. Lambda function

I created this lambda function in console assigning the already existing execution IAM role mentioned in prerequisites (as a personal rule I don't like to let AWS automatically generate a dedicated role). I didn't spend much time in building a solid use case because the sole purpose is to retrieve a single secret which I know exists. This is to say that the lambda function can be enhanced to serve a more robust scenario:

import json
import hashlib
from random import random
import argon2
import boto3

def lambda_handler(event, context):

    if 'secret_name' in event:
        secret_name = event['secret_name']
        
        secret_value = retrieve_secret(secret_name)

        return {
            'statusCode': 200,
            'body': json.dumps(passwd(secret_name))
        }
    else:
        return {
            'statusCode': 400,
            'body': "secret missing"
        }


def passwd(passphrase: str):
    algorithm = "argon2"

    ph = argon2.PasswordHasher(
        memory_cost=10240,
        time_cost=10,
        parallelism=8,
    )
    h_ph = ph.hash(passphrase)
    return ":".join((algorithm, h_ph))


def retrieve_secret(secret_name: str):

    client = boto3.client('secretsmanager')
    
    try:
        response = client.get_secret_value(SecretId=secret_name)
        secret_data = response['SecretString']
        secret = json.loads(secret_data)
        return secret
        
    except Exception as e:
        print("Error in retrieving secret:", e)

4.Instance permission

In my particular case, the instance has to be able to invoke the lambda function therefore I added appropriate permission to a policy which in turn I have attached to the instance profile.

5. Invoke the function

At this point the pseudo fragment I posted above simply become a lambda invocation and that's it:

ARGON_HASHED_PWD=some-existing-linux-function($PLAIN_PWD)

become

ARGON_HASHED_PWD=$(aws lambda invoke --function-name encrypt-secret --payload '{"secret_name": "my/secret/pwd"}' .... lot of other parameter)

Answered By - Leonardo