AWS Client VPN with SimpleAD

In this post, I’m going to guide how to set up AWS client VPN from scratch including Simple AD deployment.

AWS Client VPN can be used to connect to private segment directly from your client. It is well documented here in official document “AWS Client VPN Administrator Guide“.

In VPN settings, there are two main part you need to consider first:

  • Authentication … AWS Client VPN supports two types:
    • Active Directory Authentication
    • Mutual (Certificate) Authentication
  • Authorization
    • Security Groups
    • Network-based Authorization (work with Active Directory Authentication)

In my opinion, Active Directory Authentication is more flexible and intuitive compared to Mutual Authentication. It doesn’t support MFA yet, but it provides user/password authentication as well as it allows specific groups of users in AD to be able to connect to SSLVPN, which is a requirement of most clients.

Setup procedure is below:

  1. Generate certs and keys using easy-rsa, and register them on ACM
  2. Procure Simple AD
  3. Create users and groups on Simple AD
  4. Procure Client VPN Endpoint
  5. (Option) Configure a web server for connection test
  6. Configure OpenVPN client

By the end of this guide, the test environment looks like below:


1. Generate Certificate and Keys

You need to generate certificate and keys for servers to process client vpn first. You can follow the official steps here. It is not required to generate client certificate/key because we use Active Directory Authentication.

{
 git clone https://github.com/OpenVPN/easy-rsa.git
 cd easy-rsa/easyrsa3
 ./easyrsa init-pki
 ./easyrsa build-ca nopass
 ./easyrsa build-server-full lab_server nopass
 mkdir /temp_folder
 cp pki/ca.crt /temp_folder/
 cp pki/issued/lab_server.crt /temp_folder/
 cp pki/private/lab_server.key /temp_folder/
 cd /temp_folder/
 }

Once they are generated, register them into the AWS Certificate Manager(ACM). Please note you need to register these to the region you are going to have your VPN connection.

aws acm import-certificate --certificate file://lab_server.crt --private-key file://lab_server.key --certificate-chain file://ca.crt --region eu-west-1

If it returns arn, you are successfully registered certificate/key on ACM.


2. Procure Simple AD

Simple AD is not available yet in all regions. You can check the availability here.

First, select “Simple AD”, and select your VPC and subnet you want this service.

Once created, wait for a ew minutes till Directory service is ready. Note the DNS address listed in Directory detail, this information is required later to have management server join this domain.


3. Create Users and Groups for VPN access

In order to manage this AD, I procure another Windows server(management server) in public subnet.

For DNS configuration, you can either change DHCP option of your subnet or set DNS server statically in the servers.

After restart, reconnect to the server, this time with domain administrator account – password is the one you set during Simple AD setup. Install RSAT.

Launch “Active Directory Users and Computers”. If you logged in with domain administrator account, you should be able to see your domain is listed. I created two users “shogo.kobayashi” and “john.smith”, and only “shogo.kobayashi” is a member of “SSLVPN_USERS” group.


4. Procure Client VPN Endpoint

In Create Client VPN Endpoint wizard, you need to specify IPv4 CIDR which should be different from your existing VPC.

  • Server certificate ARN … Select arn, which you received in step 1.
  • Authentication Options … Select “Use Active Directory authentication”
  • Directory ID … select Directory ID you created on step 2.
  • DNS Server 1/2 IP Address … Use DNS IP address of your Simple AD

Associate this endpoint with your subnet in order to use it. Note that you will be charged once you associate endpoint with subnet. I used private subnet for this.

  • VPC … VPC you want to use this VPN Endpoint in
  • Subnet … Subnet you want to use this VPN Endpoint in

This is to authorize which network is reachable for each group of users. In this lab, I created only one rule that allows VPN connection to communicate anyone in the network as long as the authenticated user is in “SSLVPN_USERS” AD security group.

  • Destination network to enable … Which network this group of users can access
  • Grant access to … Allow access to users in a specific Active Directory group
  • Active Directory Group ID … SID of the SSLVPN_USERS group in AD

5. (Optional) Setup web server for connection test

We are going to setup web server for connection test. We use VPC endpoint to retrieve httpd packages because we want this server to be in private subnet, and hence there is no direct internet connectivity.

And minimum setup to bootstrap web server.


6. Configure OpenVPN Client

You can download configuration file from AWS console.

Launch OpenVPN client of your choice, and use the configuration file you just downloaded. Note you need to have a root CA certificate, which you generate in step 1, in the same folder you have your configuration file.

Now you should …

  • Be able to access web server if you login with user who is a member of “SSLVPN_USERS”, and
  • Not be able to access web server if you login with user who is NOT a member of “SSLVPN_USERS”.

Terraform Basics – AWS / GCP / Aliyun

What is Terraform?

It’s a tool to create, manage infrastructure as a code. Infrastructure includes not only servers but also network resources –e.g. DNS, loadbalancer. The benefit you can get is as follows:

  • Versioning of your changes
  • Management of all services as a whole (orchestration)
  • Single management of multi-cloud platform
  • and so on …

Let’s Try

I make two compute instances and make modifications, and finally delete all resources to demonstrate how to use Terraform.

  • on AWS (amazon web service), GCP (Google cloud platform) and Aliyun (Alibaba cloud)
  1. Install Terraform
  2. Get credentials
  3. Create servers
  4. Modify servers
  5. Delete all procured resources
Continue reading “Terraform Basics – AWS / GCP / Aliyun”

Python 100 project #42: Slack Bot – AWS EC2 list

Following up the previous project, I created Slack bot to get EC2 instance list (of all regions) in one shot.

So now no need to open the terminal to invoke the command every time. Just need to ask Slack “/100p ec2 list” and the result is posted.

I used AWS API Gateway to receive the slash command from Slack. So it is easy to add functions.

 

Output Example:

 

Here is the code:

This is the receiver code which is invoked when Slack slach command post request to API Gateway.

from base64 import b64decode
import json
import os
from urllib.parse import parse_qs
import logging

import boto3


ENCRYPTED_EXPECTED_TOKEN = "kms_base64encodedkey="

kms = boto3.client('kms')
expected_token = str(kms.decrypt(CiphertextBlob = b64decode(ENCRYPTED_EXPECTED_TOKEN))['Plaintext'], 'utf-8')


logger = logging.getLogger()
logger.setLevel(logging.INFO)

def lambda_handler(event, context):
    req_body = event['body']
    params = parse_qs(req_body)
    print("received data...", params)
    token = params['token'][0]
    if token != expected_token:
        logger.error("Request token (%s) does not match exptected", token)
        raise Exception("Invalid request token")

    user = params['user_name'][0]
    command = params['command'][0]
    channel = params['channel_name'][0]
    if 'text' in params.keys():
        command_text = params['text'][0]
    else:
        command_text = ''
    response_url = params['response_url'][0]
    arg = command_text.split(' ')
    
    sns = boto3.client('sns')
    SNS_CHANNEL = os.environ['SNS_CHANNEL']
    topic_arn = sns.create_topic(Name=SNS_CHANNEL)['TopicArn']
    message={"user_name": user, "command": command, "channel": channel, "command_text": command_text, "response_url": response_url}
    message=json.dumps(message)
    message=json.dumps({'default': message, 'lambda': message})
    response = sns.publish(
        TopicArn=topic_arn,
        Subject='/100p',
        MessageStructure='json',
        Message=message
    )
    return { "text": "received command - %s . Please wait for a few seconds for the reply to be posted." % (command_text) }

 

And this is the actual code to post the result to the Slack.

import json
import sys

import boto3
import requests

def get_regions(service):
    credential = boto3.session.Session()
    return credential.get_available_regions(service)


def list_ec2_servers(region):
    credential = boto3.session.Session()
    ec2 = credential.client('ec2', region_name=region)
    instances = ec2.describe_instances()
    servers_list = []
    for reservations in instances['Reservations']:
        for instance in reservations['Instances']:
            tags = parse_keyvalue_sets(instance['Tags'])
            state = instance['State']['Name']
            servers_list.append([region, instance['InstanceId'], tags['Name'], state])
    return servers_list


def parse_keyvalue_sets(tags):
    result = {}
    for tag in tags:
        key = tag['Key']
        val = tag['Value']
        result[key] = val
    return result


def lambda_handler(event, context):
    message = event['Records'][0]['Sns']['Message']
    try:
        message = json.loads(message)
        user_name = message['user_name']
        command = message['command']
        command_text = message['command_text']
        response_url = message['response_url']
        arg = command_text.split(' ')

        if arg[0] == 'ec2':
            resp = ec2_helper(arg[1:])
        # TODO else: statement for other functions

        # if response_type is not specified, act as the same as ephemeral
        # ephemeral, response message will be visible only to the user
        slack_message = {
            'channel': '@%s' % user_name,
            # 'response_type': 'in_channel',
            'response_type': 'ephemeral',
            'isDelayedResponse': 'true',
            'text': resp
        }
        print("Send message to %s %s" % (response_url, slack_message))
        header = {'Content-Type': 'application/json'}
        response = requests.post(response_url, headers=header, data=json.dumps(slack_message))
        if response.status_code == 200:
            print("Message posted to %s" % slack_message['channel'])
    except requests.exceptions.RequestException as e:
        print(e)
    except:
        e = sys.exc_info()[0]
        print("Something wrong happened...", e)

def ec2_helper(command):
    regions = get_regions('ec2')

    if command[0] == 'list':
        region_servers = []
        for region in regions:
            servers = list_ec2_servers(region)
            region_servers.extend(servers)

        msg = ""
        for server in region_servers:
            msg += '\t'.join(server)
            msg += "\n"
    # TODO else for other functions

    return msg

 

Python 100 project #42: AWS Data Post to Slack – Billing

Sometimes, I forgot to stop the AWS instance, and I’m lazy to check the running instance for a while. Then it suddenly comes clear when I receives the email from AWS for the billing of the previous month.

To avoid this surprise, I created Lambda function to post the estimated cost of the period every morning to the slack.

 

Output Example:

 

Here is the (main handler) code:

import datetime
import logging
import os

import boto3
import requests

import slack

SLACK_CHANNEL = os.environ['SLACK_CHANNEL']

logger = logging.getLogger()
logger.setLevel(logging.INFO)


def estimated_cost():
    response = boto3.client('cloudwatch', region_name='us-east-1')

    get_metric_statistics = response.get_metric_statistics(
        Namespace='AWS/Billing',
        MetricName='EstimatedCharges',
        Dimensions=[
            {
                'Name': 'Currency',
                'Value': 'USD'
            }
        ],
        StartTime=datetime.datetime.today() - datetime.timedelta(days=1),
        EndTime=datetime.datetime.today(),
        Period=86400,
        Statistics=['Maximum']
    )
    
    return get_metric_statistics['Datapoints'][0]['Maximum']


def lambda_handler(event, context):
    date = get_metric_statistics['Datapoints'][0]['Timestamp'].strftime('%Y-%m-%d')
    cost = estimated_cost()
    content = "Estimated cost is %s as of %s" % (cost, date)

    try:
        slack.post(content, SLACK_CHANNEL, context.function_name)
        logger.info("Message posted to %s, %s" % (SLACK_CHANNEL, content))
    except requests.exceptions.RequestException as e:
        logger.error("Request failed: %s", e)

 

Python 100 project #26: AWS EC2 Listing

One of the most frustrating things on AWS is scattered resources all over the regions. I’m not sure if there is any way to check all the resources in one single pain of the glass, but in order to check all the resources, it is quite time consuming. As I’m using AWS mainly for test purpose, there is a high possibility that I forget to stop/terminate unnecessary instances. I made a script to list all the EC2 instances in all regions along with its status.

 

Output Example:

$ python3 aws_ec2list.py 
eu-central-1    i-01a0eab7412d22cc4     ubuntu_test     stopped
us-east-1       i-065513a3b3b2ab8a7     AlexaBed        stopped
us-east-1       i-0fcde0e20853242c3     aws-cloud9-cloud9dev-2d8b27e8cbe2405c8ab08968f65c70e2   stopped
us-east-1       i-008f961394cc993c5     python_package  stopped

 

Here is the code:

import boto3

from data_source.aws_credentials import *

def gen_credential():
    credential = boto3.session.Session(
        aws_access_key_id=aws_access_key_id,
        aws_secret_access_key=aws_secret_access_key)
    return credential


def get_regions(service):
    credential = gen_credential()
    return credential.get_available_regions(service)


def list_ec2_servers(region):
    credential = gen_credential()
    ec2 = credential.client('ec2', region_name=region)
    instances = ec2.describe_instances()
    for reservations in instances['Reservations']:
        for instance in reservations['Instances']:
            tags = parse_keyvalue_sets(instance['Tags'])
            state = instance['State']['Name']
            print(f"{region}\t{instance['InstanceId']}\t{tags['Name']}\t{state}")


def parse_keyvalue_sets(tags):
    result = {}
    for tag in tags:
        key = tag['Key']
        val = tag['Value']
        result[key] = val
    return result


if __name__ == "__main__":

    regions = get_regions('ec2')
    for region in regions:
        list_ec2_servers(region)