Posted on

Cisco NSO – Create Service

In NSO, service is defined in YANG model. And once YANG model is defined and compiled, it will then be encoded to XML. There are few variations to define encode, such as “template only” and “python and template”. As name suggests, template is the most basic pattern, and it directly map the YANG model to XML. While with python some arbitrary operation can be configured based on YANG model before passing any values for XML encode.


Continue reading “Cisco NSO – Create Service”
Posted on

Fortigate config management in Github

After Github opened its free repository function to free users, I’m using Github private repository to store lots of my applications config file. I usually don’t use version management because they usually never changes after initial deployment. However, especially while I write blog post I need to make changes just to check the functionality. And sometimes I forgot to rollback config and need to check manually on the device.

In this post, I show you how to integrate Fortigate config backup script and Github API. And in the next post I will deploy them in CloudFunction so that it can be invoked by Fortigate automation stitch.

Continue reading “Fortigate config management in Github”
Posted on

Fortigate RestAPI Config Backup – FortiOS 6.0.4

Previously I wrote a post how to backup the Fortigate config using session based authentication. As per the API reference, this is considered legacy, and other authentication method –API token, is preferred. In this post, I demonstrate how to use FortiOS RestAPI with API token. And I will introduce how to parse current configuration.

I used FortiOS 6.0.4 to deploy this, and it is most likely not working with other version(especially 5.x).

The flow is as follows:

  1. Create access profile for API user
  2. Create API user in Fortigate
  3. Generate API token for API user
  4. Send request and get the backup config
Continue reading “Fortigate RestAPI Config Backup – FortiOS 6.0.4”
Posted on

Fortigate Config Change Notification

Whenever changes are made in configuration, Fortigate posts notification at Slack channel.

Fortigate automation is composed of three elements:

  1. automation trigger … available trigger -HA Failover, Config change, Log, IOC, High CPU, Conserve mode
  2. automation action … available action -Email, IP Ban, AWS lambda, Webhook
  3. automation stitch … Combination of trigger and action
Continue reading “Fortigate Config Change Notification”
Posted on

Python 100 project #51: Web scraping – Sunshine duration across countries

It’s said to be London is always covered with cloud. As I moved to London roughly two years ago, I realized it is actually not the case.

I searched the web and found very useful wikipedia page to list the (typically average) sunshine duration among each month of a year. This is a very basic task for web scraping (just 1 page).

 

Output:

 

 

Code:

# _*_ coding: utf-8 _*_

import csv
import re
from urllib.parse import urljoin

from bs4 import BeautifulSoup
import requests

base = "https://en.wikipedia.org"
target_url = base + "/wiki/List_of_cities_by_sunshine_duration"
req = requests.get(target_url, verify=False)

bs = BeautifulSoup(req.text, "html.parser")

tables = bs.find_all("table", {"class": "wikitable"})

cities_list = []

for table in tables:
    cities = table.find_all("tr")
    for city in cities:
        city_row = []
        # for text data collection. country_name, country_url, city_name, city_url
        for text_elem in city.find_all("td", style=re.compile("text-align:left")):
            elem_text = text_elem.get_text()
            city_row.append(elem_text)
            if text_elem.find("a"):
                city_row.append(urljoin(base, text_elem.find("a").get("href")))
            else:
                city_row.append("")
        # for sunshine hours data in monthly sequence.
        for data_elem in city.find_all("td", style=re.compile("background.*")):
            elem_text = data_elem.get_text()
            city_row.append(elem_text)

        cities_list.append(city_row)

with open('sunshine_hours.csv', 'w', newline='') as csvfile:
    writer = csv.writer(csvfile)
    writer.writerows(cities_list)

 

Posted on

Python 100 project #50: Get Audit Report on Slack

In this project, I extended the previous project “PDF to TXT”, and now it’s posted to Slack every day.

So in short, every day the sophos XG firewall sends the security audit report(PDF) to the python powered server, and the server interpret the PDF into the text, (and of course it selects the necessary part only) and post the daily summary on slack.

 

Output:

 

 

 

Code:

import base64
from io import BytesIO
from pprint import pprint
import tempfile

import aiosmtpd.controller
import asyncio
import email

import audit_reader
import slack


class CustomSMTPHandler:
    async def handle_DATA(self, server, session, envelope):

        msg = email.message_from_string(str(envelope.content,'utf-8'))

        for part in msg.walk():
            if part.get_content_type().startswith("application/pdf"):
                pdf_bytes = BytesIO(part.get_payload(decode=True))
                data = audit_reader.retrieve_data(pdf_bytes)
                slack.post(data, 'security_logs', envelope.mail_from)
        print('from:', envelope.mail_from)
        return '250 OK'


async def main(loop):
    handler = CustomSMTPHandler()
    server = aiosmtpd.controller.Controller(handler,hostname='XX.XX.XX.XX', port=XXXX)
    server.start()


if __name__ == '__main__':
    loop = asyncio.get_event_loop()
    loop.create_task(main(loop=loop))
    try:
        print("server running...")
        loop.run_forever()
    except KeyboardInterrupt:
        pass