Category: Vendor Products

Derrick Smith Header Image
Recent Posts

As an IT Director, I spend most of my time referencing information found inside the IT Service Management (ITSM) system and one of the better open source ITSM tools in the marketplace is GLPI. The feature set is massive and I’ve used this tool on and off for the last decade for IT inventory and helpdesk. On the security side, Single Sign On is paramount for authentication consistency and I tend to use SAML in most applications.

GLPI comes with several authentication options, including CAS, but it doesn’t include a way to authenticate users through SAML. As you may know, “Security Assertion Markup Language (SAML) is an open standard that allows identity providers (IdP) to pass authorization credentials to service providers (SP).” (Varonis) In our case GLPI is the SAML service provider and any number of SAML identity providers (Azure AD, ADFS, Okta, One login, Google IdP) can be used to authenticate users.

The good news is that GLPI has a plugin API that can be used to augment the software with additional features. Having worked with GLPI previously on my Nagios Event Handlers project, I was ready to dig in and create the functionality. Using the API.

The plugin relies on OneLogin’s SAML PHP Toolkit, which is a PHP library that helps web developers create a SAML Service Provider in existing PHP software. The plugin stores the SAML configuration in the database and the software takes several inputs that I mapped to fields in the plugin setup.

GLPI Plugin Settings

You can find the plugin at the GLPI Plugins website or at my repo on github. Please submit any bugs or feature requests to the github repo. If you found this plugin helpful, please leave a comment below.

One of the better features of Palo Alto’s PAN-OS is the ability to define dynamic block lists as firewall objects. This feature enables the firewall to poll a HTTP/HTTPS source for a list of IPs or URLs which can then be acted upon by firewall policies. There are a number of publicly available IP lists and threat intelligence feeds that can be consumed by these objects on the Palo Alto firewall but generating IP lists from internal sources, such as IDS/IPS can be cumbersome. Depending on the type of threats you encounter, it may be beneficial to block IP addresses or URLs picked up by your IDS systems. To solve this problem I wrote a python script that can be run at the command line to generate a list of IP addresses that can be consumed by Palo Alto.

Use this on a SIEM to dynamically block threats that match IDS/IPS rules. All data generated from the script is stored in a local JSON file and an IP list is outputted to a text file at the designated location. The JSON file stores current, historical and excluded IPs and the script logs information to a syslog file and to the terminal if verbose is enabled.

I wanted a relatively low maintenance solution and the initial use case for the was to temporarily block reconnaissance traffic such as port scanning. Given this, I used a penalty system to increment the time an IP is blocked each time the IP is seen. Of course, you could utilize the script to block an IP indefinitely also.

You can find the tool on Github at https://github.com/derricksmith/siem/tree/master/av2pa and the command usage is below. If you find this script useful, please drop me a comment below or on Github.

Usage

Arguments

-a, –action           Define the action [add,remove,exclude,clear]

-v, –verbose        Output to terminal

-i, –ip                  IP address (required when action = add,remove,exclude)

-p, –penalty        Penalty for IP address

Actions

add

Add an IP address to the block list. If no penalty is defined the penalty is incremented each time the IP address is added until it reaches 16(indefinite).

remove

Remove an IP address from the block list. Removes the IP from the current, history and exclude dictionaries.

exclude

Add an IP to the exclude list. Excluded IPs will not be processed.

clear

Removes all IPs from the current, history and exclude dictionaries.

cycle

Cycles the blocklist and checks IPs and Penalty times. If penalty has been reached, IPs are removed from the blocklist. IPs with a penalty of 16 are not removed. Suggest running this on a cron job to periodically cycle through the IP list.

Penalties

1 = 1 minute

2 = 5 minutes

3 = 10 minutes

4 = 15 minutes

5 = 30 minutes

6 = 60 minutes (1 hour)

7 = 180 minutes (3 hours)

8 = 360 minutes (6 hours)

9 = 720 minutes (12 hours)

10 = 1440 minutes (1 day)

11 = 4320 minutes (3 days)

12 = 10080 minutes (7 days)

13 = 20160 minutes (14 days)

14 = 43200 minutes (30 days)

15 = 525600 minutes (1 year)

16 = indefinite

Run at command line

Block 1.2.3.4 temporarily, penalty is incremented

./av2pa.py -a add -i 1.2.3.4

Block 1.2.3.4 for 1 day

./av2pa.py -a add -i 1.2.3.4 -p ’10’

Block 1.2.3.4 indefinately

./av2pa.py -a add -i 1.2.3.4 -p ‘~’

Remove 1.2.3.4

./av2pa.py -a remove -i 1.2.3.4

Exclude 1.2.3.4

./av2pa.py -a exclude -i 1.2.3.4

Run Cron

* * * * * python path/to/your/av2pa.py

Script Settings

blocklist

Set location of the blocklist (e.g ‘/var/www/block_inbound.txt’). This file should be https accessible if you intend to setup a Palo Alto External Dynamic List

log

Set a log file location (e.g ‘/var/log/block_inbound’)

timezone

Set the local timezone(e.g ‘America/Denver’)


Gone are the days when a customer cannot obtain logs related to the security operations of their hosted environments. Many vendors are coming around to the idea of providing their logs for analysis with on-premise solutions. Some are even providing APIs to directly provide those logs. Even SaaS providers are starting to offer log exports.  Proofpoint is one vendor that makes pulling logs easy.

For those unaware, Proofpoint provides hosted spam filtering among other email related services including the anti-phishing solution called Targeted Attack Protection (TAP). The information included in the TAP logs, such as, phishing messages allowed and/or blocked are invaluable for security operations and could help organizations more easily identify and possibly prevent further attack. But the real value of this information is when it can be correlated with other events to gain better insight into the attack.

Proofpoint’s TAP solution includes a webservice API that can be used to gather system logs. The API is fully documented here and they have even created a basic script to help you export logs accordingly.  For my purposes, the script needed to be modified to interact with the Alienvault USM SIEM.

To accomplish this I needed to modify the script to append new logs to a single log file.  After a few simple changes to the script I was able to get it going with a cron job set to run every minute.  I then configured Alienvault to look for that log via the proofpoint-tap plugin.  Finally, I added a logrotate entry to clear the log accordingly.  Below are the changes needed. I’ve also uploaded to Github for those interested.

Modify /etc/ossim/agent/plugins/proofpoint-tap.cfg

location=/var/log/ossim/proofpoint-tap.log

Create a logrotate entry for the plugin.
/etc/logrotate.d/proofpoint-tap

/var/log/ossim/proofpoint-tap.log
{
# save 4 days of logs
rotate 4
# rotate files daily
daily
missingok
notifempty
compress
delaycompress
sharedscripts
# run a script after log rotation
postrotate
invoke-rc.d rsyslog rotate > /dev/null
endscript
}

Copy the following shell script to your SIEM. Modify the file with your principal and secret variables.  Then create a CRON job to run every minute.
* * * * * sh /opt/proofpoint-tap/retrieve-tap-siem-logs.sh

#author          :Proofpoint
#date            :2017-04-07
#version         :1.1
#usage           :bash retrieve-tap-siem-logs.sh

############################################################
#Version History
# 1.0    Initial Release
# 1.1    Some old versions of the 'date' command didn't support
#         ISO8601-formatted timestamps. Fixed to be friendlier to
#        those old versions.
# 1.2	 Modified by Derrick Smith - modified for alienvault directories, logs to single /var/log/ossim/proofpoint-tap.log file
############################################################


#=============USER CONFIGURABLE SETTINGS===================#
# The service principal and secret are used to authenticate to the SIEM API. They are generated on the settings page of the Threat Insight Dashboard.
PRINCIPAL=""
SECRET=""

# Determines which API method is used. Valid values are: "all", "issues",
# "messages/blocked", "messages/delivered", "clicks/permitted", and "clicks/blocked"
ACTION="all"

# Determines which format the log is downloaded in. Valid values are "syslog" and "json".
FORMAT="syslog"

# Determines where log file are downloaded. Defaults to the current working directory.
LOGDIR="/var/log/ossim"
#=============END USER CONFIGURABLE SETTINGS===================#

LASTRETRIEVALFILE="$LOGDIR/lastretrieval"
LOGFILESUFFIX="proofpoint-tap.log"
ERRORFILESUFFIX="tap-siem.error"
TMPFILESUFFIX="tap-siem.tmp"
CURRENTTIME_ISO=`date -Iseconds | tr "T" " "`
echo $CURRENTTIME_ISO
CURRENTTIME_SECS=`date -d "$CURRENTTIME_ISO" +%s`
echo $CURRENTTIME_SECS
function interpretResults {
        local STATUS=$1
        local EXITCODE=$2
        local TIME_ISO=$3
        local TIME_SECS=$4
        if [[ $EXITCODE -eq 0 ]] && [[ $STATUS -eq 200 ]]; then
                echo $TIME_ISO > $LASTRETRIEVALFILE
                cat "$LOGDIR/$TIME_SECS-$TMPFILESUFFIX" >> "$LOGDIR/$LOGFILESUFFIX"
				rm "$LOGDIR/$TIME_SECS-$TMPFILESUFFIX"
                echo "Retrieval successful. $LOGDIR/$LOGFILESUFFIX created."
                return 0
        fi
        if [[ $EXITCODE -eq 0 ]] && [[ $STATUS -eq 204 ]]; then
                echo $TIME_ISO > $LASTRETRIEVALFILE
                rm "$LOGDIR/$TIME_SECS-$TMPFILESUFFIX"
                echo "Retrieval successful. No new records found."
                return 0
        fi

        mv "$LOGDIR/$TIME_SECS-$TMPFILESUFFIX" "$LOGDIR/$TIME_SECS-$ERRORFILESUFFIX"
        echo "Retrieval unsuccessful. $LOGDIR/$TIME_SECS-$ERRORFILESUFFIX created."
        logger -p user.err "Failed to retrieve TAP SIEM logs. Error in $LOGDIR/$TIME_SECS-$ERRORFILESUFFIX."
        return 1
}

function retrieveSinceSeconds {
        SECONDS=$1
        STATUS=$(curl -X GET -w %{http_code} -o "$LOGDIR/$CURRENTTIME_SECS-$TMPFILESUFFIX" "https://tap-api-v2.proofpoint.com/v2/siem/$ACTION?format=$FORMAT&sinceSeconds=$SECONDS" --user "$PRINCIPAL:$SECRET" -s)
        EXITCODE=$?
        interpretResults $STATUS $EXITCODE "$CURRENTTIME_ISO" "$CURRENTTIME_SECS"
}

function retrieveSinceTime {
        TIME=$1
        STATUS=$(curl -X GET -w %{http_code} -o "$LOGDIR/$CURRENTTIME_SECS-$TMPFILESUFFIX" "https://tap-api-v2.proofpoint.com/v2/siem/$ACTION?format=$FORMAT&sinceTime=$TIME" --user "$PRINCIPAL:$SECRET" -s)
        EXITCODE=$?
        interpretResults $STATUS $EXITCODE "$CURRENTTIME_ISO" "$CURRENTTIME_SECS"
}

function retrieveInterval {
        START_ISO=$1
        END_ISO=$2
        END_SECS=$3
        STATUS=$(curl -X GET -w %{http_code} -o "$LOGDIR/$END_SECS-$TMPFILESUFFIX" "https://tap-api-v2.proofpoint.com/v2/siem/$ACTION?format=$FORMAT&interval=$START_ISO/$END_ISO" --user "$PRINCIPAL:$SECRET" -s)
        EXITCODE=$?
        interpretResults $STATUS $EXITCODE "$END_ISO" "$END_SECS"
}

if ! [[ -f $LASTRETRIEVALFILE ]]; then
        echo "No interval file found. Retrieving past hour's worth of data."
        retrieveSinceSeconds 3600
else
        LASTRETRIEVAL_ISO=`date -f "$LASTRETRIEVALFILE" -Iseconds `
        LASTRETRIEVAL_SECS=`date -d "$LASTRETRIEVAL_ISO" +%s`
    (( DIFF=$CURRENTTIME_SECS - $LASTRETRIEVAL_SECS ))

        if [ $DIFF -lt 60 ]; then
                echo "Last retrieval was $DIFF seconds ago. Minimum amount of time between requests is 60 seconds."
                logger -p user.err "Last retrieval was $DIFF seconds ago. Minimum amount of time between requests is 60 seconds. Exiting."
                exit 0
        fi

        if [ $DIFF -gt 43200 ]; then
            echo "Last successful retrieval of SIEM logs was $DIFF seconds ago. Maximum amount of time to look back is 43200 seconds (12 hours). Resetting last interval. Information older than 12 hours will not be retrieved."
                        logger -p user.warn "Last successful retrieval of SIEM logs was $DIFF seconds ago. Maximum amount of time to look back is 43200 seconds (12 hours). Resetting last interval. Information older than 12 hours will not be retrieved."
                        ((LASTRETRIEVAL_SECS=$CURRENTTIME_SECS-43140))
                        LASTRETRIEVAL_ISO=`date -d @$LASTRETRIEVAL_SECS -Iseconds`
                        (( DIFF= $CURRENTTIME_SECS - $LASTRETRIEVAL_SECS ))
    fi

        if [ $DIFF -gt 3600 ]; then
                echo "Last retrieval was $DIFF seconds ago. Maximum amount of allowable time for one request is 3600 seconds. Will split into several requests."
                START_ISO=$LASTRETRIEVAL_ISO
                START_SECS=$LASTRETRIEVAL_SECS
                while [ $DIFF -gt 3600 ]; do
                        ((END_SECS=$START_SECS+3600))
                        END_ISO=`date -d @$END_SECS -Iseconds`
                        (( DIFF=$CURRENTTIME_SECS - $END_SECS ))
                        retrieveInterval $START_ISO $END_ISO $END_SECS
                        START_SECS=$END_SECS
                        START_ISO=$END_ISO
                done
                LASTRETRIEVAL_ISO=$END_ISO
        fi

        if [ $DIFF -le 3600 ]; then
                retrieveSinceTime $LASTRETRIEVAL_ISO
        fi
fi

Be sure to reconfigure Alienvault with the alienvault-reconfig command. Afterwards, enjoy all the Proofpoint goodness inside Alienvault.