Thursday, November 28, 2013

Vulnerability Data into ELSA

At Security BSides Augusta I released a script that would take a variety of vulnerability scanner data and import it into ELSA. I have been meaning to get a blog post about its usage but just haven't gotten around to in. With a couple days off of the holiday, here it is.

First the script is called VAtoELSA.py and you get find it at my Github account. I have created Nessus and OpenVAS to ELSA scripts in the past. This script combines all of the above plus it adds support for NMap and Nikto all in one place.

The script is very straight forward to use. Simply give it a Nessus, OpenVAS, NMap, or Nikto output report in XML format and an ELSA IP address and you should be off to the races.


$ python VAtoELSA.py –i report.nessus –r nessus –e elsa_ip


Before running the script for the first time you will want to create the XML and SQL file for ELSA to recognize the syslog output the script provides.  The -x and -s option will automatically create it for you and output them to files.


"Usage: VAtoELSA.py [-i input_file | input_file=input_file] [-e elsa_ip | elsa_ip=elsa_ip_address] [-r report_type | --report_type=type] [-s | --create-sql-file] [-x | --create-xml-file][-h | --help]"


As always I welcome feedback and would be happy to add any more vulnerability assessment tools to it if you have recommendations. I would ask that you send me a sanitized output report file since I might have limited access to the tool.

Wednesday, October 23, 2013

ELSA Parsing Video Part 1

I have decided to do a couple videos on creating parsers for ELSA. This first one is on creating the parsers for syslog-ng. Forgive the text size on my terminal. You will probably have to go full screen to see all the details.




Thursday, May 30, 2013

My Presentation on Risk Assessments

Recently at the Jack Henry Smokey Mountain User Group meeting I did a presentation on risk assessments. You can find a link to a video I created here or watch it below.

video


Here is the presentation.

Monday, May 20, 2013

OpenVAS 6 and CentOS 6.4

On April 17th OpenVAS 6 was released. The OpenVAS folks have provided install instructions for installation on a variety of Red Hat flavors that can be found here.  Sometime soon they will be releasing a virtual appliance with everything up and running. However, in the mean time I wanted to try   out the new release.
The install instructions for CentOS did not quite get me there. After I followed the instructions on the OpenVAS site I hit the Greenbone UI at https://localhost:9392. When I tried to log in a red error message came up that said the OMP service was down. OMP stands for OpenVAS Management Protocol. It is for doing things like starting / stopping scans and creating / deleting users. In order to save some of you the trouble I created a complete list of commands to get OpenVAS 6 up and running on CentOS 6.4.


# wget -q -O - http://www.atomicorp.com/installers/atomic |sh
# yum install openvas
# openvas-setup
# openvas-certdata-sync
# openvasmd --rebuild
# openvasmd

Note the 'yum install openvas' and 'openvasmd --rebuild' commands will take a few minutes. So be patient with them. I look forward to working with the new version and seeing what it can do.

Saturday, March 16, 2013

pfSense into ELSA


Introduction
One good piece of open source software deserves another right? It is time for a match made in heaven: pfSense and ELSA. pfSense is my favorite open source firewall. I run it in multiple places and it has always been rock solid. If you have read my blog at all you know my appreciation of ELSA. Therefore, I thought it would be a good post to combine the two.

Prepping pfSense
Before you start firing off syslogs to your ELSA server you will need to fix a small issue with pfSense. Apparently, pfSense logs to a binary data file which is then sent to tcpdump to parse. Something in the parsing causes issues with newlines. Single log entries will be sent as two syslogs making parsing them in ELSA (or any other parser) extremely difficult. Fortunately, this site has a simple script and fix for the issue. A PHP script is used to modify the filter file in pfSense. It uses SED in a clever way to weed out all the newlines in the log entries giving you one syslog per entry. You can check out the link above for more information but, I have included the script for convenience. Before you run the script you will probably have to remount the file system as read/write. If you haven't already done so enable SSH on you pfSense firewall to perform these actions.

Mount the file system as read/write

# /etc/rc.conf_mount_rw

Use vi on the firewall to create this file. Make sure to enclose script in PHP brackets

$filter=file_get_contents ('/etc/inc/filter.inc');
$filternew =
    str_replace(
        "-ttt -i pflog0 | logger -t pf -p local0.info",
        "-ttt -i pflog0 | /usr/bin/sed -e 'N;s/\\\\n //;P;D;' | logger -t pf -p local0.info",$filter);
if (strcmp($filter, $filternew) !=0) {
    file_put_contents('filter.inc.new',$filternew);
    file_put_contents('filter.inc.org',$filter);
}
Execute the script to create the new filter and move it over. I had to reboot my device for this to take effect.

# chmod +x chgfilter.php
# php chgfilter.php
# mv filter.inc.new /etc/inc/filter.inc


Now your pfSense firewall should be syslogging correctly. Just point it at your ELSA server and you should be done configuring it.

Creating the patterndb XML file
First kudos to InfoSec matters blog on configuring Vyatta for ELSA. It made my work a lot easier here. Essentially I used two preexisting ELSA classes, FIREWALL_ACCESS_DENY and FIREWALL_CONNECTION_END, for parsing the pfSense firewall logs. The FIREWALL_CONNECTION_END log is for pass traffic and the FIREWALL_ACCESS_DENY log is for blocked traffic. You might have to turn on logging in pfSense for whatever rules you want to send to syslog. Below is the XML you can add to the patterndb.xml file.
After you have added that code simply restart syslog-ng and all your pfSense firewall logs should be parsing correctly in ELSA.

# service syslog-ng restart
Conclusion
I would like to get some dashboards up soon. The code for the above examples is also on Github. Hopefully, that will be another blog post in the near future. As always, I welcome your comments and feedback.

Saturday, March 9, 2013

Process Data to ELSA

Process Data to ELSA

Introduction

It seems the useful things ELSA can help a security professional with are endless. A couple of weeks ago I thought it would be interesting to include a snapshot of running process data from Windows hosts in ELSA. Therefore, I wrote a simple script in Python to input a list of hosts, by IP address, capture all the running processes via WMI, and send them to the ELSA via syslog. If you want to use this script but do not have Python installed on a machine, I created a Windows executable.  Check my GitHub account to download the Python script/Windows executable, MySQL file to alter the ELSA database schema and XML file for appending patterndb.xml  used with syslog-ng. First let's take a look at how to configure ELSA for use with the script and then we will look at what you can do with it.

Setup

For this example I will assume you downloaded the Windows executable version of my script. You will also want to download patterndb_process.xml and PROCESS_db_setup.sql. First cut all of the text from patterndb_process.xml and add it to patterndb.xml as shown.


Next, update the MySQL database schema and restart syslog-ng since you modified patterndb.xml. 

# mysql < PROCESS_db_setup.sql
# service syslog-ng restart

Now your ELSA installation should be configured and ready to accept syslogs containing Windows process data. 

Go to the Windows host which you will want to run the script from. It might be helpful to create a separate folder. Copy the script into that folder and create a text file containing the list of hosts which you plan on analyzing process information. The script reads one address per line. The script will require a hosts file with which to pull syslog data from (-i), a Windows username (-u) and password (-p), and the ELSA server IP address (-l). Here is a screenshot on how to run the script from Windows.


You can also include a file called scanID.txt. The script might throw an error the first time looking for the file, however if you run it again it with the same parameters it should create the file automatically and work fine. The idea behind the scanID.txt file is to give each scan or process data pull a unique scan ID number to more easily track them in ELSA. Every subsequent time you run the script, this ID will increment. I designed the script to work well as a scheduled task in Windows. 



Analysis

I tried to capture interesting information about processes that would be helpful to an analyst or security professional. Data such as process name, operating system type, process ID, executable directory, parent process ID and parent process name (if available) are all collected. 



I think the opportunities from a security perspective can be pretty interesting. For example suppose an analyst wants to see all the places where Java is currently running in the environment. You could simply do a query in ELSA such as 'class=PROCESS +java'. 


Furthermore, suppose an attacker is hiding a backdoor in an unusual directory. You can search ELSA for process names and sort by directory. 



In the example I searched for cmd.exe and noticed it was running in three different places. However, in the last log entry it is not running in the standard directory. Hmm.....


There is a lot more interesting data that can be mined from taking periodic snapshots of processes. An example might be searching for anomalies in parent processes. Searching for a parent process of iexplorer.exe might yield some interesting results. I would love to hear other cool ideas on how to use this data. As always I welcome you feedback.


Tuesday, February 19, 2013

Nessus Dashboard

It is an understatement to say that data visualization is important. Fortunately, ELSA comes with dashboard functionality using the Google Visualization API. I have created a sample Nessus dashboard. It uses the Nessus class I created in the previous blog post. I did not create this to handle every functional need you might have in your vulnerability management program. However, I did create some charts with, what I would consider, important information that is worth highlighting. I also use a sampling of the charts available. As always you can get this dashboard here.

After you have downloaded and saved off nessus_dasboard.txt from my Gitub repo you will want to import it into ELSA. To do this, open up the nessus_dashboard.txt text file and copy all of the text in it (Ctrl-A). Next, from the ELSA console click the ELSA menu button in the top left and select Dashboards.

You will want to select Create/import new dashboard. A box comes up that will prompt you for a Title, Alias, Auth and then the option to paste your dashboard in for importing. The import box looks small, but don't worry if you just paste all the text from the dashboard in it will work fine.

A new dashboard will have been created. Now under the Action drop down box select View and your Nessus dashboard should open in a new tab. The dashboard I created has six charts associated with it. I will briefly cover each one. The top chart is a column chart that shows the ports found open in a Nessus scan. I found this information by querying for the Nessus ID (nid) 22964. I maximized this chart by selecting the 'Maximize' option when editing it. Notice the port numbers show up on the bars and the frequency of occurrence is along the y-axis.


Next I have a pie chart that shows the count of vulnerabilities by severity. In the Nessus world severity is determined by CVSS score. Severity 3 is the highest and 0 is the lowest. This maybe a bug or more likely I did something wrong, but I could not get my title to show on this pie chart. If anyone knows how to fix this please let me know. In the chart below you see that the most common severity is 1 (blue) followed by 3 (red), 0 (green) and 2 (orange). 


Directly below the pie chart is an area graph that shows high severity Adobe Reader vulnerabilities by host IP address. Similar concept below it but this time it is a bar chart for Oracle Java vulnerabilities. Notice how when you hover over each bar or column the host IP address shows up. This can be helpful if you have a lot of hosts. 


A couple things to note here. I have a single query per chart. There is nothing stopping you from including multiple queries on one chart as long as they have the same 'groupby:' parameter. It might make sense to do a single chart with multiple third party vulnerabilities graphed on it. You can accomplish this when you edit the charts by adding another query. I have an example of how this might look.


On the ELSA wiki Martin calls out a table chart that is somewhat hidden among the 'More' category when selecting chart types. This table works well if you have if you have a lot of text to display. I have created a table chart below that lists high severity Microsoft Security Bulletins, their associated short summary and their frequency.


Finally, I saved the coolest chart type for last; the gauge chart. This chart is really only practical when the 'groupby' category of your query has less then 10 items. Anymore then that and it is very difficult to see the gauges. Unfortunately, I may have run into the same bug as the pie chart above because this chart also will not show its title. Again, if you know why please let me know and I will get it fixed. CVSS Base score can be a helpful indicator of a vulnerability severity. Nessus also provides a helpful field on whether there is a public exploit available. The gauges below list the number of vulnerabilities with a high severity rating (CVSS Base > 7) and a public exploit available grouped by Nessus ID. The areas of green, orange and red on the gauge can be dynamically assigned to values of your choosing. Very cool stuff. 



Feel free to experiment and if you have any good suggestions I would love to hear about them. Check back on my Github account. I will post a similar dashboard soon that goes with my OpenVAS class.