syslog file questions

[eric266]

A client wants to know which webstes employees are going to, so I set up a quick log of http traffic. Is there a way to mass convert DST addresses to domain names? Heres what I'm getting:

Access Log *HTTP* IN=br0 OUT=eth1 SRC=10.16.66.252 DST=216.52.17.206 LEN=48 TOS=0x00 PREC=0x00 TTL=126 ID=5128 DF PROTO=TCP SPT=2047 DPT=80 WINDOW=65535 RES=0x00 SYN URGP=0

My background is hardware, i'm wondering if theres a simpler way of tracking employee browsing?

Thanks.

 

[fluxion]

you could certainly run it through a script to parse it out and reconstruct the data how you'd like, but as far as keep a running log that's nicely presented, im not sure how you'd go about it. it would be expensive to keep parsing it over and over, what would be nice is to have all the new info parsed and appended to another file.

actually if there's a 1-to-1 correspondance between the number of lines in the original file, and the number of lines in the parsed file, you could simply count the number of lines (using wc -l), and tail -n the original file (where n is the difference between the number of lines in the parsed log and the original log), run it through the parser and appended to the parsed log, and just run this process in a continuous loop. that is error prone though, as the original log might get a new entry after the wc -l comparison, and you'd end up losing a line. im sure there's easier ways though.

in any case, the parser could be something like this:

parseLog.pl

#!/usr/bin/perl

while (<>) {
  my @fields = split(/\s+/, $_);
  my $src_IP = (split(/=/, $fields[5]))[1];
  my $dst_IP = (split(/=/, $fields[6]))[1];
  my $dst_Hostname = convertIPtoHostname($dst_IP);
  print "SRC=$src_IP DST=$dst_Hostname $fields[3] $fields[4] $fields[15]\n";    
}

sub convertIPtoHostname {
  my $addr_IP = shift;
  my $addr_Hostname = `dig -x $addr_IP +short | head -1`; 
  chomp($addr_Hostname);
  chop($addr_Hostname);
  return $addr_Hostname;
}

testlog

Access Log *HTTP* IN=br0 OUT=eth1 SRC=10.16.66.252 DST=216.52.17.206 LEN=48 TOS=0x00 PREC=0x00 TTL=126 ID=5128 DF PROTO=TCP SPT=2047 DPT=80 WINDOW=65535 RES=0x00 SYN URGP=0
Access Log *HTTP* IN=br0 OUT=eth1 SRC=10.16.66.252 DST=216.52.17.206 LEN=48 TOS=0x00 PREC=0x00 TTL=126 ID=5128 DF PROTO=TCP SPT=2047 DPT=80 WINDOW=65535 RES=0x00 SYN URGP=0
Access Log *HTTP* IN=br0 OUT=eth1 SRC=10.16.66.252 DST=216.52.17.206 LEN=48 TOS=0x00 PREC=0x00 TTL=126 ID=5128 DF PROTO=TCP SPT=2047 DPT=80 WINDOW=65535 RES=0x00 SYN URGP=0
Access Log *HTTP* IN=br0 OUT=eth1 SRC=10.16.66.252 DST=216.52.17.206 LEN=48 TOS=0x00 PREC=0x00 TTL=126 ID=5128 DF PROTO=TCP SPT=2047 DPT=80 WINDOW=65535 RES=0x00 SYN URGP=0
Access Log *HTTP* IN=br0 OUT=eth1 SRC=10.16.66.252 DST=64.233.187.99 LEN=48 TOS=0x00 PREC=0x00 TTL=126 ID=5128 DF PROTO=TCP SPT=2047 DPT=80 WINDOW=65535 RES=0x00 SYN URGP=0

fluxion@purity:~/temp$ ./parseLog.pl  testlog 
SRC=10.16.66.252 DST=omniture.112.2o7.net IN=br0 OUT=eth1 DPT=80
SRC=10.16.66.252 DST=omniture.112.2o7.net IN=br0 OUT=eth1 DPT=80
SRC=10.16.66.252 DST=omniture.112.2o7.net IN=br0 OUT=eth1 DPT=80
SRC=10.16.66.252 DST=omniture.112.2o7.net IN=br0 OUT=eth1 DPT=80
SRC=10.16.66.252 DST=jc-in-f99.google.com IN=br0 OUT=eth1 DPT=80
fluxion@purity:~/temp$

the fields are the whitespace delimited feilds of the log entries, so adding new fields should be fairly obvious; adding $fields[0] $fields[1] to the print statement to print "Access Log" for example

 

[eric266]

Thanks, now I have something to do at work today!