使用 perl 和哈希数据结构从日志文件中打印记录

Question

我有以下脚本，最初由@zdim 编写，我对其进行了一些调整。

#!/usr/bin/perl

use warnings;
use strict;

my $file = "/home/tsec/prototype/logs/extractedlogs/cowrieresult.log";
open (LOG, $file);

# Assemble results for required output in data structure:
# %rept = { $port => { $usr => { $status => $freq } };

my %testhash;#new code
my %rept;
my ($ip, $port);

while (my $line = <LOG>)
{
    if ($line =~ /New connection/) {
        ($ip, $port) = $line =~ /New connection:\s+([^:]+):(\d+)/;
        #new code here
        if($ip){
                $testhash{$ip}++;
        }
        #end
        next;
    }

    my ($usr, $status) =  $line =~ m/login\ attempt \s+ \[ ( [^\]]+ ) \] \s+ (\w+)/x;
    if ($usr and $status) {
        $rept{$port}{$usr}{$status}++;
    }
    else { warn "Line with an unexpected format:\n$line" }
}
#close(LOG);
#open (LOG, $file);
#my $frequency = 0;
#while (my $line = <LOG>){
#       if($line =~ /login attempt/){

        #split string, get the ip and match it with original $ip
#       my ($testip) = (split /[\s,:\[\]\/]+/, $line)[-6];
        #print "$testip\n";
        #this two lines above print ips from login attempt line.
#       if($testip =~ /$ip/){
#               $frequency++;
#       }
        #elsif($testip =~ /^(?!$ip)/) {
                # stop frequency counter and start another one?
        #       print "$frequency\n";
        #       $frequency = 0;
        #}

#       }
#}
#print "$frequency\n";
#close(LOG);

#new code
print "ConnectionsOnIP\n";
foreach my $ip (sort keys %testhash){
        print "$testhash{$ip}\n";
}

print "\n";

#new code
print "Port,Status,AttemptOnPort,AttemptsOnIP,Malicious\n";
foreach my $ip (sort keys %testhash){
foreach my $port (sort keys %rept) {
    foreach my $usr (sort keys %{$rept{$port}}) {
        foreach my $stat ( sort keys %{$rept{$port}{$usr}} ) {
                if($port ne ""){
            print "$port,$stat,$rept{$port}{$usr}{$stat},$testhash{$ip}\n";
                }
        }
   }

}
}
#new code

正如所见，除了最后一个变量（AttemptsOnIP）之外，我希望获得目前正在工作的所需输出我希望 AttemptsOnIP 变量在某种程度上执行 AttemptsOnPort 的操作：

Port,Status,AttemptsOnPort,ConnectionsOnIP,Malicious
15853,failed,4,18
15853,succeeded,4,18
18693,failed,1,18
18942,failed,1,18
18942,succeeded,1,18
31130,succeeded,1,18
43041,failed,1,18
43041,succeeded,1,18
46321,failed,1,18
46321,succeeded,1,18
47417,failed,3,18
47417,succeeded,3,18
48713,failed,1,18
48713,succeeded,1,18
53653,failed,1,18
53653,succeeded,1,18
60563,failed,1,18
60563,succeeded,1,18

我创建了一个名为 testhash 的散列，并将 ip 变量传递给它以增加它。现在我想根据单行输出的 IP 增加这个散列变量。这是日志文件：

2016-05-02 10:20:56+0000 [SSHService ssh-userauth on HoneyPotTransport,14,183.3.202.172] login attempt [root/!@] failed
2016-05-02 10:20:57+0000 [SSHService ssh-userauth on HoneyPotTransport,15,183.3.202.172] login attempt [root/!@] failed
2016-05-02 10:20:57+0000 [SSHService ssh-userauth on HoneyPotTransport,14,183.3.202.172] login attempt [root/123456] succeeded
2016-05-02 10:20:58+0000 [SSHService ssh-userauth on HoneyPotTransport,15,183.3.202.172] login attempt [root/123456] succeeded
2016-05-02 10:43:32+0000 [cowrie.ssh.transport.HoneyPotSSHFactory] New connection: 183.3.202.172:55157 (172.17.0.5:2222) [session: 43283650]
2016-05-02 10:43:46+0000 [cowrie.ssh.transport.HoneyPotSSHFactory] New connection: 183.3.202.172:10319 (172.17.0.5:2222) [session: c7702f86]
2016-05-02 10:43:53+0000 [cowrie.ssh.transport.HoneyPotSSHFactory] New connection: 183.3.202.172:46321 (172.17.0.5:2222) [session: fe7bb804]
2016-05-02 10:43:57+0000 [SSHService ssh-userauth on HoneyPotTransport,17,183.3.202.172] login attempt [root/!@] failed
2016-05-02 10:43:58+0000 [SSHService ssh-userauth on HoneyPotTransport,17,183.3.202.172] login attempt [root/123456] succeeded
2016-05-02 10:43:59+0000 [cowrie.ssh.transport.HoneyPotSSHFactory] New connection: 183.3.202.172:18693 (172.17.0.5:2222) [session: d74eae96]
2016-05-02 10:44:02+0000 [SSHService ssh-userauth on HoneyPotTransport,18,183.3.202.172] login attempt [root/!@] failed
2016-05-02 10:44:03+0000 [cowrie.ssh.transport.HoneyPotSSHFactory] New connection: 183.3.202.172:31130 (172.17.0.5:2222) [session: 3bde7820]
2016-05-02 10:44:03+0000 [SSHService ssh-userauth on HoneyPotTransport,18,183.3.202.172] login attempt [root/123456] succeeded
2016-05-02 10:44:05+0000 [cowrie.ssh.transport.HoneyPotSSHFactory] New connection: 183.3.202.172:47417 (172.17.0.5:2222) [session: 3e177c02]
2016-05-02 10:44:06+0000 [SSHService ssh-userauth on HoneyPotTransport,19,183.3.202.172] login attempt [root/!@] failed
2016-05-02 10:44:09+0000 [SSHService ssh-userauth on HoneyPotTransport,19,183.3.202.172] login attempt [root/123456] succeeded
2016-05-02 10:44:10+0000 [SSHService ssh-userauth on HoneyPotTransport,21,183.3.202.172] login attempt [root/!@] failed
2016-05-02 10:44:11+0000 [SSHService ssh-userauth on HoneyPotTransport,21,183.3.202.172] login attempt [root/123456] succeeded
2016-05-02 10:44:13+0000 [SSHService ssh-userauth on HoneyPotTransport,20,183.3.202.172] login attempt [root/!@] failed
2016-05-02 10:44:14+0000 [SSHService ssh-userauth on HoneyPotTransport,20,183.3.202.172] login attempt [root/123456] succeeded
2016-05-02 11:06:55+0000 [cowrie.ssh.transport.HoneyPotSSHFactory] New connection: 183.3.202.172:13849 (172.17.0.5:2222) [session: b20915b6]
2016-05-02 11:07:06+0000 [cowrie.ssh.transport.HoneyPotSSHFactory] New connection: 183.3.202.172:61338 (172.17.0.5:2222) [session: cd38fe51]
2016-05-02 11:07:14+0000 [cowrie.ssh.transport.HoneyPotSSHFactory] New connection: 183.3.202.172:23048 (172.17.0.5:2222) [session: 01b12825]
2016-05-02 11:07:21+0000 [cowrie.ssh.transport.HoneyPotSSHFactory] New connection: 183.3.202.172:60563 (172.17.0.5:2222) [session: ad64232b]
2016-05-02 11:07:26+0000 [SSHService ssh-userauth on HoneyPotTransport,23,183.3.202.172] login attempt [root/!@] failed
2016-05-02 11:07:27+0000 [SSHService ssh-userauth on HoneyPotTransport,23,183.3.202.172] login attempt [root/123456] succeeded
2016-05-02 11:07:33+0000 [cowrie.ssh.transport.HoneyPotSSHFactory] New connection: 183.3.202.172:53653 (172.17.0.5:2222) [session: 9c48415b]
2016-05-02 11:07:41+0000 [SSHService ssh-userauth on HoneyPotTransport,26,183.3.202.172] login attempt [root/!@] failed
2016-05-02 11:07:47+0000 [SSHService ssh-userauth on HoneyPotTransport,26,183.3.202.172] login attempt [root/123456] succeeded
2016-05-02 11:12:25+0000 [cowrie.ssh.transport.HoneyPotSSHFactory] New connection: 183.3.202.172:18942 (172.17.0.5:2222) [session: a4dc4901]
2016-05-02 11:12:34+0000 [SSHService ssh-userauth on HoneyPotTransport,27,183.3.202.172] login attempt [root/!@] failed
2016-05-02 11:12:36+0000 [SSHService ssh-userauth on HoneyPotTransport,27,183.3.202.172] login attempt [root/123456] succeeded
2016-05-02 11:32:40+0000 [cowrie.ssh.transport.HoneyPotSSHFactory] New connection: 183.3.202.172:40091 (172.17.0.5:2222) [session: aeb36234]
2016-05-02 11:32:43+0000 [cowrie.ssh.transport.HoneyPotSSHFactory] New connection: 183.3.202.172:53505 (172.17.0.5:2222) [session: 9022c831]
2016-05-02 11:32:48+0000 [cowrie.ssh.transport.HoneyPotSSHFactory] New connection: 183.3.202.172:15131 (172.17.0.5:2222) [session: cf62fb9a]
2016-05-02 11:32:48+0000 [cowrie.ssh.transport.HoneyPotSSHFactory] New connection: 183.3.202.172:15853 (172.17.0.5:2222) [session: f2f6c254]
2016-05-02 11:32:50+0000 [SSHService ssh-userauth on HoneyPotTransport,28,183.3.202.172] login attempt [root/!@] failed
2016-05-02 11:32:52+0000 [SSHService ssh-userauth on HoneyPotTransport,28,183.3.202.172] login attempt [root/123456] succeeded
2016-05-02 11:32:55+0000 [SSHService ssh-userauth on HoneyPotTransport,29,183.3.202.172] login attempt [root/!@] failed
2016-05-02 11:32:55+0000 [SSHService ssh-userauth on HoneyPotTransport,30,183.3.202.172] login attempt [root/!@] failed
2016-05-02 11:32:56+0000 [SSHService ssh-userauth on HoneyPotTransport,30,183.3.202.172] login attempt [root/123456] succeeded
2016-05-02 11:32:57+0000 [SSHService ssh-userauth on HoneyPotTransport,31,183.3.202.172] login attempt [root/!@] failed
2016-05-02 11:32:59+0000 [SSHService ssh-userauth on HoneyPotTransport,31,183.3.202.172] login attempt [root/123456] succeeded

因此前两行的输出应该如下

Port,Status,AttemptsOnPort,ConnectionsOnIP,Malicious
    15853,failed,4,(total no of times the IP using this port is seen in log, even if it used other ports)
    15853,succeeded,4,18

Answer 1

此代码以下列格式打印报告。如果不需要，请删除 (IP) 字段。

Port,Status,AttemptOnPort,(IP),ConnectionsOnIP

为每个用户打印这样一行。但是，ConnectionsOnIP 是所有用户和端口看到的此 IP 的总数。该代码还单独打印一份关于 IP 的报告。查看相关问题的评论。

use strict;
use warnings;

my $file = 'logfile.txt';
open my $fh_in, '<', $file;

# Assemble results for required output in data structure:
# %rept = {
#    $port => {
#       $ip => {
#           $usr => { 
#               $status => $freq 
#          },
#       },
#   },
# };
# Auxiliary: %ip_tot = { $ip => { $status => $freq } } 

my (%rept, %ip_tot);
my ($ip, $port);

while (my $line = <$fh_in>) 
{
    if ($line =~ /New connection/) {
        ($ip, $port) = $line =~ /New connection:\s+([^:]+):(\d+)/;
        next;
    }   
    elsif (!$ip or !$port) { next }  # First lines come before New connection

    my ($usr, $status) = $line =~ m/login attempt\s+\[([^\]]+)\]\s+(\w+)/;
    if ($usr and $status) {
        $rept{$port}{$ip}{$usr}{$status}++;
        $ip_tot{$ip}{$status}++;
    }   
    else { warn "Line with an unexpected format:\n$line" }
}

print "Port,Status,AttemptOnPort,(IP),ConnectionsOnIP\n";
foreach my $port (sort keys %rept) {
    foreach my $ip (sort keys %{$rept{$port}}) {
        foreach my $usr (sort keys %{$rept{$port}{$ip}}) {
            foreach my $stat ( sort keys %{$rept{$port}{$ip}{$usr}} ) { 
                print "$port,$stat,$rept{$port}{$ip}{$usr}{$stat}";
                print "$,(ip),$ip_tot{$ip}{$stat}\n"; 
            }   
        }   
    }   
}

print "\n";
print "IP,Status,Occurences\n";
foreach my $ip (sort keys %ip_tot) {
    foreach my $stat ( sort keys %{$ip_tot{$ip}} ) {
        print "$ip,$stat,$ip_tot{$ip}{$stat}\n"; 
    }
}

将提供的输入 logfile.txt 输出为

Port,Status,AttemptOnPort,(IP),ConnectionsOnIP
15853,failed,4,(183.3.202.172),12
15853,succeeded,3,(183.3.202.172),11
18693,failed,1,(183.3.202.172),12
18942,failed,1,(183.3.202.172),12
18942,succeeded,1,(183.3.202.172),11
31130,succeeded,1,(183.3.202.172),11
46321,failed,1,(183.3.202.172),12
46321,succeeded,1,(183.3.202.172),11
47417,failed,3,(183.3.202.172),12
47417,succeeded,3,(183.3.202.172),11
53653,failed,1,(183.3.202.172),12
53653,succeeded,1,(183.3.202.172),11
60563,failed,1,(183.3.202.172),12
60563,succeeded,1,(183.3.202.172),11

IP,Status,Occurences
183.3.202.172,failed,12
183.3.202.172,succeeded,11

正则表达式解释。这是有效代码，由 /x 提供。 \s+ 在评论中被忽略。

my ($usr, $status) =  $line =~ m/ 
    login\ attempt \s+         # literal, serves as a 'post' to help matching 
    \[                         # literal [ within which our pattern is 
        (                      # start capture 
            [^\]]+             # any char which is not ], 1 or more times  
        )                      # end of capture 
    \] \s+                     # closing literal ] 
    (\w+)                      # next capture: any 'word' char, 1 or more times 
/x;

它的核心是否定字符class、[ ^\] ]。它说：匹配任何一个字符（[...]）是不是（^）一个括号（\]），需要转义（ \) 表示文字字符。后面的+表示一次或多次次。例如

my $str = 'a5_".-]B1'; 
if ($str =~ m/([^\]]+)/) { say "Got: " }

这会打印 Got: a5_".-。匹配（并捕获）第一个 ] 之前的所有内容。这是指定 非贪婪匹配 的一种方法，一直到 给定字符的第一个 出现。请注意，.+] 之类的东西会匹配 last ] 之前的所有内容，它是 greedy.

参见 Regular Expressions Tutorial。搜索 SO，以获取具体问题和迷你教程。

使用 perl 和哈希数据结构从日志文件中打印记录

Print records from log file using perl and hash data structure

perl

hash

logging

output