日志文件的两行之间的时间差

如果这太基本，我很抱歉。我真的不想找人做这项工作，而是指向正确的方向。我有一个日志文件可以追溯到几年，我希望通过这些日志文件来确定在性能变慢时需要花费多长时间来查找模式。我能够阅读每一行，但无法读取上一行以获得时间。日志文件的两行之间的时间差

日志文件如下：

~ 
Other Stuff 
~ 
12/21/11 18:58:15 Inserting data into ST_ITEMS 
ST_ITEMS Row: 10000 inserted at 12/21/11 19:40:06 
ST_ITEMS Row: 20000 inserted at 12/21/11 20:05:58 
ST_ITEMS Row: 30000 inserted at 12/21/11 20:37:03 
ST_ITEMS Row: 40000 inserted at 12/21/11 20:59:25 
ST_ITEMS Row: 50000 inserted at 12/21/11 21:17:43 
ST_ITEMS Row: 60000 inserted at 12/21/11 21:54:47 
12/21/11 21:59:24 Finished inserting data into Staging Tables 

~ 
Other Stuff 
~ 

12/21/11 22:04:43 Inserting data into ST_ITEMS 
ST_ITEMS Row: 10000 inserted at 12/21/11 22:38:53 
ST_ITEMS Row: 20000 inserted at 12/21/11 23:06:33 
ST_ITEMS Row: 30000 inserted at 12/21/11 23:33:03 
ST_ITEMS Row: 40000 inserted at 12/22/11 00:05:38 
ST_ITEMS Row: 50000 inserted at 12/22/11 00:45:59 
ST_ITEMS Row: 60000 inserted at 12/22/11 01:12:42 
ST_ITEMS Row: 70000 inserted at 12/22/11 01:40:02 
ST_ITEMS Row: 80000 inserted at 12/22/11 02:14:23 
ST_ITEMS Row: 90000 inserted at 12/22/11 03:04:15 
ST_ITEMS Row: 100000 inserted at 12/22/11 03:47:13 
ST_ITEMS Row: 110000 inserted at 12/22/11 04:36:21 
ST_ITEMS Row: 120000 inserted at 12/22/11 05:44:47 
ST_ITEMS Row: 130000 inserted at 12/22/11 06:28:24 
ST_ITEMS Row: 140000 inserted at 12/22/11 07:10:55 
ST_ITEMS Row: 150000 inserted at 12/22/11 07:35:16 
12/22/11 07:40:28 Finished inserting data into Staging Tables 

~ 
Other Stuff 
~

从本质上讲，我想计算减去它上面的一行的日期/时间每10000行多久。我将Perl和Bash看作是选项，但Perl似乎提供了更多的可能性。

PERL ＃！的/ usr/bin中/ perl的

use strict; 
use warnings; 

use Date::Parse; 
use Date::Format; 

my $start = "2007-11-17 12:50:22"; 
my $stop = "2007-11-17 12:53:22"; 
my $diff = str2time($stop) - str2time($start); 

#printf "diff between %s and %s is %d seconds\n", $start, $stop, $diff; 

open(LOG,"info_refresh_tvl.log.122111_185800") or die "Unable to open logfile:$!\n"; 
while(my $line = <LOG>){ 


     if ($line=~/inserted\b/) 

     { 
     #Pseudocode 
      #Parse time from Pervious Line 
      #Parse time from Current Line 
      #Calculate Difference of Time 
        #my $diff = str2time($stop) - str2time($start); 
      #printf "diff between %s and %s is %d seconds\n", $start, $stop,  $diff; ') 

      printf $line ; 


     } 

} 
close(LOG);

BASH

grep 'ST_ITEMS Row:' logfile122111.log | while read line 
    do 
     event=$(echo "$line" | awk '{print $6 " " $7}') 

     case $event in 
      "10000") 
       ;; 
      *) 
       past=$(echo "$line" | awk '{print $6 " " $7}') 
       current=$(echo "$line" | awk '{print $6 " " $7}' 
       echo $past 
       echo $current) 
       ;; 
     esac 



echo $event 


    done

来源

2011-12-23 user739866

我正在考虑把信息转换成的次阵列，然后从该阵列解析时间和减去。而（我的$线= ）{ 如果（$线=〜/插入\ B /） { 推（@times，$线）; } print“@times”; } close（LOG）; – user739866 2011-12-23 18:13:41

只需保存每个行，当你在比较之后继续。完成后用当前行覆盖它。

伪代码：

$CurrentLine = $line; 
#Parse time from $CurrentLine 
#Parse time from $LastLine 
#Calculate difference of time 
$LastLine = $line;

来源

2011-12-23 18:13:46

grep -B1 gets the previous line before the line that is currently matched

来源

2011-12-23 18:18:20

你可以定义两个变量来保存时间。在伪代码，这将使：

my $old = undef; 
my $current; 

while (my $line = <LOG>) { 
    $line =~ /inserted at (.*)/ or next; 
    $current = parse_time($1); 
    if (defined $old) { 
     printf("Time to insert 10k rows: %d\n", datediff($current, $old)); 
    } 
    $old = $current; 
}

（填补国内空白的parse_time()和datediff()，你应该设置）

来源

2011-12-23 18:18:23 fge

如前所述别人，只保留以前的时间以供参考。下面是简单的例子使用Time::Piece，这是因为Perl 5.10核心模块：

use Time::Piece; 

my $lasttime; 
while(<DATA>) { 
    chomp; 

    my $diff; 
    if(m{(\d+/\d+/\d+ \d+:\d+:\d+)}) { 
     my $t = Time::Piece->strptime($1, "%D %H:%M:%S"); 
     if(defined $lasttime) { 
      $diff = $t - $lasttime; 
     } 
     $lasttime = $t; 
    } 
    undef $lasttime if m{Finished inserting data}; 

    print "$_\t", ($diff && $diff->pretty) , "\n"; 
} 

__DATA__ 
~ 
Other Stuff 
~ 
12/21/11 18:58:15 Inserting data into ST_ITEMS 
ST_ITEMS Row: 10000 inserted at 12/21/11 19:40:06 
ST_ITEMS Row: 20000 inserted at 12/21/11 20:05:58 
ST_ITEMS Row: 30000 inserted at 12/21/11 20:37:03 
ST_ITEMS Row: 40000 inserted at 12/21/11 20:59:25 
...

打印

~ 
Other Stuff 
~ 
12/21/11 18:58:15 Inserting data into ST_ITEMS 
ST_ITEMS Row: 10000 inserted at 12/21/11 19:40:06 41 minutes, 51 seconds 
ST_ITEMS Row: 20000 inserted at 12/21/11 20:05:58 25 minutes, 52 seconds 
ST_ITEMS Row: 30000 inserted at 12/21/11 20:37:03 31 minutes, 5 seconds 
ST_ITEMS Row: 40000 inserted at 12/21/11 20:59:25 22 minutes, 22 seconds

来源

2011-12-23 18:31:08 bvr

日志文件的两行之间的时间差

回答

相关问题