I have a text file that has tail output from multiple application log
files. The application log files are all from the same application,
but individual nodes running on various servers. The log information
is ssh'ed (tail -200...) from the remote servers to the local text
file.
Since the retrieval process isn't very refined, I'm stuck with log
output that spans several minutes, but I'm only interested in the
output from the past minute (date +%H:%M --date "1 minute ago"). I've
tried to write a small awk process that would go through the log file
every 5 minutes and capture the output from the past minute and write
it out to individual log files locally.
Here's the format of the log file...
==> file name <==
date_stamp | time_stam (which can be now minus several past minutes) |
trigger text
date_stamp | time_stam (which can be now minus several past minutes) |
text
more text
more text
date_stamp | time_stam (which can be now minus several past minutes) |
trigger text
date_stamp | time_stam (which can be now minus several past minutes) |
text
more text
more text
date_stamp | time_stam (which can be now minus several past minutes) |
trigger text
date_stamp | time_stam (which can be now minus several past minutes) |
text
more text
more text
==> file name <==
date_stamp | time_stam (which can be now minus several past minutes) |
trigger text
date_stamp | time_stam (which can be now minus several past minutes) |
text
more text
more text
date_stamp | time_stam (which can be now minus several past minutes) |
trigger text
date_stamp | time_stam (which can be now minus several past minutes) |
text
more text
more text
date_stamp | time_stam (which can be now minus several past minutes) |
trigger text
date_stamp | time_stam (which can be now minus several past minutes) |
text
more text
more text
________________________________________
some arbitrary text that
spans multiple lines since I'm stuck with the
tail -200 output. Followed by the meat of the
content
==> /dir-path/file1.out <==
2007-03-23 hh:mm:ss [num] channels
2007-03-23 17:33:01 Some text ....
[Num] text I need
[Num] text I need
[Num] text I need
[Num] text I need
[Num] text I need
[Num] text I need
[Num] text I need
[Num] text I need
Arbitrary text spanning
several lines...
2007-03-23 17:34:01 [num] channels
2007-03-23 17:34:01 Some text ....
[Num] text I need
[Num] text I need
[Num] text I need
[Num] text I need
[Num] text I need
[Num] text I need
[Num] text I need
[Num] text I need
Arbitrary text spanning
several lines...
==> /dir-path/file2.out <==
2007-03-23 hh:mm:ss [num] channels
2007-03-23 17:33:01 Some text ....
[Num] text I need
[Num] text I need
[Num] text I need
[Num] text I need
[Num] text I need
[Num] text I need
[Num] text I need
[Num] text I need
Arbitrary text spanning
several lines...
2007-03-23 17:34:01 [num] channels
2007-03-23 17:34:01 Some text ....
[Num] text I need
[Num] text I need
[Num] text I need
[Num] text I need
[Num] text I need
[Num] text I need
[Num] text I need
[Num] text I need
Arbitrary text spanning
several lines...
________________________________________
For every file output (==> file name <==) I would like to retrieve all
the information from the last minute and dump it to a separate file.
I have a sample awk but it doesn't process correctly.
awk '{
# Here I want to capture what tail output file I'm in.
if($2~/"file_name1"/){
filelabel="file1";
}
else if($2~/"file_name2"/){
filelabel="file2";
}
etc...
# Here I'm trying to say that I set a marker that indicates that I'm
at the correct position in the file and the time stamp is correc.
if(NF==4 && $2~timeval && $4~/channels/)
start="yes";
# Here I'm at a potentially correct position, but it's not the
correct time stamp ($2!~timeval)
else if(NF==4 && $2!~timeval && $4~/channels/)
start="no";
while(start=="yes"){
if($2~/chancount/){
start="no";
next;
}
else printf("%s %s\n",filelabel, $0);
}
}' timeval=`date +%H:%M --date "1 minute ago"` ${infile}
What am I doing wrong?
Thanks.
Emma
