将来自多行日志的信息结合到单行

我有一个java应用程序日志文件，其中包含一些参数，我想在不同的行上选择，我希望将它全部放在一行上。将来自多行日志的信息结合到单行

日志示例：

2017-05-31 16:30:38.889 [http-listener-1(2)] INFO com.domain.api.provider.ClientRequestDataFilter - ContainerRequestContextFilter start 
2017-05-31 16:30:38.889 [http-listener-1(2)] INFO com.domain.api.provider.ClientRequestDataFilter - Client IP Address : 172.25.4.32 
2017-05-31 16:30:38.889 [http-listener-1(2)] INFO com.domain.api.provider.ClientRequestDataFilter - Client User-Agent : Dalvik/2.1.0 (Linux; U; Android 7.1.2; Nexus 5X Build/N2G47O) 
2017-05-31 16:30:38.889 [http-listener-1(2)] INFO com.domain.api.provider.ClientRequestDataFilter - ContainerRequestContextFilter end 
2017-05-31 16:30:38.891 [http-listener-1(2)] INFO com.domain.api.controller.MyDataController - Start getMyPageResponse method, input = {"token":"AppleBallCatDogElephantFoxGoatHorse"} 
2017-05-31 16:30:38.891 [http-listener-1(2)] INFO com.domain.biz.service.impl.MyTestService - Start getMyPageInfo method, requestDto = TokenRequestDto [token=AppleBallCatDogElephantFoxGoatHorse] 
2017-05-31 16:30:38.891 [http-listener-1(2)] INFO com.domain.biz.logic.impl.UserAuthBizLogic - Start authentication token=AppleBallCatDogElephantFoxGoatHorse 
2017-05-31 16:30:38.895 [http-listener-1(2)] INFO com.domain.biz.logic.impl.UserAuthBizLogic - End authentication, UserInfo=UserInfoDto[userID=123456789] 
2017-05-31 16:30:38.895 [http-listener-1(2)] DEBUG com.domain.biz.service.impl.MyTestService - userID = 123456789, token = AppleBallCatDogElephantFoxGoatHorse 
2017-05-31 16:30:39.422 [http-listener-1(2)] INFO com.domain.biz.service.impl.MyTestService - End getMyPageInfo method 
2017-05-31 16:30:39.423 [http-listener-1(3)] INFO com.domain.biz.service.impl.MyTestService - Start getMyPageResponse method 
2017-05-31 16:30:39.423 [http-listener-1(2)] INFO com.domain.biz.service.impl.MyTestService - End getMyPageResponse method 
2017-05-31 16:30:39.423 [http-listener-1(2)] INFO com.domain.api.controller.MyDataController - End getMyPageResponse method, Total Computation/Execution Time : 0.532s 

2017-05-31 16:42:36.500 [http-listener-1(3)] INFO com.domain.api.provider.ClientRequestDataFilter - ContainerRequestContextFilter start 
2017-05-31 16:42:36.501 [http-listener-1(3)] INFO com.domain.api.provider.ClientRequestDataFilter - Client IP Address : 172.25.4.32 
2017-05-31 16:42:36.501 [http-listener-1(3)] INFO com.domain.api.provider.ClientRequestDataFilter - Client User-Agent : MyiOSApp/v2.2.2 (build:888; iOS 10.3.2) Alamofire/4.4.0 
2017-05-31 16:42:36.501 [http-listener-1(3)] INFO com.domain.api.provider.ClientRequestDataFilter - ContainerRequestContextFilter end 
2017-05-31 16:42:36.506 [http-listener-1(3)] INFO com.domain.api.controller.MyDataController - Start getMyPageResponse method, input = {"token":"KingLionMonkeyNestOwlParrotQueenRabbitSnakeTiger"} 
2017-05-31 16:42:36.507 [http-listener-1(3)] INFO com.domain.biz.logic.impl.UserAuthBizLogic - Start authentication token=KingLionMonkeyNestOwlParrotQueenRabbitSnakeTiger 
2017-05-31 16:42:36.507 [http-listener-1(3)] INFO com.domain.biz.service.impl.MyTestService - Start getMyPageInfo method 
2017-05-31 16:42:36.515 [http-listener-1(3)] INFO com.domain.biz.logic.impl.UserAuthBizLogic - End authentication, UserInfo=UserInfoDto[userID=987654321] 
2017-05-31 16:42:36.515 [http-listener-1(3)] DEBUG com.domain.biz.service.impl.MyTestService - userID = 987654321, token = KingLionMonkeyNestOwlParrotQueenRabbitSnakeTiger 
2017-05-31 16:42:36.743 [http-listener-1(3)] INFO com.domain.biz.service.impl.MyTestService - End getMyPageInfo method 
2017-05-31 16:42:36.743 [http-listener-1(3)] INFO com.domain.biz.service.impl.MyTestService - Start getMyPageResponse method 
2017-05-31 16:42:36.743 [http-listener-1(3)] INFO com.domain.biz.service.impl.MyTestService - End getMyPageResponse method 
2017-05-31 16:42:36.743 [http-listener-1(3)] INFO com.domain.api.controller.MyDataController - End getMyPageResponse method, Total Computation/Execution Time : 0.237s

过滤器实施例1：

cat /usr/local/sun/glassfish/domains/universal-api/logs/my-new-api/system.log | grep -e "Filter" -e "userID" 

2017-05-31 16:30:38.889 [http-listener-1(2)] INFO com.domain.api.provider.ClientRequestDataFilter - ContainerRequestContextFilter start                          
2017-05-31 16:30:38.889 [http-listener-1(2)] INFO com.domain.api.provider.ClientRequestDataFilter - Client IP Address : 172.25.4.32                           
2017-05-31 16:30:38.889 [http-listener-1(2)] INFO com.domain.api.provider.ClientRequestDataFilter - Client User-Agent : MyiOSApp/v2.2.2 (build:888; iOS 10.3.2) Alamofire/4.4.0              
2017-05-31 16:30:38.889 [http-listener-1(2)] INFO com.domain.api.provider.ClientRequestDataFilter - ContainerRequestContextFilter end                          
2017-05-31 16:30:38.895 [http-listener-1(2)] INFO com.domain.biz.logic.impl.UserAuthBizLogic - End authentication, UserInfo=UserInfoDto[userID=123456789]                     
2017-05-31 16:30:38.895 [http-listener-1(2)] DEBUG com.domain.biz.service.impl.MyTestService - userID = 123456789, token = AppleBallCatDogElephantFoxGoatHorse                    
2017-05-31 16:42:36.500 [http-listener-1(3)] INFO com.domain.api.provider.ClientRequestDataFilter - ContainerRequestContextFilter start                          
2017-05-31 16:42:36.501 [http-listener-1(3)] INFO com.domain.api.provider.ClientRequestDataFilter - Client IP Address : 172.25.4.32                           
2017-05-31 16:42:36.501 [http-listener-1(3)] INFO com.domain.api.provider.ClientRequestDataFilter - Client User-Agent : Dalvik/2.1.0 (Linux; U; Android 7.1.2; Nexus 5X Build/N2G47O)              
2017-05-31 16:42:36.501 [http-listener-1(3)] INFO com.domain.api.provider.ClientRequestDataFilter - ContainerRequestContextFilter end                          
2017-05-31 16:42:36.515 [http-listener-1(3)] INFO com.domain.biz.logic.impl.UserAuthBizLogic - End authentication, UserInfo=UserInfoDto[userID=987654321]                     
2017-05-31 16:42:36.515 [http-listener-1(3)] DEBUG com.domain.biz.service.impl.MyTestService - userID = 987654321, token = KingLionMonkeyNestOwlParrotQueenRabbitSnakeTiger

过滤器实施例2：

cat /usr/local/sun/glassfish/domains/universal-api/logs/my-new-api/system.log | grep -e "IP Address" -e "User-Agent" -e "userID" 

2017-05-31 16:30:38.889 [http-listener-1(2)] INFO com.domain.api.provider.ClientRequestDataFilter - Client IP Address : 172.25.4.32                           
2017-05-31 16:30:38.889 [http-listener-1(2)] INFO com.domain.api.provider.ClientRequestDataFilter - Client User-Agent : MyiOSApp/v2.2.2 (build:888; iOS 10.3.2) Alamofire/4.4.0              
2017-05-31 16:30:38.895 [http-listener-1(2)] INFO com.domain.biz.logic.impl.UserAuthBizLogic - End authentication, UserInfo=UserInfoDto[userID=123456789]                     
2017-05-31 16:30:38.895 [http-listener-1(2)] DEBUG com.domain.biz.service.impl.MyTestService - userID = 123456789, token = AppleBallCatDogElephantFoxGoatHorse                    
2017-05-31 16:42:36.501 [http-listener-1(3)] INFO com.domain.api.provider.ClientRequestDataFilter - Client IP Address : 172.25.4.32                           
2017-05-31 16:42:36.501 [http-listener-1(3)] INFO com.domain.api.provider.ClientRequestDataFilter - Client User-Agent : Dalvik/2.1.0 (Linux; U; Android 7.1.2; Nexus 5X Build/N2G47O)              
2017-05-31 16:42:36.515 [http-listener-1(3)] INFO com.domain.biz.logic.impl.UserAuthBizLogic - End authentication, UserInfo=UserInfoDto[userID=987654321]                     
2017-05-31 16:42:36.515 [http-listener-1(3)] DEBUG com.domain.biz.service.impl.MyTestService - userID = 987654321, token = KingLionMonkeyNestOwlParrotQueenRabbitSnakeTiger

我想从这些过滤掉4两件事：

时间，IP地址，我们ER-代理，用户ID，令牌

样品期望输出：

2017-05-31 16:30:38.889, 172.25.4.32, MyiOSApp/v2.2.2 (build:888; iOS 10.3.2) Alamofire/4.4.0, 123456789 
2017-05-31 16:30:38.889, 172.25.4.32, Dalvik/2.1.0 (Linux; U; Android 7.1.2; Nexus 5X Build/N2G47O), 987654321

和打印他们都在同一行。

它们将始终以相同的顺序出现。

，以帮助文本选择我加入ContainerRequestContextFilter start和ContainerRequestContextFilter end

我相信grep组合，awk，sed & tr可以帮我做的工作，有什么其他有效的方法？

更新：问题是有很多这样的项目陆续为多个用户。我想知道编写一个shell脚本，python代码或类似的技术会更有效率。

我只能在多行日志上使用awk获得1行，我需要每行1行。

来源

2017-05-31 computingfreak

你做一个尝试自己来解决这个？ – Inian

这是你想要的吗？ 'awk -v OFS ='/ ContainerRequestContextFilter start/{time = $ 1“”$ 2; next} /客户端IP地址/ {ipAddress = $ NF; next} /客户端用户代理/ {n = split（$ 0，a， “User User Agent”）; userAgent = a [n]; next}/userID =/{m = split（$ 0，b，“userID =”）; userIDAndToken = b [m]; next} END {print time ，ipAddress，userAgent，userIDAndToken}''？ –

为什么它应该从'12：05：11.813'开始？ – RomanPerekhrest

呆子溶液：

awk '!f && /ContainerRequestContextFilter/{ f=1; t=$1 FS $2; next } 
    /Client IP Address/{ ip=$NF; next } 
    /Client User-Agent/{ uagent=$10; for(i=11;i<=NF;i++) uagent=uagent FS $i; next } 
    /userID =.* token =/{ match($0,/userID = ([0-9]+), token = (.+)/,a); f=0; 
     printf("%s, %s, %s, %s, %s\n",t,ip,uagent,a[1],a[2]); next} 
' system.log

输出：

2017-05-31 16:30:38.889, 172.25.4.32, Dalvik/2.1.0 (Linux; U; Android 7.1.2; Nexus 5X Build/N2G47O), 123456789, AppleBallCatDogElephantFoxGoatHorse 
2017-05-31 16:42:36.500, 172.25.4.32, MyiOSApp/v2.2.2 (build:888; iOS 10.3.2) Alamofire/4.4.0, 987654321, KingLionMonkeyNestOwlParrotQueenRabbitSnakeTiger

!f && /ContainerRequestContextFilter/ - 处理所述起始ContainerRequestContextFilter线
/Client IP Address/{ ip=$NF; next } - 包含加工生产线Client IP Address值
/Client User-Agent/ - 加工生产线配套Client User-Agent
/userID =.* token =/ - 加工生产线配套userID =和token =

来源

2017-05-31 10:17:45 RomanPerekhrest

我删除了“！f && “它的功能就像一个魅力，非常感谢！ – computingfreak

我想你已经将cat /usr/local/sun/glassfish/domains/universal-api/logs/my-new-api/system.log | grep -e "Filter" -e "userID"的输出指向log这个文件。

然后你可以尝试以下awk命令来实现自己的目标，

awk ' 
    /ContainerRequestContextFilter start/ {printf "%s %s,",$1,$2} 
    /Client User-Agent/ {for(i=10;i<=NF;i++) printf "%s ",$i} 
    /userID/ {printf ", %s",$9} 
    /token/ {printf " %s\n",$12} 
' log

输出形式应为您的要求。

来源

2017-05-31 06:41:05 CWLiu

将来自多行日志的信息结合到单行

回答

相关问题