使用R和正则表达式将分隔文件转换为csv

我想将原始气象站数据从数据记录器转换为易于使用的csv文件。数据如下所示。数据以这种空格分隔的格式出现，其中第一行数据有47列，第一列值等于111，第47列等于329.1。第二行的第一个值也是111，第47行是354.2。并非所有行都具有相同数量的列，并且任何数字前面的“ - ”符号表示负数。使用R和正则表达式将分隔文件转换为csv

01+0111. 02+0262. 03+1900. 04-15.68 05+64.50 06+08.82 07+1.013 08+0.943 
09+342.1 10+21.26 11+0.000 12+31.76 13+18.46 14+16.50 15+1800. 16+5.250 
17+69.46 18+1859. 19+55.25 20+27.61 21+1808. 22+50.25 23+2.367 24+1806. 
25+15.25 26+14.78 27+1859. 28+55.25 29+60.11 30+1800. 31-5.250 32+0.000 
33+1854. 34+5.250 35+0.447 36+1819. 37+50.25 38+14.80 39+69.40 40+0.073 
41+275.3 42+0.447 43+18.29 44+22.30 45+22.43 46+2.367 47+329.1 
01+0111. 02+0262. 03+2000. 04-14.28 05+070.7 06+0.012 07+0.755 08+0.694 
09+337.5 10+22.90 11+0.000 12+0.044 13+18.13 14+14.78 15+1900. 16+15.25 
17+072.6 18+1908. 19+15.25 20+0.146 21+1946. 22+10.25 23+1.567 24+1948. 
25+25.25 26+14.02 27+1959. 28+20.25 29+69.21 30+1936. 31-25.25 32+0.000 
33+1900. 34+20.25 35+0.447 36+1900. 37+5.250 38+14.02 39+69.95 40+0.000 
41+343.6 42+0.607 43+17.97 44+21.97 45+22.13 46+1.567 47+354.2 
01+0111. 02+0262. 03+2100. 04-13.01 05+075.7 06+0.007 07+0.617 08+0.528 
09+20.10 10+30.68 11+0.000 12+0.026 13+17.79 14+14.02 15+2000. 16+5.250 
17+082.7 18+2050. 19+55.25 20+0.146 21+2028. 22+30.25 23+1.407 24+2001. 
25+25.25 26+11.78 27+2051. 28+40.25 29+69.68 30+2001. 31-25.25 32+0.000 
33+2000. 34+5.250 35+0.447 36+2002. 37+25.25 38+12.00 39+081.0 40+0.000 
41+39.42 42+0.447 43+17.61 44+21.68 45+21.82 46+1.407 47+349.4 
01+0111. 02+0262. 03+1900. 04-15.68 05+64.50 06+08.82 
01+0111. 02+0262. 03+2100. 04-13.01 05+075.7 06+0.007 07+0.617 08+0.528 
09+20.10 10+30.68 11+0.000

我读在像这样的数据：

test <- readLines(data)

这导致与每个观测由所述数据的行的，以字符格式，这是没有用的数据帧。也许有办法解决这个问题，但我尝试了很多方法，没有运气。我确信有一种方法可以将上面的数据读入5行，每行包含适当数量的列和逗号分隔，并且每个值都没有前面的列号（如下所示）。但是，我不知道如何做到这一点，尤其是使用正则表达式。如果有人能够提供帮助，我会非常感激。谢谢。

111,262,1900,-15.68,64.50,8.82,1.013,0.943,342.1,21.26,0,31.76,18.46,16.50,1800,5.250,69.46,1859,55.25,27.61,1808,50.25,2.367,1806,15.25,14.78,1859,55.25,60.11,1800,-5.250,0,1854,5.250,0.447,1819,50.25,14.80,69.40,0.073,275.3,0.447,18.29,22.30,22.43,2.367,329.1 
111,262,2000,-14.28,70.7,0.012,0.755,0.694,337.5,22.90,0,0.044,18.13,14.78,1900,15.25,072.6,1908,15.25,0.146,1946,10.25,1.567,1948,25.25,14.02,1959,20.25,69.21,1936,-25.25,0,1900,20.25,0.447,1900,5.250,14.02,69.95,0,343.6,0.607,17.97,21.97,22.13,1.567,354.2 
111,262,2100,-13.01,75.7,0.007,0.617,0.528,20.10,30.68,0,0.026,17.79,14.02,2000,5.250,082.7,2050,55.25,0.146,2028,30.25,1.407,2001,25.25,11.78,2051,40.25,69.68,2001,-25.25,0,2000,5.250,0.447,2002,25.25,12.00,081.0,0,39.42,0.447,17.61,21.68,21.82,1.407,349.4 
111,262,1900,-15.68,64.50,8.82 
111,262,2100,-13.01,75.7,0.007,0.617,0.528,20.10,30.68,0 

This is what my data looks like after I read it in: 
c("01+0111. 02+0262. 03+1900. 04-15.68 05+64.50 06+08.82 
07+1.013  08+0.943", 
"09+342.1 10+21.26 11+0.000 12+31.76 13+18.46 14+16.50 15+1800. 
16+5.250", 
"17+69.46 18+1859. 19+55.25 20+27.61 21+1808. 22+50.25 23+2.367 
24+1806.", 
"25+15.25 26+14.78 27+1859. 28+55.25 29+60.11 30+1800. 31-5.250 
32+0.000", 
"33+1854. 34+5.250 35+0.447 36+1819. 37+50.25 38+14.80 39+69.40 
40+0.073", 
"41+275.3 42+0.447 43+18.29 44+22.30 45+22.43 46+2.367 47+329.1", 
"01+0111. 02+0262. 03+2000. 04-14.28 05+070.7 06+0.012 07+0.755 
08+0.694", 
"09+337.5 10+22.90 11+0.000 12+0.044 13+18.13 14+14.78 15+1900. 
16+15.25", 
"17+072.6 18+1908. 19+15.25 20+0.146 21+1946. 22+10.25 23+1.567 
24+1948.", 
"25+25.25 26+14.02 27+1959. 28+20.25 29+69.21 30+1936. 31-25.25 
32+0.000", 
"33+1900. 34+20.25 35+0.447 36+1900. 37+5.250 38+14.02 39+69.95  
40+0.000", 
"41+343.6 42+0.607 43+17.97 44+21.97 45+22.13 46+1.567 47+354.2", 
"01+0111. 02+0262. 03+2100. 04-13.01 05+075.7 06+0.007 07+0.617 
08+0.528", 
"09+20.10 10+30.68 11+0.000 12+0.026 13+17.79 14+14.02 15+2000.  
16+5.250", 
"17+082.7 18+2050. 19+55.25 20+0.146 21+2028. 22+30.25 23+1.407 
24+2001.", 
"25+25.25 26+11.78 27+2051. 28+40.25 29+69.68 30+2001. 31-25.25 
32+0.000", 
"33+2000. 34+5.250 35+0.447 36+2002. 37+25.25 38+12.00 39+081.0 
40+0.000", 
"41+39.42 42+0.447 43+17.61 44+21.68 45+21.82 46+1.407 47+349.4", 
"01+0111. 02+0262. 03+1900. 04-15.68 05+64.50 06+08.82", 
"01+0111. 02+0262. 03+2100. 04-13.01 05+075.7 06+0.007 07+0.617 
08+0.528", 
"09+20.10 10+30.68 11+0.000") 

This is the result of dput(text) after running the code: 
structure(list(X01 = c(111, 342.1, 69.46, 15.25, 1854, 275.3, 
111, 337.5, 72.6, 25.25, 1900, 343.6, 111, 20.1, 82.7, 25.25, 
2000, 39.42, 111, 111, 20.1), X02 = c(262, 21.26, 1859, 14.78, 
5.25, 0.447, 262, 22.9, 1908, 14.02, 20.25, 0.607, 262, 30.68, 
2050, 11.78, 5.25, 0.447, 262, 262, 30.68), X03 = c(1900, 0, 
55.25, 1859, 0.447, 18.29, 2000, 0, 15.25, 1959, 0.447, 17.97, 
2100, 0, 55.25, 2051, 0.447, 17.61, 1900, 2100, 0), X04 = c(-15.68, 
31.76, 27.61, 55.25, 1819, 22.3, -14.28, 0.044, 0.146, 20.25, 
1900, 21.97, -13.01, 0.026, 0.146, 40.25, 2002, 21.68, -15.68, 
-13.01, NA), X05 = c(64.5, 18.46, 1808, 60.11, 50.25, 22.43, 
70.7, 18.13, 1946, 69.21, 5.25, 22.13, 75.7, 17.79, 2028, 69.68, 
25.25, 21.82, 64.5, 75.7, NA), X06 = c(8.82, 16.5, 50.25, 1800, 
14.8, 2.367, 0.012, 14.78, 10.25, 1936, 14.02, 1.567, 0.007, 
14.02, 30.25, 2001, 12, 1.407, 8.82, 0.007, NA), X07 = c(1.013, 
1800, 2.367, -5.25, 69.4, 329.1, 0.755, 1900, 1.567, -25.25, 
69.95, 354.2, 0.617, 2000, 1.407, -25.25, 81, 349.4, NA, 0.617, 
NA), X08 = c(0.943, 5.25, 1806, 0, 0.073, NA, 0.694, 15.25, 1948, 
0, 0, NA, 0.528, 5.25, 2001, 0, 0, NA, NA, 0.528, NA), X09 = c(NA_real_, 
NA_real_, NA_real_, NA_real_, NA_real_, NA_real_, NA_real_, NA_real_, 
NA_real_, NA_real_, NA_real_, NA_real_, NA_real_, NA_real_, NA_real_, 
NA_real_, NA_real_, NA_real_, NA_real_, NA_real_, NA_real_), 
X10 = c(NA_real_, NA_real_, NA_real_, NA_real_, NA_real_, 
NA_real_, NA_real_, NA_real_, NA_real_, NA_real_, NA_real_, 
NA_real_, NA_real_, NA_real_, NA_real_, NA_real_, NA_real_, 
NA_real_, NA_real_, NA_real_, NA_real_), X11 = c(NA_real_, 
.....)

来源

2017-09-26 user8229029

使用'dput（测试）'你的问题，以显示你的数据是什么样子，你看它在 – Tunn

什么是你在读取数据的原始文件类型后从？ –

Richard，它是一个.dat文件。 – user8229029

你不明确你的数据的格式，但它好像前缀，每个字段的nn+是列数，这应该被忽略。如果是这样，一个简单的模式可能会有所帮助。但是这对于负数不起作用。

> line <- "01+0111. 02+0262. 03+1900. 04-15.68 05+64.50 06+08.82 07+1.013 08+0.943" 
> gsub("\\d\\d\\+", "", line) 
[1] "0111. 0262. 1900. 04-15.68 64.50 08.82 1.013 0.943"

是否有任何理由让这些在5列高达47列的形式，但如果列数可以通过行有什么不同？

来源

2017-09-26 01:46:24 user101089

我无法控制逐行不同数量的列，但这是因为某些行按小时或每日汇总数据。 – user8229029

假设你可以忽略+/-号前的数量和他们的顺序为每一行，你可以转换成数字的数据帧这与以下：

library(dplyr) 
library(stringr) 
library(tidyr) 
test %>% 
    as_data_frame %>% 
    separate(value, into = sprintf("X%02d", 1:47), sep = "\\s+") %>% 
    mutate_all(str_replace, pattern = "\\d{2}((\\+)|(?=\\-))", replacement = "") %>% 
    mutate_all(as.numeric) 
# A tibble: 3 x 47 
    X01 X02 X03 X04 X05 X06 X07 X08 X09 X10 X11 X12 X13 
    <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> 
1 111 262 1900 -15.68 64.5 8.820 1.013 0.943 342.1 21.26  0 31.760 18.46 
2 111 262 2000 -14.28 70.7 0.012 0.755 0.694 337.5 22.90  0 0.044 18.13 
3 111 262 2100 -13.01 75.7 0.007 0.617 0.528 20.1 30.68  0 0.026 17.79 
# ... with 34 more variables: X14 <dbl>, X15 <dbl>, X16 <dbl>, X17 <dbl>, 
# X18 <dbl>, X19 <dbl>, X20 <dbl>, X21 <dbl>, X22 <dbl>, X23 <dbl>, X24 <dbl>, 
# X25 <dbl>, X26 <dbl>, X27 <dbl>, X28 <dbl>, X29 <dbl>, X30 <dbl>, X31 <dbl>, 
# X32 <dbl>, X33 <dbl>, X34 <dbl>, X35 <dbl>, X36 <dbl>, X37 <dbl>, X38 <dbl>, 
# X39 <dbl>, X40 <dbl>, X41 <dbl>, X42 <dbl>, X43 <dbl>, X44 <dbl>, X45 <dbl>, 
# X46 <dbl>, X47 <dbl>

首先，我对每个每个条目分开行到它自己的专栏。

下一步与正则表达式有点棘手。每个+或 - 前面都有两个数字，所以我们匹配。我们也想要匹配和更多的+，如果它在那里，但只匹配并留下一个 - 。如果字符串包含 - 超前值，但不包含 - ，则使用正匹配。我在这里玩的正则表达式：https://regex101.com/r/VvIRlu/1

之后，我可以将所有列mutatate数值，我们完成。

来源

2017-09-26 01:57:38 beigel

谢谢，但我的数据没有被转换成3行，就像你的数据一样。矿井的行数保持不变，每行都附加一堆NA。我不确定有什么不同，因为我正在使用您的确切代码。 – user8229029

你可以运行然后分享'dput（test）'的结果吗？ – beigel

我添加了读取mt数据和运行代码的dput结果。 – user8229029

我已经通过假定01前缀表示一个新的情况下（使用str_split("^01|\n01")的开始和每个所需列的前缀是前面有一个空格的双位数（即str_split("\\s[0-9]{2}），这曾在此一展身手拆分文本成单独的箱子。然后

这些情况转换为单个tibble与帮助从purrr::map2_df和tidyr::spread。一个小的修改在端作出把列在使用gtools::mixedorder正确的顺序。

要将数据转换为逗号分隔，你可以写出它o一个csv文件。例如，仅仅在溶液中的最后一行更改为df %>% select(gtools::mixedorder(names(.))) %>% write_csv("df.csv")

解决方案

library(tidyverse) 
library(stringr) 

cases <- txt %>% 
    str_split("^01|\n01") %>% unlist() %>% 
    str_split("\\s[0-9]{2}") %>% .[-1] 

df <- map2_df(cases, 1:length(cases), function(x, y) { 
    x %>% 
    as.numeric() %>% 
    as_tibble() %>% 
    rownames_to_column() %>% 
    mutate(rowname = paste0("V", rowname)) %>% 
    mutate(id = y) 
    }) %>% spread(rowname, value) 

df %>% select(gtools::mixedorder(names(.))) 

#> # A tibble: 5 x 48 
#>  id V1 V2 V3  V4 V5 V6 V7 V8 V9 V10 V11 
#> * <int> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> 
#> 1  1 111 262 1900 -15.68 64.5 8.820 1.013 0.943 342.1 21.26  0 
#> 2  2 111 262 2000 -14.28 70.7 0.012 0.755 0.694 337.5 22.90  0 
#> 3  3 111 262 2100 -13.01 75.7 0.007 0.617 0.528 20.1 30.68  0 
#> 4  4 111 262 1900 -15.68 64.5 8.820 NA NA NA NA NA 
#> 5  5 111 262 2100 -13.01 75.7 0.007 0.617 0.528 20.1 30.68  0 
#> # ... with 36 more variables: V12 <dbl>, V13 <dbl>, V14 <dbl>, V15 <dbl>, 
#> # V16 <dbl>, V17 <dbl>, V18 <dbl>, V19 <dbl>, V20 <dbl>, V21 <dbl>, 
#> # V22 <dbl>, V23 <dbl>, V24 <dbl>, V25 <dbl>, V26 <dbl>, V27 <dbl>, 
#> # V28 <dbl>, V29 <dbl>, V30 <dbl>, V31 <dbl>, V32 <dbl>, V33 <dbl>, 
#> # V34 <dbl>, V35 <dbl>, V36 <dbl>, V37 <dbl>, V38 <dbl>, V39 <dbl>, 
#> # V40 <dbl>, V41 <dbl>, V42 <dbl>, V43 <dbl>, V44 <dbl>, V45 <dbl>, 
#> # V46 <dbl>, V47 <dbl>

源数据
上述解决方案不会尝试将文本拆分成进口线使用readLines。相反，readr::read_file应该与建议的解决方案一起使用。在这个例子中使用的文字txt是：

txt <- "01+0111. 02+0262. 03+1900. 04-15.68 05+64.50 06+08.82 07+1.013 08+0.943 
09+342.1 10+21.26 11+0.000 12+31.76 13+18.46 14+16.50 15+1800. 16+5.250 
17+69.46 18+1859. 19+55.25 20+27.61 21+1808. 22+50.25 23+2.367 24+1806. 
25+15.25 26+14.78 27+1859. 28+55.25 29+60.11 30+1800. 31-5.250 32+0.000 
33+1854. 34+5.250 35+0.447 36+1819. 37+50.25 38+14.80 39+69.40 40+0.073 
41+275.3 42+0.447 43+18.29 44+22.30 45+22.43 46+2.367 47+329.1 
01+0111. 02+0262. 03+2000. 04-14.28 05+070.7 06+0.012 07+0.755 08+0.694 
09+337.5 10+22.90 11+0.000 12+0.044 13+18.13 14+14.78 15+1900. 16+15.25 
17+072.6 18+1908. 19+15.25 20+0.146 21+1946. 22+10.25 23+1.567 24+1948. 
25+25.25 26+14.02 27+1959. 28+20.25 29+69.21 30+1936. 31-25.25 32+0.000 
33+1900. 34+20.25 35+0.447 36+1900. 37+5.250 38+14.02 39+69.95 40+0.000 
41+343.6 42+0.607 43+17.97 44+21.97 45+22.13 46+1.567 47+354.2 
01+0111. 02+0262. 03+2100. 04-13.01 05+075.7 06+0.007 07+0.617 08+0.528 
09+20.10 10+30.68 11+0.000 12+0.026 13+17.79 14+14.02 15+2000. 16+5.250 
17+082.7 18+2050. 19+55.25 20+0.146 21+2028. 22+30.25 23+1.407 24+2001. 
25+25.25 26+11.78 27+2051. 28+40.25 29+69.68 30+2001. 31-25.25 32+0.000 
33+2000. 34+5.250 35+0.447 36+2002. 37+25.25 38+12.00 39+081.0 40+0.000 
41+39.42 42+0.447 43+17.61 44+21.68 45+21.82 46+1.407 47+349.4 
01+0111. 02+0262. 03+1900. 04-15.68 05+64.50 06+08.82 
01+0111. 02+0262. 03+2100. 04-13.01 05+075.7 06+0.007 07+0.617 08+0.528 
09+20.10 10+30.68 11+0.000 "

来源

2017-09-27 01:42:40 markdly

使用R和正则表达式将分隔文件转换为csv

回答

相关问题