如何在python中读取多个csv并获得一个csv作为输出

Question

我已经问过如何在熊猫中解决这个问题。但现在我需要一个非熊猫版本。

我的代码

import glob 
import os 

## path 
path = r'C:/x/x/Desktop/xxx/' 
all_files = glob.glob(os.path.join(path, '*.csv')) 

## column 
column_headers = ['Date', 'Time', 'Duration', 'IP', 'Request'] 

## open only one csv. -- I want to read here not only 1 file -- 
## my approach: 
## with open(all_files) as log, .... 
with open('log.csv') as log, open('out355.csv', 'w') as out: 
    out.write(';'.join(column_headers)+'\n') 
    while True: 
     try: 
      lines = [next(log).strip('\n').split(' ',4) for i in range(6)][3:] 
      out.write(';'.join(lines[1][:2]+[l[4] for l in lines])+'\n') 
     except StopIteration: 
      break 

由于我是新来的蟒蛇，我不能随便修改我运行的代码只是这么好。所以我会很高兴，如果我能得到完整的代码。

谢谢！

Answer 1

你很近，你需要阅读每个*.csv文件并将它们连接起来。所以你必须打开一个新文件并使用glob读取每个csv文件。确保当你这样做有每个CSV文件，在其末端有一个尾随新行，或者你想最终的file_x最后一行和file_x+1第一数据线在同一直线上

from glob import glob 

with open('combined.csv', 'a') as combinedFile: 
    combinedFile.write('a,b,c,d,e\n') # Headers 
    for eachFile in glob('*.csv'): 
     if eachFile == 'combined.csv': 
      pass 
     else: 
      count = 0 
      for line in open(eachFile, 'r'): 
       if count != 0: # So that you don't read 1st line of every file if it contains the headers. 
        combinedFile.write(line) 
       count = 1 

运行：

a.csv

a,b,c,d,e 
1,2,3,4,5 
6,7,8,9,10 

b.csv

a,b,c,d,e 
11,12,13,14,15 
16,17,18,19,20 

combined.csv

a,b,c,d,e  
1,2,3,4,5 
6,7,8,9,10 
11,12,13,14,15 
16,17,18,19,20 

Answer 2

沿着这些线路的东西应该工作：

with open('out355.csv', 'w') as out: 
    for csvfile in all_files: 
    with open(csvfile) as log: 
     out.write(...) 
     .. the rest of your script .. 

Answer 3

这应该工作

import glob 
import os 

## path 
path = r'C:/x/x/Desktop/xxx/' 
all_files = glob.glob(os.path.join(path, '*.csv')) 

## column 
column_headers = ['Date', 'Time', 'Duration', 'IP', 'Request'] 

out = open('out355.csv', 'w') 
out.write(';'.join(column_headers)+'\n') 
for file_ in all_files: 
    log = open(file_) 
    while True: 
     try: 
      lines = [next(log).strip('\n').split(' ',4) for i in range(6)][3:] 
      out.write(';'.join(lines[1][:2]+[l[4] for l in lines])+'\n') 
     except StopIteration: 
      break