如何使用附加列显示文件名进行BULK INSERT？

Question

我还是习惯了SQL，所以在开始使用存储过程之前，我想先了解如何有效地使用BULK INSERT。

我需要合并50个csv文件并将它们转储到SQL表中。问题是，我希望能够区分每个记录（如每个记录属于某个csv文件，我将通过文件名识别）。

这里有我想要的一个小例子：

CREATE TABLE ResultsDump 
(
    PC FLOAT, 
    Amp VARCHAR(50), 
    RCS VARCHAR(50), 
    CW VARCHAR(50), 
    State0 VARCHAR(50), 
    State1 VARCHAR(50), 
) 
BULK INSERT ResultsDump 
    FROM 'c:\distance1000_7_13_2010_1_13PM_Avery DennisonAD_2300008_10S_Lock.csv' 
    WITH 
    ( 
     FIRSTROW = 2, 
     MAXERRORS = 0, 
     FIELDTERMINATOR = ',', 
     ROWTERMINATOR = '\n' 
    ) 
BULK INSERT ResultsDump 
    FROM 'c:\distance1000_7_13_2010_2_27PM_Avery DennisonAD_2300009_10S_Lock.csv' 
    WITH 
    ( 
     FIRSTROW = 2, 
     MAXERRORS = 0, 
     FIELDTERMINATOR = ',', 
     ROWTERMINATOR = '\n' 
    ) 
BULK INSERT ResultsDump 
    FROM 'C:\distance1000_7_13_2010_2_58PM_Avery DennisonAD_230000A_10S_Lock.csv' 
    WITH 
    ( 
     FIRSTROW = 2, 
     MAXERRORS = 0, 
     FIELDTERMINATOR = ',', 
     ROWTERMINATOR = '\n' 
    ) 
BULK INSERT ResultsDump 
    FROM 'c:\distance1000_7_13_2010_3_21PM_Avery DennisonAD_230000B_10S_Lock.csv' 
    WITH 
    ( 
     FIRSTROW = 2, 
     MAXERRORS = 0, 
     FIELDTERMINATOR = ',', 
     ROWTERMINATOR = '\n' 
    ) 
BULK INSERT ResultsDump 
    FROM 'c:\distance1000_7_13_2010_3_41PM_Avery DennisonAD_230000C_10S_Lock.csv' 
    WITH 
    ( 
     FIRSTROW = 2, 
     MAXERRORS = 0, 
     FIELDTERMINATOR = ',', 
     ROWTERMINATOR = '\n' 
    ) 

我知道这是做事情的方式效率极低，但我绝对想弄清楚如何手工转储一个文件中的SQL表中格式我想在开始创建存储过程之前。

在新表，我想是这样的：

FileName,PC,Amp,RCS,CW,State0,State1 
c:\distance1000_7_13_2010_1_13PM_Avery DennisonAD_2300008_10S_Lock.csv, ... 
... 
... 
c:\distance1000_7_13_2010_2_27PM_Avery DennisonAD_2300009_10S_Lock.csv, ... 
... 
... 
c:\distance1000_7_13_2010_2_58PM_Avery DennisonAD_230000A_10S_Lock.csv, ... 
... 
... 

任何简单的建议或推荐特定的功能将是巨大的！请记住，我已经习惯了SQL，如果我可以一次完成这一步，那将是非常棒的，这就是为什么我要从这样一个简单的问题开始。

提前致谢！

Answer 1

您可以将列FileName varchar(max)添加到ResultsDump表中，使用新列创建表的视图，批量插入视图中，并且在每次插入之后，为仍具有其默认值的列设置文件名称null ：

CREATE TABLE dbo.ResultsDump 
(
    PC FLOAT, 
    Amp VARCHAR(50), 
    RCS VARCHAR(50), 
    CW VARCHAR(50), 
    State0 VARCHAR(50), 
    State1 VARCHAR(50), 
) 
GO 

ALTER TABLE dbo.ResultsDump ADD [FileName] VARCHAR(300) NULL 
GO 

CREATE VIEW dbo.vw_ResultsDump AS 
SELECT 
    PC, 
    Amp, 
    RCS, 
    CW, 
    State0, 
    State1 
FROM 
    ResultsDump 
GO 

BULK INSERT vw_ResultsDump 
    FROM 'c:\distance1000_7_13_2010_1_13PM_Avery DennisonAD_2300008_10S_Lock.csv' 
    WITH 
    ( 
     FIRSTROW = 2, 
     MAXERRORS = 0, 
     FIELDTERMINATOR = ',', 
     ROWTERMINATOR = '\n' 
    ) 

UPDATE dbo.ResultsDump 
SET [FileName] = 'c:\distance1000_7_13_2010_1_13PM_Avery DennisonAD_2300008_10S_Lock.csv' 
WHERE [FileName] IS NULL 

BULK INSERT vw_ResultsDump 
    FROM 'c:\distance1000_7_13_2010_2_27PM_Avery DennisonAD_2300009_10S_Lock.csv' 
    WITH 
    ( 
     FIRSTROW = 2, 
     MAXERRORS = 0, 
     FIELDTERMINATOR = ',', 
     ROWTERMINATOR = '\n' 
    ) 

UPDATE dbo.ResultsDump 
SET [FileName] = 'distance1000_7_13_2010_2_27PM_Avery DennisonAD_2300009_10S_Lock.csv' 
WHERE [FileName] IS NULL 

Answer 2

试试这个，

    ALTER PROCEDURE [dbo].[ReadandUpdateFileNames_SP] 
      (
      @spRequestId NVARCHAR(50) 
      ,@LoopCounter INT =0 
      ,@MaxFIVId INT=0 
      ,@spFileName NVARCHAR(100)=NULL) 
      AS 
      BEGIN 
      SET NOCOUNT ON 
       BEGIN TRY 
        BEGIN TRAN 
        -- To read filename's from the Request Id and store it in a temp table 
         DECLARE @cmd nvarchar(500) 
         SET @cmd = 'dir D:\Input\REQUEST-'+@spRequestId+' /b ' 
         --PRINT @cmd 
         DECLARE @DirOutput TABLE(
             ID INT IDENTITY 
            , files varchar(500)) 

         INSERT INTO @DirOutput 
         EXEC master.dbo.xp_cmdshell @cmd 
         SELECT * FROM @DirOutput WHERE files IS NOT NULL ORDER BY ID 

         ----Read files by RequestId BEGIN 

         SELECT @LoopCounter = min(ID) , @MaxFIVId = max(ID) 
          FROM @DirOutput 

         WHILE(@LoopCounter IS NOT NULL AND @LoopCounter<@MaxFIVId) 
         BEGIN 
         -- Create temp table to store FIVItems 
          CREATE TABLE Items_TEMP 
          (
            ControlID NVARCHAR(50) 
           , UNRS_Code NUMERIC(18,0) 
           , UNRS_Code_S NUMERIC(18,0) 
           , Ordered_Quantity NUMERIC(18,3) 
           , Sent_Quantity NUMERIC(18,3) 
           , Accepted_Quantity NUMERIC(18,3) 
           , Unit_Food_Price NUMERIC(18,2) 
           , Total_Price NUMERIC(18,2) 
          ) 
         SELECT @spFileName=files FROM @DirOutput WHERE ID=@LoopCounter 
          PRINT @LoopCounter 
          DECLARE @spControlId NVARCHAR(50) 
          SET @spControlId='FFO'+ Substring(@spFileName, 4, (len(@spFileName)-7)) 
          --PRINT @spControlId 
          DECLARE @sqlCmd NVARCHAR(MAX) 
          SET @sqlCmd='BULK INSERT 
          Items_TEMP 
          FROM ''D:\Input\REQUEST-'+@spRequestId+'\'+@spFileName+''' 
          WITH(
            FIELDTERMINATOR='','' 
           , ROWTERMINATOR=''\n'' 
          )' 
          PRINT @sqlCmd 
          EXECUTE sp_executesql @sqlCmd 
          ---Add a new column to the table which is not present in the CSV 
          ALTER TABLE Items_TEMP ADD OrderId NUMERIC(18,0)NULL 
          UPDATE Items_TEMP SET ControlID=@spControlId,OrderId=(SELECT OrderId FROM dbo.Orders WHERE ControlID=@spControlId) 
          SELECT * FROM Items_TEMP 

          --DROP FIVItems_TEMP table once CSV output generated 
          DROP TABLE Items_TEMP 


          SET @LoopCounter=@LoopCounter+1 
         END 
        ----****END*** 
        COMMIT TRAN 
       END TRY 
       BEGIN CATCH 
        ROLLBACK TRAN 
        PRINT 'ROLLBACK' 
        PRINT Error_Message() 
        SELECT ERROR_LINE() AS ErrorLine; 
       END CATCH 
      SET NOCOUNT OFF 
      END