2012-07-26 54 views
2

我每隔几个月左右使用F#一次,似乎我忘记了所有事情,所以我希望你能原谅我的无知。我的下面的代码是从雅虎拉数据。这是一个很好的例子,代表我需要做的事情。返回的第一行具有列标题。我需要获取数据(列表的尾部)并将其插入到数据库中。根据返回的列标题生成插入语句的最佳方式是什么(列标题与数据库列名匹配)?使用F创建插入语句#

在下面的例子dataWithHeaders。[0]将包含“日期,开放,高,低,关闭,音量,调整关闭”。我应该采取该字符串,并围绕标题创建插入括号?然后在insertData中添加该值作为参数?有没有更优雅的解决方案?

let url = System.String.Format("http://ichart.finance.yahoo.com/table.csv?s={0}&g=d&ignore=.csv", "FB") 

let splitLineIntoArray (line : string) = 
    line.Split(",".ToCharArray()) 

let insertData (data : string[]) = 
    // insert data 
    () 

let client = new WebClient() 
let dataWithHeaders = 
    client.DownloadString(url).Split(Environment.NewLine.ToCharArray()) 

let data = 
    dataWithHeaders 
    |> Array.toList 
    |> List.tail 
    |> List.map(splitLineIntoArray) 
    |> List.iter insertData 
+0

您是否想过使用ORM?我认为don syme在这里使用引号来生成sql脚本。 http://blogs.msdn.com/b/dsyme/archive/2011/04/16/soma-sql-oriented-mapping-framework-for-f.aspx – 2012-07-26 20:09:19

+0

您是否正在将数据加载到SQL Server中? – Daniel 2012-07-26 20:12:30

+0

我不确定ORM是否会过度杀伤。这不是很复杂。我正在将数据加载到SQL Server中。我期待能够使用类型提供者,但我不知道这是否有助于这种情况。 – nickfinity 2012-07-26 20:21:08

回答

8

如果你将数据装入SQL Server,您可以使用this excellent CSV reader(免费)和SqlBulkCopy类。它简单而高效。

let loadStockPrices ticker = 
    use client = new WebClient() 
    let url = sprintf "http://ichart.finance.yahoo.com/table.csv?s=%s&g=d&ignore=.csv" ticker 
    use stringReader = new StringReader(client.DownloadString(url)) 
    use csvReader = new CsvReader(stringReader, hasHeaders=true) 
    use con = new SqlConnection("<connection_string>") 
    con.Open() 
    use bulkCopy = new SqlBulkCopy(con, DestinationTableName="<destination_table>") 
    bulkCopy.WriteToServer(csvReader) 

目标表应该与传入数据(OHLC等)具有相同的列。

+0

谢谢,这似乎会做到这一点。如果我用Oracle或其他数据库遇到类似情况,我仍然对其他方法感兴趣。 – nickfinity 2012-07-26 20:28:28

+0

有['OracleBulkCopy'](http://docs.oracle.com/cd/E11882_01/win.112/e23174/OracleBulkCopyClass.htm)。我认为大多数数据库系统都有类似的东西。一次插入一行是非常低效的。 – Daniel 2012-07-26 20:33:31

+0

+1为简化SqlBulkCopy – 2012-07-26 20:34:22

4

编辑:类型供应商可能是一个很好的路要走,但SqlBulkCopy的是高清。以其简单而闻名。

用于插入

类型提供代码:http://msdn.microsoft.com/en-us/library/hh361033(v=vs.110).aspx#BKMK_UpdateDB

type dbSchema = SqlDataConnection<"Data Source=MYSERVER\INSTANCE;Initial Catalog=MyDatabase;Integrated Security=SSPI;"> 
let db = dbSchema.GetDataContext() 

// Enable the logging of database activity to the console. 
db.DataContext.Log <- System.Console.Out 

let newRecord = new dbSchema.ServiceTypes.Table1(Id = 100, 
               TestData1 = 35, 
               TestData2 = 2.0, 
               Name = "Testing123") 
let newValues = 
    [ for i in [1 .. 10] -> 
      new dbSchema.ServiceTypes.Table3(Id = 700 + i, 
              Name = "Testing" + i.ToString(), 
              Data = i) ] 
// Insert the new data into the database. 
db.Table1.InsertOnSubmit(newRecord) 
db.Table3.InsertAllOnSubmit(newValues) 
try 
    db.DataContext.SubmitChanges() 
    printfn "Successfully inserted new rows." 
with 
    | exn -> printfn "Exception:\n%s" exn.Message 

我做了类似的事情。实际上,我在观看Luca Bolognese时写的代码给出了F#的介绍。这实际上会刮去雅虎的饲料,并返回标准的开发。和股价的变化。

全部项目位置:https://github.com/djohnsonm/Stock-Ticker-App

open System.Net 
open System.IO 

let internal loadPrices ticker = async { 
let url = @"http://ichart.finance.yahoo.com/table.csv?s=" + ticker + "&d=6&e=22&f=2011&g=d&a=2&b=13&c=1986&ignore=.csv" 
let req = WebRequest.Create(url) 
let resp = req.GetResponse() 
let stream = resp.GetResponseStream() 
let reader = new StreamReader(stream) 
let csv = reader.ReadToEnd() 
let prices = 
    csv.Split([|'\n'|]) 
    |> Seq.skip 1 
    |> Seq.map (fun line -> line.Split([|','|])) 
    |> Seq.filter(fun values -> values |> Seq.length = 7) 
    |> Seq.map(fun values -> 
     System.DateTime.Parse(values.[0]), 
     float values.[6]) 
return prices} 

type StockAnalyzer (lprices, days) = 
    let prices = 
     lprices 
     |> Seq.map snd 
     |> Seq.take days 
    static member GetAnalyzers (tickers, days) = 
     tickers 
     |> Seq.map loadPrices 
     |> Async.Parallel 
     |> Async.RunSynchronously 
     |> Seq.map (fun prices -> new StockAnalyzer(prices, days)) 
    member s.Return = 
     let lastPrice = prices |> Seq.nth 0 
     let startPrice = prices |> Seq.nth(days-1) 
     lastPrice/startPrice - 1. 
    member s.StdDev = 
     let logRets = 
      prices 
      |> Seq.pairwise 
      |> Seq.map (fun (x,y) -> log(x/y)) 
     let mean = logRets |> Seq.average 
     let sqr x = x * x 
     let var = logRets |> Seq.averageBy (fun r -> sqr (r-mean)) 
     sqrt var