2013-03-15 60 views
2

我的工作在数据帧的转换,并在一前一后段错误

Previous Post

阿伦工作阿伦和里卡多,提出了一个辉煌的解决方案(矩阵乘法)来实现我试图做。

这解决方案为小数据集像我在例子中提到,现在我运行它具有以下尺寸的数据帧在同一个解决方案:

Total rows: 143345 
Total Persons: 98461 
Total Items : 30 

现在,当我运行以下命令

A <- acast(Person~Item+BorS,data=df,fun.aggregate=length,drop=FALSE) 

我得到这个错误..

Error: segfault from C stack overflow 

这是becau se,我没有足够的处理/记忆能力。我的机器有4 GB RAM,2.8 GHz i7处理器(Macbook)?我们如何处理这些类型的案件?

+0

多少“BorS”的级别有(是B还是S?) – mnel 2013-03-15 02:17:09

+0

您是否尝试过使用稀疏矩阵(Matrix包)? – ndoogan 2013-03-15 02:38:14

+0

是的只有两个级别B和S.不,我没有尝试过稀疏矩阵,现在我会阅读它。 – user2171177 2013-03-15 03:18:49

回答

4

A data.table解决方案。这是通过聚合,然后再创建新的data.table,并通过引用填充

library(data.table) 

# some sample data 
DT <- data.table(Person = sample(98461, 144000, replace = TRUE), item = sample(c(letters,LETTERS[1:4]), 144000, replace = TRUE), BorS = sample(c('B','S'), 144000, replace = TRUE)) 
# aggregate to get the number of rows in each subgroup by list item and BorS 
# the `length` of each subgroup 
DTl <- DT[,.N , by = list(Person, item, BorS)] 
# the columns you want to create 
newn <- sort(DT[, do.call(paste0,do.call(expand.grid,list(unique(item),unique(BorS))))]) 
# create a column which has this id combination in DTl 
DTl[, comnb := paste0(item, BorS)] 
# set the key so we can join/subset easily 
setkey(DTl, comnb) 
# create a data.table that has 1 row for each person, and has columns for all the combinations 
# of item and BorS 
DTb <- DTl[, list(Person)][, c(newn) := 0L] 
# set the key so we can join/subset easily 
setkey(DTb, Person) 
# this bit could be far quicker, but I think 
# would require a feature request to data.table 
for(nn in newn){ 
    # for each of the cominations extract which persons have 
    # this combination >0 
    pp <- DTl[list(nn), list(Person,N)] 
    # for the people who have N > 0 
    # assign the correct numbers in the correct column in DTb 
    DTb[list(pp[['Person']]), c(nn) := pp[['N']]] 
} 

要完成你initital问题,您可以提取DTb相应列的矩阵

A <- DTb[,-1,with = FALSE] 

results <- crossprod(A) 
+0

你能否从第二行开始解释你的解决方案? – user2171177 2013-03-15 03:27:56

+0

@ user2171177 - 我希望这样做更有意义。 – mnel 2013-03-15 03:32:38

+0

对不起,我的无知... – user2171177 2013-03-15 03:53:41