I am trying to get my head around how to use data.tables. It is not going well.
I have a large data.table with a bunch of retus and AUM. I subsetted that data.table into two data.tables, one with retus, and one with AUM. I now want to subset the retus data.table, to get only the retus from funds with AUM less than the 50th percentile.
To give you an idea, this is my code:
fundDetails <- data.table(read.table("Fund_Deets.csv", sep = ",", fill = TRUE, quote=""", header=TRUE))
fundNAV <- data.table(read.table("NAV_AUM.csv", sep = ",", fill = TRUE, quote=""", header=TRUE))
allFundDetails <- fundDetails[Currency == 'USD']
allFundNAV <- fundNAV[Fund.ID %in% allFundDetails$Fund.ID]
allFundAUM <- allFundNAV[Type == 'AUM', -c(1,3), with = FALSE]
allFundAUM <- setnames(data.table(t(sapply(allFundAUM[,-1, with = FALSE],as.numeric))), as.character(allFundAUM$Fund.ID))
allFundRetus <- allFundNAV[Type == 'Retu', -c(1,3), with = FALSE]
allFundRetus <- setnames(data.table(t(sapply(allFundRetus[,-1, with = FALSE],as.numeric)/100)), as.character(allFundRetus$Fund.ID))
smallFundRetus <- data.table(sapply(allFundRetus, function(x) rep(NA, length(x))))
This Produces the following three tables (smallFundRetus is obviously just NA's):
> allFundAUM[,1:10, with = FALSE]
33992 33261 38102 33264 33275 5606 41695 40483 41526 45993
1: NA NA NA NA NA NA NA NA 1 27
2: NA NA NA NA NA NA 117 NA 1 27
3: NA NA NA NA NA NA 120 NA 1 27
4: NA NA NA NA NA NA 133 NA 1 27
5: NA NA NA NA NA NA 146 NA 1 29
---
260: NA NA NA NA NA NA NA NA NA NA
261: NA NA NA NA NA NA NA NA NA NA
262: NA NA NA NA NA NA NA NA NA NA
263: NA NA NA NA NA NA NA NA NA NA
264: NA NA NA NA NA NA NA NA NA NA
> allFundRetus[,1:10, with = FALSE]
33992 33261 38102 33264 33275 5606 41695 40483 41526 45993
1: NA NA NA NA NA NA NA NA 0.0188 -0.0116
2: NA NA NA NA NA NA -0.0315 NA -0.0120 0.0134
3: NA NA NA NA NA NA -0.0978 NA -0.0908 -0.0206
4: NA NA NA NA NA NA -0.0445 NA -0.0269 -0.0287
5: NA NA NA NA NA NA 0.0139 NA 0.0298 -0.0141
---
260: NA NA NA NA NA NA NA NA NA NA
261: NA NA NA NA NA NA NA NA NA NA
262: NA NA NA NA NA NA NA NA NA NA
263: NA NA NA NA NA NA NA NA NA NA
264: NA NA NA NA NA NA NA NA NA NA
> smallFundRetus[,1:10, with = FALSE]
33992 33261 38102 33264 33275 5606 41695 40483 41526 45993
1: NA NA NA NA NA NA NA NA NA NA
2: NA NA NA NA NA NA NA NA NA NA
3: NA NA NA NA NA NA NA NA NA NA
4: NA NA NA NA NA NA NA NA NA NA
5: NA NA NA NA NA NA NA NA NA NA
---
260: NA NA NA NA NA NA NA NA NA NA
261: NA NA NA NA NA NA NA NA NA NA
262: NA NA NA NA NA NA NA NA NA NA
263: NA NA NA NA NA NA NA NA NA NA
264: NA NA NA NA NA NA NA NA NA NA
for (i in 1:nrow(allFundRetus)){
theSubset <- as.vector(allFundRetus[i,] <= as.numeric(quantile(allFundAUM[i,], .5, na.rm = TRUE)))
theSubset[is.na(theSubset)] <- FALSE
theSubset <- colnames(allFundRetus)[theSubset]
smallFundRetus[i,theSubset, with = FALSE] = allFundRetus[i,theSubset, with = FALSE]
}
I am trying to subset using this for loop (using a for loop in an attempt to debug):
for (i in 1:nrow(allFundRetus)){
theSubset <- as.vector(allFundRetus[i,] <= as.numeric(quantile(allFundAUM[i,], .5, na.rm = TRUE)))
theSubset[is.na(theSubset)] <- FALSE
theSubset <- colnames(allFundRetus)[theSubset]
smallFundRetus[i,theSubset, with = FALSE] = allFundRetus[i,theSubset, with = FALSE]
}
This produces an error:
Error in `[<-.data.table`(`*tmp*`, i, theSubset, with = FALSE, value = list( :
unused argument (with = FALSE)
I tried removing the 'with' part, but this spits out a bunch of waings:
> waings()
Waing messages:
1: In `[<-.data.table`(`*tmp*`, i, theSubset, value = c("41526", ... :
Supplied 3020 items to be assigned to 1 items of column '41526' (3019 unused)
2: In `[<-.data.table`(`*tmp*`, i, theSubset, value = c("41526", ... :
Supplied 3020 items to be assigned to 1 items of column '45993' (3019 unused)
3: In `[<-.data.table`(`*tmp*`, i, theSubset, value = c("41526", ... :
Supplied 3020 items to be assigned to 1 items of column '45994' (3019 unused)
4: In `[<-.data.table`(`*tmp*`, i, theSubset, value = c("41526", ... :
I am confused on how to do this. Any ideas on how I can subset the second data.table by the subset on the first?
