This is my dataset
set.seed(2026)
n <- 1000
img_id <- sprintf("IMG_%04d", seq_len(n))
strata <- sample(c("A", "B", "C"), size = n, replace = TRUE, prob = c(0.8, 0.1, 0.1))
split <- sample(c("train", "test", "val"), size = n, replace = TRUE, prob = c(0.7, 0.2, 0.1))
dataset <- data.frame(img_id, split, strata)
dataset %>% count(split, strata)
split strata n
1 test A 154
2 test B 21
3 test C 9
4 train A 573
5 train B 64
6 train C 77
7 val A 79
8 val B 12
9 val C 11
It's an unbalanced dataset. I am trying to proportionate the stratification, as it is proposed here. I want to sample 300 images from training, 80 from val and 80 from test.
allocation <- dataset %>%
count(split, strata, name = "Nh") %>%
group_by(split) %>%
mutate(N_split = sum(Nh)) %>%
ungroup() %>%
left_join(target, by = "split") %>%
mutate(
nh = round(Nh / N_split * n_target)
) %>%
arrange(split, strata)
split strata Nh N_split n_target nh
<chr> <chr> <int> <int> <dbl> <dbl>
1 test A 154 184 80 67
2 test B 21 184 80 9
3 test C 9 184 80 4
4 train A 573 714 300 241
5 train B 64 714 300 27
6 train C 77 714 300 32
7 val A 79 102 80 62
8 val B 12 102 80 9
9 val C 11 102 80 9
Nh: available images per Strat, per split. N_split: sum of images per split. n_target: how many images I would like to have per split. nh: proportionated stratification.
Now, let's see the test split. For Strat B, there are still 12 images available (Nh - nh). I would like to make a condition for all strats: if less than 20 images were computed in nh, take all the available images and discount them from the strata with the biggest representation, in this case, strat A, in order to match with the desired amount of sampled images per split.
برچسب:
نویسنده: استخدام کار