First use of array is slow

I have encountered a weird "bug" in PHP and since I'm a novice I'm at the end of my knowledge.

I'm developing a TYPO3 extension that has some major performance issues with data, or so I thought. It tus out that the first use of the array, which stores all the objects I got from my database query, is taking way to long. Every use or loop after that is fast again.

The code looks like this:

        $productsArr = $this->productRepository->findByDetail($category, $properties);

        $newSortArr = array();
        $familyProductList = array();

        $counter = count($productsArr);
        /** @var Product $product */
        for($i = 0; $i < $counter; $i++) {

            //it takes to long to do this
            $product = $productsArr[$i];

            if(!empty($productsArr[$i])) {
                $newSortArr[$product->getInFamily()->getUid()][] = $product;
            }
        }

It doesn't matter where I first use the object array. The first use of the array is always taking around 30 sec.

Has anybody encountered something similar? If you need more information I will gladly provide that.

Thanks in advance!