All closed-source frontier labs use tons of open-source all over the stack, starting from python, PyTorch, huggingface all the way down to RoPE, GQA, flash-attention, or any tiny improvements released by open-source players.
The whole transformers architecture (the T in gpT)