I've tested Clausewitz-engine games (as you list) on my SSDs and found that load times work on a strange principle. Namely, you want the game on a separate drive from the OS, but it doesn't much matter what kind of drive. Has to do with the preparation of small files off the OS drive (Users folder). Having a faster OS drive can improve the load time but having the game on that same drive will increase load times. Yes, I've tested this. I had it load as fast on a HDD (yes, mechanical) as NVMe - the speed is limited by the OS files, and moreover having the game on the OS drive made is slower than a HDD. (the game I tested here was HOI4 - tracking the I/O, it seems bottlenecked by files in Users that are generated on startup)
But more generally: depends on the game. Unity-engine games, for example Pathfinder, Pillars of Eternity, etc., can find some benefit on NVMe (up to 15% in my testing). Online games also can sometimes benefit. I would say the majority of games see little to no benefit, however. I think this might change in the future as the upcoming consoles are designed to leverage NVMe for game loading. As for now, the best indicator does tend to be low queue depth 4K read performance - which is why Optane leads the way, followed by SMI-based drives, with E12-based falling behind. But these differences are small and even DRAM-less drives (like the SN550) can load fast under the right circumstances (general read performance), since you're bottlenecked elsewhere.