I don't think there is any evidence that non-EU and non-Commonwealth have it more difficult than others.
Perhaps you misunderstood me. I am not saying that it is harder to get ILR for non-EU non-Commonwealth migrant when you are eligible for it. But it is much harder to complete this journey. If you are from EU country, you can simply come to UK, live here for 5 years and apply, you can even manage it without getting any visas. If you're from British Commonwealth, it is much harder than EU because you will need visa first and then visas throughout your journey. But at least people from BC countries have some good relations and some cultural links with UK. Also some of BC countries are similar in economical terms.
However, if you are from a country like Iran, Afghanistan, Russia, China, Somalia, etc. - it is hardest. Not only your original country is outside many international agreements, you are also on a different economical stage of development, unless you were born in very rich family or became rich. Africa is also tough but at least many African countries had/have some trade/cultural links with UK in the past.
The hardest of them all is probably North Korea because you also can't get out of it in the first place.
It is very different depending on your original country. Imagine, 20 years old Canadian from Canadian or Norwegian village and compare with the same 20 years old from Chinese or Somalian village. Who have better chances of getting UK ILR in their life?