I want my family to return to UK by September due to my daughter's nursery. I think if I apply 'COE' in Malaysia the processing time might delay their travel so I prefer applying in UK
I do understand that applying in Malaysia is ideal however, I prefer applying in UK as long as the application is valid. Please can you share your view on this?
Thanks in advance
I am proceeding on the assumption that your new-born was born after your were naturalised. As he was born in Malaysia, your son is automatically a British citizen by descent (British Nationality Act 1981, s.2(1)(a)). Technically, if your child wished to travel to the UK, he must have either (i) a British citizen passport or (ii) a certificate of entitlement to the right of abode (COE) in his Malaysian passport (Immigration Act 1971, s.3(9)(e)).
Strictly speaking, a person with the right of abode cannot be granted leave to enter (whether in the form of entry clearance or a stamp in the passport at the port of entry; see Immigration Act 1971 s.1(1); Immigration Rules r.7A; Policy Manual ROA18). So again strictly speaking, your new-born child cannot get an "on arrival visa" (what you mean is leave to enter) as they are exempt from immigration control under the Immigration Act.
Now, as a matter of Border Force operational practice, a person whom an immigration officer (IO) presumes to have the right of abode but who cannot prove this by producing a UK passport or COE will normally be asked to wait around for a bit while the IO makes some further inquiries. In practice, it may be decided to grant the child notional "leave" in his Malaysian passport (though this is technically not possible, as outlined above), or the child may be admitted without obtaining notional "leave". It is conceivable that the IO at the desk won't grasp the situation, but you shouldn't count on it. If you, as a British passport holder, travel with your ILR wife and a new-born child with nothing other than a Malaysian passport, the assumption is that the child is a British citizen and must have documentation to prove this.
In summary: from a legal perspective, you really should apply for the COE in Malaysia. However, you will be aware that an out-of-country COE application costs a whopping £423. The in-country application isn't much cheaper, at £321. What is stopping you from applying for a full passport for your child who is, as outlined in the first paragraph, a British citizen like yourself? That would set you back a "mere" £53 (plus courier fee). You should not assume that obtaining a COE is going to be more straightforward than a passport...
I am aware that Malaysia does not allow dual citizenship as a general rule. However, the problem is that your child is a British citizen, whether or not he has a UK passport. And of course, if he did not have British citizenship, he would not be entitled to a COE. If your real concern is the Malaysian authorities becoming aware of your child's dual nationality, won't a COE in his MALAYSIAN passport be even more of a signal than his having a second passport? I don't quite understand the rationale here...
But to answer your substantive question: of course your application would be VALID if made in the UK. But you are very likely to face an issue at the port of entry when you come back from Malaysia.