Will travelling together to husband’s country for several months jeopardise application for his spouse visa extension?

I’m British, my Ghanaian husband joined me on a spouse visa in April 2018; we had a baby in May 2018. He’ll be applying to extend his visa in Dec. 2020. I’d appreciate any advice on whether travelling together, with our child, to my husband’s home country for around 3 months a year is likely to jeopardise his application for leave to remain after 2.5 years.
Since he arrived, we have been to his home country three times, me for around 4 weeks each time and he for around 5 weeks: he went a week ahead of me on two occasions. Each time, I did about a week’s work (meetings and fieldwork, I’m a researcher and we met when I was on fieldwork there years ago) but also several weeks holiday seeing his elderly and unwell mother. We’ve also had to make several other short trips together for my work to other countries when the baby was nursing and under 12 months old. I have read that when applying for leave to remain after 2.5 years on a spouse visa, there are no official absence limits (and anyway he would be inside them, as he has not spent more than 180 days outside the UK in any annual rolling period). We are wondering whether the absences to his home country in particular are going to jeopardise his application for visa extension. He is keen to visit his mother again soon, and reasons that as long as he stays within the 180 day limit he’s OK. We can’t find official advice on this but would like to gather others’ opinions and experiences, if possible. Many thanks for any guidance.
For further info, when it comes to his extension, we are quite sure we be able to provide all the other necessary documentary evidence, i.e. income and accommodation requirements etc.
I’m new to posting, though I’ve been reading this board avidly for advice over the past 2 years. Hope I’ve stuck to the forum rules…

There is no absence threshold for spouse visa holders at the moment.

Thanks a lot for that reply, Seagull! I'm glad to get confirmation that there are indeed no official absence thresholds...
I'm aware of that technical side of it, so to speak, but would also be glad to hear if anyone has any opinions on the subjective impression this could give the case worker. I'm wondering if anyone's ever been refused leave to remain on the basis that the British spouse (me in this case!) could just as well go and live their partner's country, (in our case Ghana), as they've visited there frequently in the past. Or, whether folks think this is a likelihood. Friends have mooted this possibility, so we are torn between visiting his elderly parents and fear that this may lead to us having to stay there permanently if we would like to stay together as a family.
Of course, I'm aware that I'm asking people to make speculations here on the basis of experience, rather than point me to the facts.
Many thanks again!

Furthermore despite there is no absence threshold but still the heavier absences will affect the generation of cohabitation documents which are vitally needed for extension. Also the purpose of spouse visa is to live and settle with UK sponsor/family in UK.

Thanks again ! I suppose that reply is a little away from my precise question, and I don't learn so much from it as I've seen it posted earlier in reply to another thread . But in another way it's helpful as it helps me understand the mindset that might be in action when the visa extension decision's made! For info, these visits are holidays: we reside in the UK and we still have all our bills, payslips and bank account statements being sent to our house in the UK (which I own), so these absences would not affect the generation of those cohabitation documents.

Despite the cohabitation evidences are generating even someone is not present in UK but how about if caseworker matches them with heavier absence (if any). If you still think of not being answered then better to conduct a deeper search of the forum or wait for someone with refusal experience due to absence which i haven't seen yet.

There is no '180 days' absence limit for holders of a spouse visa.

Thanks very much for the helpful replies! This is a rather subjective one I know, so it's really helpful to get all the different possible angles on it. Cheers.