Just to be clear about this, as a Japanese citizen he doesn't need a visa to visit the UK, whether the visit is to see you, his civil partner, or to photograph Big Ben. You say he was refused entry last time because he didn't have a visa to visit you, but the refusal must have been because of doubts about his long-term intentions - in other words, he must have said something which suggested that he did not intend just to stay for a short visit, or he was too vague about where he planned to go or what he would do after his visit, so they thought he was moving in with you. If he's been spending more time in the UK than elsewhere since May, the suspicion must be that he has no serious prospects or intentions outside the UK.
In view of the refusal, it might be best if he applies for a visit visa. He doesn't have to go back to Japan to do that, he can apply in Holland or Belgium. The advantages of getting a visa are that he can apply as a Family Visitor, which gives him a right of appeal if it's refused, and once issued the Immigration Officer can only cancel it if he can show that deception was employed in the application or that there has been a change of circumstances since then.
Although your finances and ability to accommodate him are significant in any application, you should pay very close attention to the wording of the refusal notice, and address the reasons given in detail. If he has no job and is just subsisting in Europe for short periods in between visits to you, I can understand why he was refused entry, and he could be refused a visa for the same reasons.
Alternatively, you have to consider a settlement visa, if he really does want to come and stay with you in the UK. Whether he can apply for that outside Japan depends on his immigration status where he is staying now. Investigate all the requirements very carefully, it's an expensive process. Start here:-
http://www.ukba.homeoffice.gov.uk/visas ... s-settled/
