Want to get married but don't have passport

[frank4man0011]

Hi,
I am going to make this simple and clear as I need to have my options made clearly to me as well.

I had a Post Study Work Visa (PSW) for 2years due to expire on Sept 2013
May 2013 --------- Got married to a French national (with UK marriage certificate issued to legalize our marriage)
June 2013--applied for a RC as an EEA family member. A French citizen
Feb 2014 (8mnths after application)----sent a change of address letter to HO but they never got it apparently.
April 2014 (9mnts after application) ----HO case worker visited old address hence refused my application because we don't live there anymore. HO held my passport!
May 2014----filed for appeal at the upper tribunal
Oct 2014---- Appeal was heard and I lost the appeal on the ground that the Judge felt satisfied that the marriage is more probable than not to be a marriage of convenience. Absolutely unjust!
**Note Home Office argued on the hearing date that my marriage was a marriage of convenience but they actually refused me because they did not meet me and my then wife at home on conducting the visit.
The complications we (me and my wife) were going through in addition to the immigration trouble led her to give up, file for divorce and went to France 
No I am in a new relationship with another EEA national with a chance of it developing into marriage as she want to have children but would like to be married before starting a family. I’ve known her for over 3years as we were on an off in relationship before I initially got married to the French national I mentioned earlier.
The problem now is that HO has detained my passport because the PSW visa on it has expired, but I would like to get married to my new partner and give my application another shot. However, I am fully aware that the registrar will not allow us to get married unless I have a passport. What can be done?
I greatly appreciate your input on this matter.