Married for 9 years and living with EU citizen partner for 2

[MCosta]

Hi, i need some help with this matter, i would really appreciate if anyone can give me any advise.
i came to Northern Ireland in 2003 as a tourist i over stayed and met a girl in the end of the year then in DEC 2004 we got married, i was granted a residency permit in AUG 2007 for 5 years, i this time our marriage broke down and i left our house in 2010 but just MAY 2014 the divorce came through, i met a Lithuanian girl in SEP 2012 and in MAR 2013 we moved in together.
when i applied to renew my residency card in 2012 it was denied firstly because we only sent them the petition for the divorce and second because my ex wife was out of work therefore she was not exercising her citizen rights or whatever... we appealed the decision and was refused again as she was still out of work and we tried to convince that i was earning enough to sustain myself but the HM officer basically said that it was about her as she is the EU citizen.
few months past and they sent me a letter to leave the country or if my situation changed that i should forward the new proof to them, my girlfriend is an EU citizen and we live together for more than 2 years, i wonder if there is anything we can do in this situation??
Is it legal for the HM to hold my passport since 2012??
I am having to go to the police station every week to sign on, does it make sense? if they have my passport already, why do i have to sign on every week? i could not leave the country without my passport. it sounds a bit silly but i really cant understand it.
Thanks and i would like to read your views on this situation.
Have a great day.

[Obie]

They may be wrong.

Is this EU lady from the first marriage still living in Northern Ireland.

How long was she working for after your marriage?

[MCosta]

Hi Obie, thanks for the reply.
The lady from first marriage is an UK/Irish citizen, she is still living in N. Ireland.
She left work due sickness just 3 months after we got married.
what i cant understand is that even knowing that she was not working since 2005 they still granted the residency card for 5 years in 2007 but when i tried to renew it in 2012 the fact that she was not working was an issue then.