Travel to EU countries when you are a UK resident

[m.kono]

I have exhausted all avenues and thought it prudent to ask my question here in light of the wealth of knowledge that is evident. My question is:
having already acquired residency in the UK prior to getting married to my current partner who is Slovakian, every time I travel to Slovakia or any European country I have to apply for a visa. Is there a way around this?
I am eligible to get British citizenship but wish not to exercise this as Zimbabwe does not allow dual citizenship. So would like to retain my Zimbabwean passport for travelling back home. Can I actually apply for an EU family permit when I already have ILR in the uk?

[Obie]

Nothing in Law prevents you from applying for a Residence Card as confirmation of your right of residence as a family member of an EEA national.

Alternatively, you could travel with your wife, to any of these memberstates, and produce you marriage certificate and your IDs. Provided of course you are allowed by an airline to travel.