Did you ask them not to stamp the passport?
You should ask them not to stamp your passport. It will help them understand European free movement law to realize that they are not giving you permission. If they insist them refer them to their own law:
STATUTORY INSTRUMENT
S.I. No. 226 of 2006
European Communities (Free Movement of Persons) Regulations 2006
Permission for Union citizens and qualifying family members to enter the State
[...]
4. (4) An immigration officer shall not, at the point of entry, place a stamp in the passport of a qualifying family member who presents to the officer a valid residence card.
Note that a non-Irish Residence Card counts now:
http://www.immigrationboards.com/viewtopic.php?t=78980