Just thought to post a few examples to solidify the information that the possession of a valid Residence Card issued to the non-EEA family member of an EEA national with be exempted from obtaining the visa for that particular EU state if accompanied by the EEA national-
The first one is response given by the Committee of petitions of the EC which can be seen
here. This does not mean that the Residence Card of an EU country can be only used to enter UK, while the favour is not reciprocated by the other EU states.
The second one is
this case in which the petitioner's wife is given permission to travel to Ireland with a British issued Residence Card after the implementation of the Directive while requiring visa for the travel prior to 30.4.2006.
The Schengen Handbook mentioned above not only confirms that a Residence Card should get you entry into Schengen territory without the visa if accompanied by the EEA national in 3.3.1. but also confirms that no stamp should be placed on the passport of the holder of such a permit in page 33.
Denying entrance to a EU Member state while the holder has a valid Residence Card issued by another EU Member state is like denying entrance to the holder of passport of the EU country as well... which when you think about, is no small accusation.