Basically the answer is no, but he may need an entry visa to Luxembourg (which should be trivially granted) and you will need to be working, looking for work, or studying.
Will you be working there, or otherwise exercising your treaty rights?
Where do you live now?
Once you settle there, he can apply for a Residence Card, which confirms his right to live and work there (though he can work before he gets it).
What does the Luxembourg embassy say?
Directive 2004/38/EC is the key piece of legislation for family members of EU citizens.
http://eumovement.wordpress.com/directive-200438ec/
