Exercising treaty rights and timeline

I was wondering what it would mean if you had exercised your treaty rights in the past and then returned with your non-EEA family members to your country of origin (which may subject you to national law if you do not exercise treaty rights).

What I mean is that you move to another country within the EEA , from your country of origin or from outside Europe ,to work and live there for example and then decide to leave, say to the US or Japan (or any other country outside the EEA). After some time you return to your country of origin. Would your exercising of treaty rights in the past still count?