Gainvidya, that 5-year reference is to the date of your first qualifying entry to the UK, and has nothing to do with the ILR dates/time away you've mentioned. Ie, you can apply once you complete 5 years from the date you first landed in the UK in 2004 to begin your stay [in a qualifying category], ...