With these dates when will I qualify for ILR?

The guidance says not apply more than 28 days before the end of the qualifying period and if you do, they may refuse the application with no refund of the fee. However, you must apply before your current permission to stay in the UK ends.

I was granted a five year visa work permit on 20/05/2008
The visa work permit expires on 20/05/2013
I entered the UK on 23/05/2008 on the basis of the same five year visa work permit.
When will I qualify to apply for ILR?
I want to apply in person (premium service).

If I pop in between 18/05/2013 and 19/05/2013, will they say that I do not meet the qualifying period because I entered the UK on 23/05/2008 and that I should first apply for a three days extension?

What does "not apply more than 28 days before the end of the qualifying period" actually mean?

You can apply 28 days before 20/05/2013. This is the 28 day rule.

Difference between 20/05/2008 and 23/05/2008 doesn't matter as you entered within 90 days of your entry clearance.