[Plugin: Password Protect WordPress] Is it just me…?

It must be just you.
Maybe a stupid question, but – have you logged out first?

No – that is put there to force the page to be refreshed so that some caching plugins will work.