WordPress (I tested versions 2.2.3, 2.3, and 2.3.1) works just fine with PHP5 (5.2.3, 5.2.4, 5.2.5). No issues whatsoever – in fact it is officially supported configuration!
Shirohagen, you obviously messed up somewhere. From your description it sounds like you actually turned off PHP at all!