I'm surprised no one's said this yet, but the big advantage of PHP for simple sites is that you can share elements between pages. So you might have your header HTML code and if you want to change something about it, you'd have to edit each page one by one. PHP allows you to have one single chunk of HTML code which your web pages can all refer to, so if you want to change something you only have to edit it once.
Hope this makes sense. I've written a post about this on my blog, I'd link to it but I don't want to get banned by the forum as I've only just joined.