Welcome to the wide and wonderful world of audio drama podcasting! Always nice to "see" new faces!
There are a number of different hosting programs out there, some free, some cheap, and some ... not.

I would suggest, if you are planning to put out shows very regularly, with a lot of ongoing content, and want it all to be available - get in with a reliable service right from the start, something that makes it simple for you to maintain and update. That way, down the line, you're not backtracking and moving stuff from one site to another.
If, however, you're more realistic, starting out, and want to avoid overcommitting (highly suggested), you're probably better off getting into a fee site, and just making sure your show is available. There are a number of folks here who can make suggestions (we all have our favorites), and who might even be able to host "a show" for you, until you decide if you plan to do more.
My feed is through libsyn, and they've generally been good and reliable. Their cheapest rate is $5 a month (plus a one-time $5 extra if you want a personalized "name" for your feed). That's bare bones, no frills, but (since it allows you to upload up to 50MB of material per month) is still probably more than you need, unless you plan to put out more than a single show.
A lot of folks here use wordpress, which is free for a simple blog page, but I believe there is a charge for storage space if you plan to upload files to be available.
There's my two cents.
