What is the bare minimum a person needs to produce a quality recording? By quality, I mean the ability to edit out buzzes, etc. (if this is even possible).
That depends what you consider inexpensive? If you don't have a mic preamp and don't have a mic you may consider one of the small digital recorders with inbuilt microphones for instance the ZOOM recorders - I hear many who are doing fine recordings with them and the recording can be imported into your computer. Moreover you will not be recording the unwanted noise from the computer if you use one of these devices. I would look for one with the posibility of adding external mic also - this will allow you to ad external mics later at improve the recording by mic placement. If you want to buy a microphone I would look for a condensator mic - but it is true that good microphones are expensive. In the long run it will be too expensive to buy cheap stuff as you will need to upgrade again and again.