As a stamp collector, you need some way of building up your collection. Apart from the obvious and cheapest method - which is asking friends and family to keep stamps for you instead of throwing them away - you may wish to join a local club. Other resources include online auctions, local stamp dealers, and of course an online Marketplace as featured on this site.
Ebay is the largest, devoting an entire section to ephemera, including stamps. This is a relatively safe way to build up your collection, although costs can mount up quite easily.
Many local dealers and even charities sell packets of stamps by the bagful. There is of course no guarantee that the thousand-or-so stamps in this bag won't contain large numbers of duplicates, but I have managed to complete several issues in my collection by using this method. At the higher end of the scale, you will certainly be able to buy individual stamps at a local dealer. Expect to pay catalogue prices for these though.
Large main dealers
The most well-known is of course Stanley Gibbons. Not only do they buy and sell stamps, they also provide a monthly magazine and have an online Auction section.
Local auction houses
Sometimes stamp collections come up at local auctions - and bargains can be obtained here because they're not specialised philatelic auctioneers. The local telephone directory should tell you if there are any in your area.
Exchanging with other collectors
This is my preferred method of collection - exchanging duplicate stamps with another collector. The problem here lies in finding another collector with duplicates. One solution is to join a local club and register for their exchange packet scheme. However, this has the disadvantage of being a fairly slow process - and not only that, there's no guarantee that the exchange packet you receive will have anything of interest.
This was the reasoning behind setting up this site - providing a platform whereby you can list your duplicate stamps and/or a "wants" list. Taking the format of a Classified advertisement, stamps can be exchanged on a like-for-like basis, or based on mutually-agreed prices. Or simply bought and sold. It's entirely up to you.