What ever you do, do not buy one from your local supermarket, chances are that within a couple of months, if not less, you will wish you had a better bike. Also, if there are any issues with the bike and you need to return it for repairs you are going to have to deal with someone who does not know the firs thing about a bicycle.
I would say that if you are looking just to keep fit and am not too serious about getting into cycling you should spend no more than 3500 to 4000, my assumption is that you want to buy an MTB?
Go to a bicycle shop that specializes in cycling, tell them how much you want to spend and ask them what they can do for you at that price. Raleigh and Mongoose have good fairly cheap entry level bikes.