You will struggle to get the combination of cheap to purchase and cheap to run! Depends what you mean by cheap to run, but if you look at the mileage you go and the cost of fuel, then eg a 60mpg diesel vs a 40mpg petrol will mean thousands per year saved on fuel. People know this, so small diesels have something of a premium. Also beware that older 'modern' diesels have things like ECUs, dual mass flywheels, etc which can get costly to fix if (when) they go wrong, and most diesels will have high mileage. Also, beware of ex-taxis.
So, bearing this in mind and rejecting petrol, you are somewhat limited:
Vaux Astra - crude, not that good a car but cheap
Pug 306
Citroen Xsara (Picasso)
Then you're into older (possibly rust), or larger (less economical)
Ford Mondeo
Vaux Vectra
Golf Mk3
If you really don't want diesel and want a petrol, look at Ford Fiesta, Vaux Corsa as cheap to buy and run. Fiesta/Corsa diesels will be a premium. Pug 206 petrols are too unreliable, diesels are a premium.
If you can find one, with a decent rear beam (the bearings wear out), a Pug 106 would be VERY cheap to buy and run. Beware of rust though.