You may not want to cause a fuss, but one day you may wish to sell your property, and buyers may well be put off if they spot the flue being a nuisance, or knock you down on price.
Its in your interest to be as diplomatic as possible, and to do your research properly. I'd certainly speak to...