best way to turn boiler off

I know next to nothing about boilers, radiators or heating systems. I am trying to adjust mine so that it will stop putting out the smells that it does. Currently, if I have the boiler on at night, smells seem to come into the bedroom window from the boiler. I am guessing that it is exhaust...