insulation chimney

Hi, I have just cut a hole in my boarded over fireplace for ventilation as ppaint has been flaking off. I found it has been stuffed with loft type fluffy insulation, Im not sure if this may hold damp ? If I leave it in I guess I just need to put some ducting to get a chanel of air through.