Most bell boxes have 20 minute cut off timer built-in to them which should stop any noise after that time if there is a problem. It has been known that these can fail.Post photo of your bell to see if it needs replacement.
Very old C,H,S type boxes did not have cut off timers.