Asbetsos that's breaking down really is a hazard, but if you can't get it removed professionally, it's often better to leave it and do what's called 'encapsulating' which effectively glues it all together to stop the fibres drifting free.
There's a good article here...