Yes, breathing asbestos is very bad for you long-term.
It is almost certainly asbestos cement, so (presuming it is in fair condition), it will not release fibres readily UNLESS you try to remove it with an angle grinder / lump hammer / saw.
It's a soil stack, so I presume that it will have a (healthy?) concentration of not-so-friendly bacteria inside it - that lot won't do you a lot of good.
I guess you'll be up a ladder for at least part of the job - falling of a ladder will kill you a lot quicker than breathing asbestos fibres.
Not all "specialist" asbestos contractors will do a satisfactory job - apart from the way they actually go about the work (with little consideration for the work at heights issue, I'd be willing to bet), it is not uncommon for them to treat the job casually "because it's only cement" - debris is commonly left on the customer's backyard.
If you desperately don't want to do it yourself, shop around - prices vary considerably. As previously mentioned,
www.arca.org.uk is a good place to start. But must importantly if you go down this route, make sure that the "specialists" take it down in as whole a piece as possible, and absolutely do not leave any debris at all. I'd expect them to vacuum any fragments up with their hazardous waste vacuum cleaner - absolutely not your yardbrush.
All of the following only applies to the private individual - if you are doing this for reward (ie at work), the law is much more prohibitive).
If you did decide to do it yourself:-
1. where will the waste go? Civic amenities sites sometimes accept small amounts of domestic asbestos waste, which lessens the chance of it ending up being fly-tipped. Contact them to find out whether they will accept it, in what quantity, and how it needs to be packaged. Otherwise, you may need to pay a specialist to take away the neat pile you leave them at the end of your drive.
2. how do you get the waste to the waste site? You don't really want it in the back of your car, whether correctly packaged or not. A (covered) trailer would be better.
3. how do I not get contaminated? Rules - don't release dust. Don't breathe dust. Don't get dust on your clothes. Don't leave debris. Don't fall off the ladder! Don't get sh#t on you! Wear a respirator, disposable overalls (with the sleeves fully down AND the hood fully up). Wear as little as possible under these protective outer garments - all will be potentially-contaminated, and therefore is much better disposed of as contaminated than kept.
4. How do I get the stuff down? See rules above. Cut the brackets, NOT the asbestos cement. Ease the sections apart, DON'T cut them / smash them. If you do drop any bits, damp them down and pick them up, put them into the waste bag - DON'T brush or vacuum the waste. Usually, the cement sections would be wrapped in 1000 guage polythene, taped up with polycloth tape, them wrapped in polythene a second time.
5. When finished, hose the area down, washing any residual dust away. Strip off the protective clothing and respirator, and dispose as contaminated. Have a thorough shower.
For further advice, try
http://www.hse.gov.uk/asbestos/essentials/index.htm (pick any appropriate pdfs), for instance
http://www.hse.gov.uk/pubns/guidance/a14.pdf
These documents were written to try to ensure that tradespeople had guidance on how to do "low risk" tasks with asbestos - the principles apply just as equally to the DIYer.
