No issue with it damaging pipes but it's the wrong stuff. Driers and won't move enough.
Pipes through walls should be sleeved and caps put around to make them look tidy.
Pipes expand with heat causing them to rub against masonry.
Most don't sleeve and use a natural cure silicone or polymer mastic around the seal in. That's your best bet.
Always use correct mastics for any job.