It's verticle, and the problem with the current set up is the pressure from the water coming down is seeping out of that rubber sheath. I didn't take it off but I'm guessing there's a pretty large size gap between and that the PVC isn't going inside the cast iron pipe, which it should be I would...