There's been a shift in recent decades to take a much more relaxed view of ivy than previously - it'll act as insulation, keeping the wall warmer in the cold, and cooler in the heat, and provide food for birds in the winter. There's some solid info here...