You can insulate selectively but it means there will be less wall that's cold, meaning any humidity in the air has a smaller area to condense in, meaning you're more likely to get mould on the remaining area.
I think for me it would depend if the uninsulated wall is north facing, but it's a...