Funnily enough that was the reason I expected but probably the only one he didn't give. During our discussion he implied he would be willing to replace the existing wall with a similar stone wall, of the same height. I think the height problem is the key - he's a very nosey person and likes to be able to see what's going on, even in other peoples gardens. Plus I think him and his wife are pretty lonely and they like being able to easily strike up a conversation with us. Like I've said previously, we don't mind this but it can be a bit much especially when we've got friends round. Plus I want to protect myself and him from the certainty of my son breaking his greenhouse.
It's a difficult situation all round.
I must say you are being really decent about the whole thing. Most people I know, with young children and a nosy old neighbour poking his head over the wall every 5 minutes, would put up a new fence without even asking!
Personally I'd explain to him, given his response, you have no choise but to erect up a legal 2m fence on your side. It's within your right to do this. In addition to your greenhouse concerns, explain to him you want to better protect your children and property from crime, the weather, the issue of dogs, foxes etc jumping over the wall and into your garden.
Then gauge his reaction. Say of course you would rather demolish the 'unsafe' wall and put up a fence on the official boundry line, just like the fence he has on the other side.
Maybe he'll come round to the idea. If not, build it anyway.