markyyyyyy, good evening, again.
There are / were in general two types of this material.
1/. A roll of polystyrene was / is available, you glue on to the wall
Or.
2/. The wall paper had this material pre-fitted and was applied as a normal but thick wall paper.
If you are removing all the paper probably best to remove the polystyrene at same time it is a well known spreader of flame and produces a toxic smoke when ignited. By now there is a high probability that the glue line has / is failing and in my opinion getting back to a solid plaster is no bad idea, also the original plaster will be [hopefully] fairly streetlight both ways and you will not have to allow for any bulges due to the polystyrene ?
Ken.