If you try and remove the moss you will remove the stone chippings that are protecting the felt from UV light.
If the felt is old as the previous chap said it will probably be blistered and will crack when walked on.
The moss does help to absorb UV and help maintain the temperature of...