If it is buried in the plaster, just lengths used for the switch drops, it is OK to leave.
It will have been left over from the original conduit used when the house was built. the drops to the switches are often resused as it saves chasing out the plaster at a rewire.
A couple of points:
If it is just odd short drops, buried in plaster, it does not have to be earthed; and if it is the thin, pressed-steel (unthreaded) stuff it is fairly difficult to fit a reliable earth anyway. However, if you have any exposed conduit, you should earth it with a clamp. This is particularly important in a bathroom.
The thin conduit has quite a sharp edge. Behind a switchplate this is no great problem because no-one will be yanking the cable (although it would be preferable to fit a bush or grommet). However, in the loft, you might find that the end of the conduit is exposed and the cable comes out of it and then bends horizontally. This is a place where a bush would have been valuable, but I bet you haven't got one. To fit a bush you need to take the cable out of the conduit . At the very least, nail a piece of wood or something over this end so that no-one will accidentally tread on it and push the cable against the sharp edge. If the cable comes out of an elbow, facing in the direction of the cable run, it's not so bad.
The thin old "slip-connector" conduit is obsolete now, I don't know if you can still get fittings for it.
If you do end up taking it out, you can use oval plastic conduit, which does not need earthing and is not sharp.
