Certainly better to be safe than sorry and get it tested.
Depending on how old it is, the fibres could actually just be horse hair. It used to be used historically to give lime plaster more tensile strength. It stopped being used commonly back in the 50s but is still used today by restoration...