The drying time will depend on the depth of basecoat applied (if not boarded).
It is important for the plaster to be allowed to dry fully, as decorating over this will cause paint to fail, flake and blister and wallpaper not to adhere.
Any furniture or pictures/mirrors, put up against these walls, will also trap in moisture.
There is no real given time, as there are a lot of variables, concerning plaster depth, surface applied to and drying conditions.
I would never try to force the drying though, just keep the area dry and warm.
6 weeks is often given as a rule of thumb, prior to painting. I would say with regards to the drilling of holes, that a weeks drying would be sufficient.