curtain rail secure fixings lath plaster

I'm having massive problems with securing the fixings for a B&Q 3.2m wooden curtain rail. I'm either hitting lath and plaster or plaster then solid brick. Using the hammer action to get in to the brick is damaging the surrounding plaster. I can't get any secure fixing and now have 3 horrible...