If you did want to fill it and do the job properly you would need to rake out the joint and re fill it with mortar the same colour, point it and brush it. But in all honesty, there is no need what so ever, as it really will not cause a problem at all.