Yes, the item was a dimmer controlling a group of LED lamps. And yes, they flashed until I replaced one of the LED lamps with a filament lamp. All was OK after that.
I assume the LED lamps have a switcher design PS so, when 'off', it trickled up and discharged hence the flash.
Ah, the benefits...