See if one of your three channels has less noise than any other, if so, Use only that one, for example by using the shift channels effect and then use Colorama or tint to re-introduce your duotone look.
You can also try duplicating the layer, set the duplicated layer’s transfer mode to darken, then use a lur or median, or perhaps the best choice:
dust & scratches:
first set the radius as small as possible but so you get rid of all the white noise. (it’s in essence a median)
then slowly adjust the threshold untill you start to see the noise again, for me it’s usually between 8-24%,
for example: 16 -> now the median is only applied to pixels that differ more than 16 levels from their neighbouring pixels, so soft grain and tone variations survive but harsher specks with more contrast get medianed out.
By using the darken mode, you make sure that black detail survives, and only white bugs gets squashed.