This upcoming move is rarely followed but it's worthwhile, mainly because it saves you a large number of problems later on. Use some latex primer more than the filler, as this can demonstrate up any leftover imperfections. Leave to dry as proposed and sand by using a fantastic sandpaper quality https://paintingthepastuk.co.uk/products/wood-paint-blue-no-3