For example, I set firefox to some custom sink using pavucontrol, but now I want to reset it to default. How can I do this without using pavucontrol, just cli?

EDIT:

Found it :) they are in ~/.local/state/wireplumber/restore-stream. To remove target sink, pipewire and wireplumber have to be restarted:

systemctl --user stop pipewire
sed -i '/:target/d' ~/.local/state/wireplumber/restore-stream
systemctl --user start wireplumber

Other solution is just to reroute all active streams to the default sink:

sink="$(pactl get-default-sink)"
pactl list short sink-inputs | sed -r 's/([0-9]+).*/\1/' | while read appId; do 
    pactl move-sink-input $appId "$sink"
done