Set the dialog windows to be 'transient for' the toolbox window. In XFCE4 this makes...