i2c: mux: demux-pinctrl: invalidate properly when switching fails