geneve: Fix races between socket add and release.