drivers/perf: arm_pmu: Avoid leaking pmu->irq_affinity on error