perf tools: Avoid unnecessary work in directory lookups