mm, vmscan: wake up all pfmemalloc-throttled processes at once

Kswapd in balance_pgdate() currently uses wake_up() on processes waiting
in throttle_direct_reclaim(), which only wakes up a single process.  This
might leave processes waiting for longer than necessary, until the check
is reached in the next loop iteration.  Processes might also be left
waiting if zone was fully balanced in single iteration.  Note that the
comment in balance_pgdat() also says "Wake them", so waking up a single
process does not seem intentional.

Thus, replace wake_up() with wake_up_all().

Signed-off-by: Vlastimil Babka <vbabka@suse.cz>
Cc: Mel Gorman <mgorman@suse.de>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Michal Hocko <mhocko@suse.cz>
Cc: Vladimir Davydov <vdavydov@parallels.com>
Acked-by: Rik van Riel <riel@redhat.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

diff --git a/mm/vmscan.c b/mm/vmscan.c
index dcd90c8..f756a20 100644 (file)
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -3175,7 +3175,7 @@ static unsigned long balance_pgdat(pg_data_t *pgdat, int order,
                 */
                if (waitqueue_active(&pgdat->pfmemalloc_wait) &&
                                pfmemalloc_watermark_ok(pgdat))
-                       wake_up(&pgdat->pfmemalloc_wait);
+                       wake_up_all(&pgdat->pfmemalloc_wait);
 
                /*
                 * Fragmentation may mean that the system cannot be rebalanced