svcrdma: Tail iovec leaves an orphaned DMA mapping
authorChuck Lever <chuck.lever@oracle.com>
Tue, 13 Sep 2016 14:52:50 +0000 (10:52 -0400)
committerJ. Bruce Fields <bfields@redhat.com>
Fri, 23 Sep 2016 14:18:52 +0000 (10:18 -0400)
commitcace564f8b6260e806f5e28d7f192fd0e0c603ed
treee20368a823dcfb07faeea2d9bdfb21c6dbd43358
parentbec782b4fccd395bacb6f99187a9131f7826cbd1
svcrdma: Tail iovec leaves an orphaned DMA mapping

The ctxt's count field is overloaded to mean the number of pages in
the ctxt->page array and the number of SGEs in the ctxt->sge array.
Typically these two numbers are the same.

However, when an inline RPC reply is constructed from an xdr_buf
with a tail iovec, the head and tail often occupy the same page,
but each are DMA mapped independently. In that case, ->count equals
the number of pages, but it does not equal the number of SGEs.
There's one more SGE, for the tail iovec. Hence there is one more
DMA mapping than there are pages in the ctxt->page array.

This isn't a real problem until the server's iommu is enabled. Then
each RPC reply that has content in that iovec orphans a DMA mapping
that consists of real resources.

krb5i and krb5p always populate that tail iovec. After a couple
million sent krb5i/p RPC replies, the NFS server starts behaving
erratically. Reboot is needed to clear the problem.

Fixes: 9d11b51ce7c1 ("svcrdma: Fix send_reply() scatter/gather set-up")
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
Signed-off-by: J. Bruce Fields <bfields@redhat.com>
include/linux/sunrpc/svc_rdma.h
net/sunrpc/xprtrdma/svc_rdma_backchannel.c
net/sunrpc/xprtrdma/svc_rdma_recvfrom.c
net/sunrpc/xprtrdma/svc_rdma_sendto.c
net/sunrpc/xprtrdma/svc_rdma_transport.c