NVMe and NFS, cold cache on client and server. 4.3 GiB in under three seconds.
$ cat /nfs/nvme/Various/UV.zip | pv > /dev/null 4.3GiB 0:00:02 [1.55GiB/s]
The three-disk HDD pool gets around 300 MB/s, but once the ARC picks up the data it goes at NFS + network speed. Cold cache on the client.
$ echo 3 > /proc/sys/vm/drop_caches $ cat /nfs/hdd/Videos/*.mp4 | pv > /dev/null 16.5GiB 0:00:10 [ 1.5GiB/s]
Samba is heavier somehow.
$ cat /smb/hdd/Videos/*.mp4 | pv > /dev/null 16.5GiB 0:00:13 [1.26GiB/s]
NFS over RDMA from the ARC, direct to /dev/null (which, well, it's not a very useful benchmark). But 2.8 GB/s!
$ time cat /nfs/hdd/Videos/*.mp4 > /dev/null real 0m6.269s user 0m0.007s sys 0m4.056s $ cat /nfs/hdd/Videos/*.mp4 | wc -c 17722791869 $ python -c 'print(17.7 / 6.269)' 2.82341681289 $ time cat /nfs/hdd/Videos/*.mp4 > /nfs/nvme/bigfile real 0m15.538s user 0m0.016s sys 0m9.731s # Streaming read + write at 1.13 GB/s
How about some useful work? Parallel grep at 3 GB/s. Ok, we're at the network limit, call it a day.
$ echo 3 > /proc/sys/vm/drop_caches $ time (for f in /nfs/hdd/Videos/*.mp4; do grep -o --binary-files=text XXXX "$f" & done; for job in `jobs -p`; do wait $job; done) XXXX XXXX XXXX XXXX XXXX real 0m5.825s user 0m3.567s sys 0m5.929s
No comments:
Post a Comment