Azure NetApp Files Benchmarks

Putting Azure NetApp Files to the test. These benchmarks
show the performance that Azure NetApp Files delivers

image-placeholder

SectionHeading

  • Lorem Ipsum is simply dummy text of the printing
  • Typesetting industry.
  • Lorem Ipsum has been the industry's standard dummy text ever since the 1500s, when an unknown
  • Printer took a galley of type and scrambled it to make a type specimen book.
  • Home
  • Linux
  • Windows
  • Oracle
  • MySQL

Home Directory

As our benchmark testing instrument, we used the File Service Capacity tool, which measures performance in terms of users supported without “overload,” the point at which the input to the system exceeds its processing capability. It also measures the ability to schedule/complete one user scenario in 900 seconds, and 1%-2% overload for a range of users.

Home Directory Workload –
Optimizing Performance

With Azure NetApp Files and multiple DS14v2 Azure Virtual Machines, the overload point occurred just past 14,000+ concurrent users. FSCT workload is metadata, rather than bandwidth, intensive, consuming just about 275MiB/s of bandwidth at the ~14,000 user load point while generating roughly 23,000 input/output operations, made up of ~17,000 metadata operations and ~6,000 read/write operations.

group-62-copy

Though capacity needs in an actual home directory environment vary, the 80MiB of space needed per FSCT user required just over 1TiB of space. In terms of bandwidth, there are two periods to consider for the ANF home directory volume: peak and non-peak usage.

During the peak-usage window, provision 5TiB of capacity to both the Premium Capacity Pool and ANF Volume. At 5TiB of allocated capacity, the ANF volume provides 320MiB/s of bandwidth. Because an ANF Capacity Pool is granular to the terabyte, and the minimum Capacity Pool size is 4TiB, reduce the capacity of both pool and volume to 4TiB during non-peak-usage to reduce costs.

group-36-copy

Linux (Scale Out)

Linux Workload – Throughput

The first graph represents a 64 kibibyte (KiB) sequential workload and a 1TiB working set. The graph shows that a single Azure NetApp Files volume is capable of handling between ~1,600MiB/s pure sequential writes and ~4,500MiB/s pure sequential reads.

Decrementing 10% at a time, from pure read to pure write, this graph shows what can be anticipated using varying read/write ratios [100%:0%, 90%:10%, 80%:20%, and so on].

diagram-3-copy-14

Linux Workload – IOPS

The first graph represents a 4 kibibyte (KiB) random workload and a 1TiB working set. The graph shows that a single Azure NetApp Files volume is capable of handling between ~130,000 pure random writes and ~460,000 pure random reads.

Decrementing 10% at a time, from pure read to pure write, this graph shows what can be anticipated using varying read/write ratios [100%:0%, 90%:10%, 80%:20%, and so on].

diagram-3-copy-16

Linux (Scale Up)

A change has come to the Linux 5.3 kernel, enabling what amounts to single client scale out networking for NFS–nconnect. Having recently completed the validation testing for this client-side mount option with NFSv3, we’re showcasing our results in the follow graphs. Please note that the feature has been present on SUSE (starting with SLES12SP4) and Ubuntu as of the 19.10 release. This feature is similar in concept to both SMB multichannel and Oracle Direct NFS.
The four sets of graphs compare the advantages of nconnect to a non-connected mounted volume. The top set of graphs compares sequential reads; the bottom, random reads. In both sets of graphs, FIO generated the workload from a single D32s_v3 instance in the us-west2 Azure region.

Linux – Read Throughput

Sequential Read: ~3,500MiB/s of reads with nconnect, roughly 2.3X non-nconnect.

group-29-copy

Linux – Write Throughput

Sequential Write: nconnect introduced no noticeable benefit for sequential writes, 1,500MiB/s is roughly both the sequential write volume upper limit as well as D32s_v3 instance egress limit.

group-29-copy-3

Linux – Read IOPS

Random Read: ~200,000 read IOPS with nconnect, roughly 3X non-nconnect.

group-51-copy

Linux – Write IOPS

Random Write: ~135,000 write IOPS with nconnect, roughly 3X non-nconnect.

group-51-copy-4

SMB

For the sake of consistency, and because performing multinode testing using fio with the Windows OS is not supported, the following tests were performed using SMB on Linux rather than SMB from Windows. Methodology validation was confirmed by comparing FIO results collected variously through single Windows 2012, Windows 2016, and Windows 2019 single virtual machines against the results captured from single SLES12SP4 Linux VM.

This work is not meant to validate the use of SMB from the Linux operating system. The use of Linux for SMB testing was simply the most expedient choice.

SMB – Throughput

The first graph represents a 64 kibibyte (KiB) sequential workload and a 1TiB working set. The graph shows that a single Azure NetApp Files volume is capable of handling between ~1,500MiB/s pure sequential writes and ~4,400MiB/s pure sequential reads.

Decrementing 10% at a time, from pure read to pure write, this graph shows what can be anticipated using varying read/write ratios [100%:0%, 90%:10%, 80%:20%, and so on].

diagram-3-smb-14 (1)

SMB Workload – IOPS

The first graph represents a 4 kibibyte (KiB) random workload and a 1TiB working set. The graph shows that a single Azure NetApp Files volume is capable of handling between ~120,000 pure random writes and ~470,000 pure random reads.

Decrementing 10% at a time, from pure read to pure write, this graph shows what can be anticipated using varying read/write ratios [100%:0%, 90%:10%, 80%:20%, and so on].

diagram-3-smb-16 (1)

Oracle

And Then There’s Oracle

As discussed in the orafaq link regarding Direct NFS, Oracle Direct NFS (dNFS) is an optimized NFS (Network File System) client that provides faster and more scalable access to NFS storage located on NAS storage devices (accessible over TCP/IP). Direct NFS is built directly into the database kernel—just like ASM, which is mainly used with DAS or SAN storage.

A good guideline is to use Direct NFS when implementing NAS storage and ASM when implementing SAN storage.

Direct NFS provides faster performance the operating system's NFS driver can provide because Oracle bypasses the operating system and generates exactly the requests it needs (no user configuration or tuning required). Data is cached just once in the user space, which saves memory (no second copy in the kernel space). Performance is further improved by load balancing across multiple network flows.

Direct NFS is the default option in Oracle 18c and has been the default for RAC for many years.

group-38-copy

Enabling or disabling dNFS is as simple as running two commands and restarting the database.
Enable: cd $ORACLE_HOME/rdbms/lib ; make -f ins_rdbms.mk dnfs_on
or
Disable: cd $ORACLE_HOME/rdbms/lib ; make -f ins_rdbms.mk dnfs_off

MySQL

MySQL Workload – Latency Relative to Throughput

The metrics in the graph, which represent benchmarks for MySQL on Azure NetApp Files, are taken from nfsiostat on the database server and as such represent the perspective of the NFS client. In the graph, we see 600MiB/s maximum throughput.

For load testing MySQL in Azure NetApp Files, we selected an industry standard OLTP benchmarking tool and continued increasing user count until throughput reached flatline. By design, OLTP workload generators heavily stress the compute and concurrency limitations of the database engine–stressing the storage is not the objective. That said, the tool used, rather than the storage, was the limiting factor in the graphs.

For this test, the following configuration was used:

  • Instance type: DS15_v2
  • MySQL Version: 10.3.2
  • Linux Version: Redhat Enterprise Linux 7.6
  • Workload Distribution to storage: 70/30 read/write with 4KiB operation database page size*
  • Volume Count: database volume (8TiB Extreme), 1 log volume (1TiB Standard)
  • Allocated Storage Bandwidth: database volume 1024MiB/s, log volume 16MiB/s
  • Database Size: 1.25Ti
group-42-copy