VMware vSAN 8 Express Storage Architecture (ESA) for Oracle Workloads – Performance

I need more and more and …….!!! - The ever-growing demand of Business-Critical workloads especially Oracle workloads where CPU, Memory & IO intensive operations drives critical business transactions to achieve business SLA' and targets in a timely fashion.

Enabling, sustaining, and ensuring the highest possible performance along with continued application availability is a major goal for all mission critical Oracle applications to meet the demanding business SLA's which can be easily achieved on vSAN 8 vSAN Express Storage Architecture (ESA).

Key points to take away from this blog

This blog is an exercise to showcase the performance advantages of using vSAN 8 vSAN Express Storage Architecture (ESA) for business-critical production Oracle workloads.

Disclaimer - This blog is not meant to be a performance benchmarking-oriented blog in any way.

Remember, any performance data is a result of the combination of hardware configuration, software configuration, test methodology, test tool, and workload profile used in the testing below - so the performance improvement I got with the synthetic workload generator in my lab is in NO way representative of any real production Customer workload , which actually means the performance improvements for real production Customer workloads will be way much better.

vSAN Express Storage Architecture (ESA)

The vSAN Express Storage Architecture (ESA) is an optional, alternative architecture in vSAN that is designed to process and store data with all new levels of efficiency, scalability, and performance. This optional architecture is optimized to exploit the full capabilities of the very latest in hardware.

It was introduced in vSAN 8, and enhanced in vSAN 8 U1 and U2, it can be selected at the time of creating a cluster. The ESA in vSAN is an alternative to the Original Storage Architecture (OSA) found in all previous editions of vSAN, as well as an optional architecture in the very latest version.

vSAN ESA will unlock the capabilities of modern hardware by adding optimization for high-performance, NVMe-based TLC flash devices with vSAN, building off vSAN's Original Storage Architecture(vSAN OSA). vSAN was initially designed to deliver highly-performant storage with SATA/SAS devices, the most common storage media at the time.

vSAN 8 will give our customers the freedom of choice to decide which of the two existing architectures (vSAN OSA or vSAN ESA) to leverage to best suit their needs.

More information on vSAN Express Storage Architecture (ESA) can be found at the

vSAN Express Storage Architecture (ESA) - New high-performance RAID - 5/6 erasure coding

Customers can expect RAID-5/6 to perform equal to RAID-1 with vSAN ESA. The new architecture will deliver optimal levels of resilience that are also space efficient while offering the highest levels of performance. All of this can be accomplished using RAID-6 with the assistance of the new log-structured filesystem and object format.

More information on vSAN ESA New high-performance RAID - 5/6 erasure coding can be found at in Announcing vSAN 8.

Test Bed

The Test bed is an 8 Node vSAN 8 Express Storage Architecture (ESA) Cluster with the setup as shown below -

• vCenter version was 8.0.2 build 22385739
• 8 Lenovo ThinkAgile VX7531 Node servers, 2 sockets, 28 cores per socket, Intel(R) Xeon(R) Gold 6348 CPU @ 2.60GHz, 1TB RAM
• VMware ESXi, 8.0.2, 22380479 with vSAN ESA All NVMe Storage
• Oracle 21.13 with Grid Infrastructure, ASM Storage and Linux udev (8k database block size)
• OEL UEK 8.9

Details of vSAN Express Storage Architecture (ESA) cluster 'env175' are shown as below. The ESA Cluster has 8 Lenovo ThinkAgile VX7531 Node servers as shown below.

Each Lenovo ThinkAgile VX7531 server has 2 sockets, 28 cores per socket, Intel(R) Xeon(R) Gold 6348 CPU @ 2.60GHz, 1TB RAM as shown below.

Each Lenovo ThinkAgile VX7531 server has 6 internal NVMe drives, each server contributing 6 Internal NVMe drives to the vSAN ESA datastore capacity as shown below.

Lenovo ThinkAgile VX7531 server 'env175-node1.pse.lab' HBA & Internal NVMe drive details are as shown below.

vSAN Express Storage Architecture (ESA) cluster Storage policies are shown as below.

Baseline Oracle Policy 'Oracle ESA - FTT0 - NoRAID´ was created to JUST get baseline metrics for comparison purposes, besides that, no true production workload is ever setup with FTT=0 (Failures to Tolerate) and No RAID (no protection).

• Oracle ESA - FTT0 - NoRAID
• Oracle ESA - FTT1 - R5
• Oracle ESA - FTT2 - R6

Oracle VM 'Oracle21C-OL8-DB_ESA' details are shown as below.

The VM has 28 vCPU's, 256 GB RAM, the single instance database 'ORA21C' was created with created with multi-tenant option & provisioned with Oracle Grid Infrastructure (ASM) and Database version 21.13 on O/S OEL UEK 8.9.

Oracle ASM was the storage platform with Linux udev for device persistence. Oracle SGA & PGA set to 64G and 10G respectively.

All Oracle on VMware platform best practices were followed.

Oracle VM 'Oracle21C-OL8-DB_ESA' has 4 vNVMe Controllers for added performance.

58 are assigned to the Oracle VM 'Oracle21C-OL8-DB_ESA' which are shown as below -

• NVME 0:0 - 80G for OS (/)
• NVME 0:1 - 80G for Oracle Grid Infrastructure & RDBMS binaries
• NVME 0:2 - 100G for GRID_DG
• NVME 0:3 - 200G for DATA_DG
• NVME 0:4 - 200G for DATA_DG
• NVME 0:5 - NVME 0:12 - 8 vmdk's, each 25G - REDO_DG

• NVME 1:0 - 1:14 - 15 vmdk's, each 50GB - SLOB_DG

• NVME 2:0 - 2:14 - 15 vmdk's, each 50GB - SLOB_DG

• NVME 3:0 - 3:14 - 15 vmdk's, each 50GB - SLOB_DG

Oracle VM 'Oracle21C-OL8-Customer' vmdk's storage policy details are as below

• Test 1 - Test run was with ALL vmdk's with Storage Policy 'Oracle ESA - FTT0 - NoRAID'
• Test 2 - Test run was with ALL vmdk's with Storage Policy 'Oracle ESA - FTT1 - R5'
• Test 3 - Test run was with ALL vmdk's with Storage Policy 'Oracle ESA - FTT2 - R6'

Oracle ASM Disk Group details are as below:

grid@oracle21c-ol8-nvme:+ASM:/home/grid> asmcmd lsdg
State Type Rebal Sector Logical_Sector Block AU Total_MB Free_MB Req_mir_free_MB Usable_file_MB Offline_disks Voting_files Name

MOUNTED EXTERN N 512 512 4096 4194304 409600 117916 0 117916 0 N DATA_DG/

MOUNTED EXTERN N 512 512 4096 4194304 102400 102296 0 102296 0 N GRID_DG/

MOUNTED EXTERN N 512 512 4096 1048576 204800 140614 0 140614 0 N REDO_DG/

MOUNTED EXTERN N 512 512 4096 1048576 2304000 124998 0 124998 0 N SLOB_DG/
grid@oracle21c-ol8-nvme:+ASM:/home/grid>

Test Use case

This blog is an exercise to showcase the performance advantages of using vSAN 8 vSAN Express Storage Architecture (ESA) for business-critical production Oracle workloads.

SLOB workload generator was run against a 2 TB SLOB tablespace with 3 different storage policies.

• Oracle ESA - FTT0 - NoRAID
• Oracle ESA - FTT1 - R5
• Oracle ESA - FTT2 - R6

SLOB 2.5.4.0 was chosen as the load generator for this exercise with following SLOB parameters set as below:

• UPDATE_PCT=30
• RUN_TIME=300
• SCALE=25G
• WORK_UNIT=1024

Multiple SLOB schemas were used to simulate a multi-schema workload model and the number of threads per schema was set to 20.

Work Unit size was set to 1024 to drive the maximum amount of IO without stressing REDO to study the performance metrics differences between the 3 different storage policies on vSAN ESA.

We ran multiple SLOB runs for the above use case and compared the Oracle metrics as shown in the table below.

Baseline Oracle Policy 'Oracle ESA - FTT0 - NoRAID´ was created to JUST get baseline metrics for comparison purposes, besides that, no true production workload is ever setup with FTT=0 (Failures to Tolerate) and No RAID (no protection).

We can see that vSAN Express Storage Architecture (ESA) New high-performance RAID 5/6 erasure coding database metrics is comparable to the baseline.

As mentioned earlier, Customers can expect RAID-5/6 to perform equal to RAID-1 with vSAN ESA. The new architecture will deliver optimal levels of resilience that are also space efficient while offering the highest levels of performance. All of this can be accomplished using RAID-6 with the assistance of the new log-structured filesystem and object format.

Diving deeper into core database load profiles, we see similar database metrics for Executes (SQL), Transactions, Read/Write IOPS and Read/Write MB/secs for both 'Oracle ESA - FTT1 - R5' and 'Oracle ESA - FTT2 - R6' storage policy runs

• Executes (SQL) / second & Transactions / second are comparable across all 3 runs
• Read IO requests / second & Write IO requests / second are comparable across all 3 runs
• Read IO (MB) / second and Write IO (MB) / second are comparable across all 3 runs

Diving deeper into core database wait events, we see similar database wait events with similar wait times for 'Oracle ESA - FTT1 - R5' and 'Oracle ESA - FTT2 - R6' storage policy runs.

Looking at GOS statistics, we can see comparable performance metrics for both 'Oracle ESA - FTT1 - R5' and 'Oracle ESA - FTT2 - R6' storage policy runs.

In conclusion, we were able to get comparable performance metrics from both overall database and GOS perspective for both 'Oracle ESA - FTT1 - R5' and 'Oracle ESA - FTT2 - R6' storage policy runs.

Summary

This blog is an exercise to showcase the performance advantages of using vSAN 8 vSAN Express Storage Architecture (ESA) for business-critical production Oracle workloads.

This blog is not meant to be a performance benchmarking-oriented blog in any way. Remember, any performance data is a result of the combination of hardware configuration, software configuration, test methodology, test tool, and workload profile used in the testing below - so the performance improvement I got with the synthetic workload generator in my lab is in NO way representative of any real production Customer workload , which actually means the performance improvements for real production Customer workloads will be way much better.

Baseline Oracle Policy 'Oracle ESA - FTT0 - NoRAID´ was created to JUST get baseline metrics for comparison purposes, besides that, no true production workload is ever setup with FTT=0 (Failures to Tolerate) and No RAID (no protection.

We can see that vSAN Express Storage Architecture (ESA) New high-performance RAID 5/6 erasure coding database metrics is comparable to the baseline.

As mentioned earlier, Customers can expect RAID-5/6 to perform equal to RAID-1 with vSAN ESA. The new architecture will deliver optimal levels of resilience that are also space efficient while offering the highest levels of performance. All of this can be accomplished using RAID-6 with the assistance of the new log-structured filesystem and object format.

Diving deeper into core database load profiles, we see similar database metrics for Executes (SQL), Transactions, Read/Write IOPS and Read/Write MB/secs for both 'Oracle ESA - FTT1 - R5' and 'Oracle ESA - FTT2 - R6' storage policy runs

• Executes (SQL) / second & Transactions / second are comparable across all 3 runs
• Read IO requests / second & Write IO requests / second are comparable across all 3 runs
• Read IO (MB) / second and Write IO (MB) / second are comparable across all 3 runs

Diving deeper into core database wait events, we see similar database wait events with similar wait times for 'Oracle ESA - FTT1 - R5' and 'Oracle ESA - FTT2 - R6' storage policy runs.

Looking at GOS statistics, we can see comparable performance metrics for both 'Oracle ESA - FTT1 - R5' and 'Oracle ESA - FTT2 - R6' storage policy runs.

In conclusion, we were able to get comparable performance metrics from both overall database and GOS perspective for both 'Oracle ESA - FTT1 - R5' and 'Oracle ESA - FTT2 - R6' storage policy runs.

Acknowledgements

This blog was authored by Sudhir Balasubramanian, Senior Staff Solution Architect & Global Oracle Lead - VMware.

Conclusion

The ever-growing demand of Business-Critical workloads especially Oracle workloads where CPU, Memory & IO intensive operations drives critical business transactions to achieve business SLA' and targets in a timely fashion.

Enabling, sustaining, and ensuring the highest possible performance along with continued application availability is a major goal for all mission critical Oracle applications to meet the demanding business SLA's which can be easily achieved on vSAN 8 vSAN Express Storage Architecture (ESA).

All Oracle on VMware vSphere collaterals can be found in the url below.

Oracle on VMware Collateral - One Stop Shop
https://blogs.vmware.com/apps/2017/01/oracle-vmware-collateral-one-stop-shop.html

Public link

Please use the above public link if you want to share this noodl on another website.