Encrypting Oracle Workloads becomes Cost-effective and Mandatory

Problem Statement and Solution Premise

Problem Statement

Security has become top of mind for CIOs, and CEOs. Encryption at rest is a piece of the solution, but not a big piece. Encryption over the network is another piece, but only a small piece. These and other pieces do not fit together well; they need to unencrypt and reencrypt the data when they move through the layers, leaving clear versions that create complex operational issues to monitor and detect intrusion.

Larger-scale high-value applications requiring high security often use Oracle middleware, including Java and Oracle database.  Traditional security models give the data to the processors to encrypt and unencrypt, often many times. The overhead is large, and as a result encryption is used sparingly on only a few applications. The risk to enterprises is that they may have created an illusion of security, which in reality is ripe for exploitation.

The modern best-practice security model is an end-to-end encryption architecture. The application deploys application-led encryption services in main memory. These services allows end-to-end encryption, in the stack from database to middleware (e.g., Java), locally and across the networks, and across private and hybrid clouds. Oracle and others offer such solutions.

The practical problems of implementing an end-to-end encryption solution are three-fold:

1. The cost of traditional processor solutions introduces additional compute and license cost of at least 100%.
2. To be effective, the end-to-end solution should be applied to the application ecosystem, which again impacts downstream costs and performance.
3. The implementations will almost certainly introduce increased response time and latency variance to applications that are not designed or coded for an end-to-end encryption environment; and this means application users will be being far less productive, leading to serious business impacts.

Solution Premise

The solution premise is to reduce the effective impact of encryption on utilization and response time to be as close to zero as possible. This allows the introduction of encryption integrated throughout the infrastructure stack, never allowing the data to be “clear” across the application, network and data ecosystem.

The premise of this research is that technical solutions to implementing end-to-end always on encryption are available from Oracle and others. These technical solutions use advanced processor techniques designed specifically to process encryption and encrypted data inline in real-time. This research investigates if it is practical to utilize these technologies in the case of high-value Oracle software workloads requiring high levels of security by using an integrated end-to-end encryption solution.

Executive Summary

In earlier research, Wikibon has defined high-value workloads, and analyzed the benefit of optimizing these environments for performance rather than cost. Wikibon has also looked in depth at the development of processor technologies which integrate with traditional processors, as well as the requirement for software to be written into the silicon. Wikibon has found that software performance can be optimized with the utilization of additional silicon such as:

• GPUs (Graphical Processor Units);
• FPGAs (Field-Programmable Gate Arrays);
• Additional Instructions (such as the Intel AES-NI for encryption)
• Coherent memory systems (such as OpenPOWER Foundation OpenCAPI and Nvidia’s NVLink), which allow interrupt-free coherent sharing of data between all the processing elements.

The fundamental benefit of using alternative silicon-based functionality is that many algorithms can work on vectors of data, and require only a small subset of the full instruction set in a traditional processor unit. The chip sets that have focused on extending traditional architectures include ARM (particularly with Nvidia GPUs for screen management), offerings from the OpenPOWER foundation based on POWER8 technologies, and Oracle/Fujitsu’s SPARC investments, particularly the SPARC M7 technology. Encryption is a good example of work that can be offloaded from the processor, enhancing overall system performance.

Figure 1 summarizes the findings of this research. The technologies analyzed are the Power8 (IBM, E880 processor), the Xeon 2.30GHz E5-2699 v3 (HPE Superdome), and the SPARC M7 (Oracle, T7 server series) against a base system of the Xeon E5-2699 V3. The base unit of processing is defined as that done by a single core on an Xeon E5-2699 V3, which represents the current workhorse for general-purpose processing. The analysis is shown with and without encryption, which is assisted in all the configurations by special instructions:

• In the case of Xeon technologies, the additional Intel AES-IN Instructions are deployed for encryption, which sets a base-line for comparison with alternative technologies;
• In the case of POWER8 there are five in-core vector instructions that speed up the AES (Advanced Encryption Standard) algorithm steps, but these are not inline for Oracle database software;
• In the case of SPARC M7 technology, the use of a specialized instructions in the processor pipeline for reducing the encryption overhead.

The cost numbers in Figure 1 take into account:

• System costs optimized for large high-value systems – i.e., used in large-scale single systems significantly greater that 150 cores with large system memories utilizing fast storage (e.g., flash), and are designed for very rapid fail-over and recovery.
• Software license costs
• Hardware and software maintenance over 3 years.

Figure 1 shows the results of a price:performance comparison between the different configurations. The blue bars reflect the 3-year cost/core for each architecture, without encryption. The costs include the high-value hardware configuration and Oracle database software on a per core basis. The best of breed is the SPARC M7 technology, which is 27% lower cost than the nearest alternative architecture.

The red bars in Figure 1 include the cost with encryption, which is always greater than the no encryption case. The best of breed for encryption is again the SPARC M7 technology, which is 79% less expensive than the nearest competitor. The additional cost of running end-to-end encryption is calculated as:

• 99% overhead for the X86-based solutions;
• 60% (estimated) for the POWER8 solution;
• 33% of the base SPARC M7 cost.
• For detailed calculations see Table 1 in the “Price-Performance Analysis” section below with discussion, and Table 3 in Footnotes below.

The SPARC M7 utilized a specialized Inline additional instructions in the M7 chip, designed to work with Oracle database. This is the fundamental reason why much the SPARC M7 chip provides significantly lower overhead than the other processors. The impact on response time to end-users is close to zero (~2%), which means that end-users encrypting Oracle software with SPARC M7-based servers will find little-to-no impact on productivity.

The comparative performance results of each technology with encryption are shown in Figure 2 below.

The best performance is shown in the SPARC M7 technology, which is 15% faster transactional throughput than the POWER8, and over 100% greater than the x86-based systems.

The assumptions used in Figure 1 and Figure 2 are derived from the assumptions and calculations shown in Table 1 and Table 2 in the detailed analysis sections below.

The bottom-line: the use of the encryption engine on the SPARC M7 technology dramatically reduces price performance by using fewer cores and fewer Oracle licenses. The relative cost of running end-to-end encryption on a M7-based systems drops by over half (127%) versus a system built on x86 technology with additional instructions to support encryption. The low overhead of encryption on SPARC M7 processors enables IT decision-makers to mandate the use of end-to-end encryption for all security-sensitive workloads – by far the most effective model for a high-security  business.

Price-Performance Analysis

Table 1 below shows the methodology and assumptions behind Figure1 and Figure 2 above, and Figure 3 below. The analysis is focused on single systems that can scale to at least 150 cores.

• Column a is the relative Oracle database throughput, using a core on an x86 Xeon E5-2699 V3 2.3GHz as the base, arbitrarily set to 1. Wikibon’s analysis of OLTP benchmarks (an open-source database benchmarking tool) run by Oracle results gives a relative figure of 1.55 per Oracle M7 technology core. The same source gives a figure of 1.51 for an POWER8 System core used in the IBM S824. Wikibon’s analysis of other benchmarking sources increased this figure by 8.4% for the higher performing POWER8 System core IBM Power E880. The 4.02 GHz POWER8 processor is chosen to ensure the number of cores in a system constraint could be met, and a score of 1.64/core is used against the x86 E5-2699 base of 1.
• Column b is derived from Oracle’s Processor Core Factor Table. The POWER8 cores are given a factor of 1 by Oracle, and the other systems have a factor of 0.5. This factor is multiplied by the number of cores to determine the number of Oracle Database Licenses that are required.
• Column c is the Wikibon estimate of the overhead for using encryption (using Oracle’s TDE, Transparent Data Encryption, part of the advanced Security Option in Oracle 12c). The analysis is aided by a set of benchmark programs that compared 3 difference database queries and a full database scan (1TB database). The results of this analysis are shown in Table 3 in the Footnotes below. Wikibon calculated the harmonic mean of the resources taken for each of the four programs, and divided the result with encryption by the result without encryption to determine an estimate of encryption overhead. The benchmark is not the same as the benchmark used in column a (it would have been ideal if they were). Wikibon’s methodology used to analyze the results is different from Oracle’s own analysis; but the results and analysis of the encryption technology give good confidence to the figures and the impact of encryption. The resultant encryption overhead shown in Table 3, and used in Table 1, is 33% for the SPARC M7 technology, which is inline.  The overhead is 98% for the x86 E5-2699 V3. The POWER8 has a sophisticated encryption system, but it is not an “inline” implementation. Our analysis indicates that the POWER8 solution is better than the Intel solution, but not as efficient as the SPARC M7 solution. An estimated overhead figure of 60% is used in Table 1. Wikibon is following up to determine if this estimate can be improved, and will republish the results of this research if necessary.
• Column d is an uplift percentage which reflects the fact that faster systems will sell for a higher price that slower systems. Wikibon has developed a figure of $6,664/Core for high-value systems (including fast memory, storage, networking). This is over 4 times higher than a cost optimized core of $1,506/core, but fewer (less than half) performance optimized cores are necessary. The street price uplift on the high-value core prices is estimated as 15% for the HPE Superdome, 20% for the IBM E880 POWER8 servers, and 30% for the Oracle T7 family of servers.
• Column e is the high-value system hardware cost. As discussed in column d above, the cost/core for high-value compute is $6,664. The maintenance is assumed to be 12% of the purchase price/year. The 3-year cost, column e, is calculated as $6,664 x (1 + d) x 1.36 ÷ a.
• Column f is the Oracle database software cost. The Oracle Database12c options included in the Oracle middleware costs were:
  • Enterprise Edition Processor License ($47,500)
  • RAC Processor License ($23,000)
  • Advanced Security Processor License ($15,000)
  • The total cost/core is $42,750, assuming a discount of 50% on the Oracle list price.
  • The maintenance (software update license and support) is assumed to be 22% of the software purchase price/year.
  • The software cost is calculated as f = $42,750 x b x 1.66 ÷ a.
• Column g is the cost of software and hardware, without encryption, g = ( e + f)
• Column h is the cost of software and hardware with encryption,  h = g x (1 + c).

Figure 1 shows the graphical results of column g and h. Figure 3 below shows the same data as Figure 1, but with the system costs and software costs broken out.

Figure 3 below shows the importance of software costs in calculating the total impact of encryption speed-ups. The SPARC M7 is a fast technology, with very efficient encryption, which minimizes Oracle software license cost. The x86 results are very similar, with the software and hardware cost being much higher.

Bottom-line Findings: The ability of the SPARC M7 technology to be fast, with very low encryption overheads, enables the oracle database software costs to be much lower. The POWER8 figures suffer because of the Oracle’s Processor Core Factor Table gives it a rating of 1, compared with 0.5 for everybody else, doubling the Oracle licensing costs of systems based on POWER8,

Performance Analysis

Table 2 shows the relative performance of the four technologies.

Column a is the same as Column a Table 1. It shows the relative core performance, set to a base of the Intel Xeon  2.3GHz E5-2699 v3 core = 1. See the discussion about Column a and how the numbers are derived.

Column j is derived by first calculating and estimating the End-to-end Encryption Overhead,. There is a detailed discussion under column c in the section above. J is created by taking column c (a percentage), adding 1, and then dividing the result into Column a. All the results in Table 2 are relative to the performance of the Intel E5-2699 v3 without encryption.

Figure 2 above is a graphic of Column j. It shows the 99% additional processing required on the Intel Processors.

Figure 4 below is derived from Columns a & j in Table 2. The blue columns show the performance of un-encrypted workloads, with the POWER8 showing a 5% advantage over the SPARC M7 technology. However, the performance of the M7 is 14% better when encryption is required.

Another important finding from the encryption benchmark is the elapsed time for the jobs to run, as shown in Table 3 in the Footnotes:

• The harmonic mean impact on elapsed time for queries and scans on the SPARC M7 is about 2%;
• The harmonic mean impact on elapsed time for queries and scans on the x86 E5-2699s is about 20%.

This means that for the SPARC M7, the impact on other processes needing (say) locked resources will be very low with encryption, which is a major benefit against unintended consequences of implementing end-to-end encryption on end-user productivity.                             –

Bottom-line Finding: The use of special functions to complete encryption significantly reduces the processing overhead, cost, and response time on the SPARC M7, and to a lesser extent, the POWER8 (estimate). The most important conclusion is that end-to-end encryption directly at the point of creation and usage of the data (in main memory) is the modern way of designing security into a system. When required it can be called on as a service. And the architecture is much more robust than (say) just encrypting data on the disk, or in the network. Overall the SPARC M7 architecture has lower encryption overhead and is much safer – making it the best of breed for Oracle systems needing high security. The overheads of 33%, and the low impact on user response time (~2%) make the Oracle M7 the best viable environment for implementing end-to-end encryption for high-value Oracle workloads.

Limitations of Research

This research utilizes benchmarks constructed and run by Oracle. Wikibon has reviewed and adjusted the methodologies used to calculate the results and comparisons. The choice of workloads is reasonable. The specific programs referred to in Table 3 in the Footnotes are varied, and show consistent and expected results.

The projection of the results from Table 3 to Table 1 (two different workloads) give reasonable and expected results, but should be treated with some caution. The projection to the POWER8 technology of encryption overhead is an estimate, and should be treated with caution.

In general, these results of these benchmarks are not broad enough to extend to other workloads with a high degree of confidence. Users should review and test the encryption overheads of their own specific application environment.

Conclusions

Wikibon’s research concludes that the potential for system improvement has moved from traditional cycle-time and throughput measures to improvements in specific functionality, such as the ability to manage data reduction and encryption as a single integrated operation, or to use analytics to automate business procedures (e.g., fraud systems). The use of specialized inline instructions, FPGAs and GPUs can also assist enterprises to write higher function applications and improve performance bottlenecks in either transactional or batch system components.

Wikibon believes that implementation of end-to-end encryption databases, and integrating the encryption processes with specialized encryption off-load functions is the only viable path forward for implementing truly secure systems with acceptable encryption overhead.

Action Item:

Wikibon recommends that large-scale users of high-value applications using Oracle database with high security needs use end-to-end encryption as the most cost effective and most secure architecture. The Oracle SPARC M7-based servers and systems like it that use advanced offload technologies for encryption of systems, and have very low encryption overheads are an important prerequisite for implementing best-of-breed security without impacting business productivity. 

Footnotes:

Tables 3 below shows the calculations of encryption utilization overhead for the SPARC M7 and X86 Xeon technology based on benchmarks, and the calculation of response-time overhead for the same technologies.