Maximizing application performance presents constant challenges. One
day, network latencies slow applications; another day, limits of
processors affect applications; and when everything else is going
smoothly, increasing user demands make I/O bottlenecks appear.
Solving performance problems and I/O bottlenecks can be a difficult
task. While many approaches can reduce I/O bottlenecks, few are cost
Solid state disks are a cost-effective way to improve application
performance. Traditional hard disk based systems incur performance
penalty because of the high access times of hard disk drives. Solid state
disks resolve this problem with low access times and high bandwidth.
To quantify this advantage, this whitepaper discusses two landmark
benchmarks: IOMeter benchmark results and Storage Performance
Council SPC-1 IOPST results. These benchmarks definitively
demonstrate how solid state disks offer unparalleled performance at
Companies who have implemented solid state disks make better use of
their server infrastructure, support increasing concurrent user loads
with low response times, and improve user satisfaction.
Table of Contents
Chapter 1. Problem - Performance Contstrained by I/O Bottlenecks
Chapter 2. Approaches to Solving I/O Bottlenecks
Chapter 3. Solution - Reduce I/O Bottlenecks with Solid State Disks
Chapter 4. Benchmark Results
Chapter 5. Results - Solid State Disk Storage Benefits
Chapter 6. Conculsion
IT professionals spend billions of dollars annually attempting to correct
application performance issues. The Holy Grail for most data center
administrators is deriving the most performance for the least price. I/O
bottlenecks are at the root of many performance issues. These
bottlenecks are extremely frustrating because they are often difficult to
diagnose, and solutions are expensive and rarely long-term.
Performance Constrained by I/O Bottlenecks
Understanding I/O Bottlenecks
An I/O solution without bottlenecks has three main components:
Most data centers have two of these three components. Servers are
blazingly fast with new and better clock speeds reached every few
months. Fast Fibre Channel networks are common, with 2Gbit Fibre
Channel widely implemented and 4Gbit Fibre Channel on the horizon.
The piece missing from most data centers is a high performing storage
device. Figure 1 (below) shows that processor performance has
increased over 25x in the last decade while hard disk drive access
times have barely improved 2x.
- A high-performance server,
- A high-bandwidth network, and
- A high-performance storage device.
So why have hard disk drive access times lagged behind processor
improvements? A quick look at the mechanical characteristics of the
disk drive exposes the problem.
Hard Disk Drive Based Storage Devices Have High Data Access Times
Figure 1: Increasing Gap between Processor Speeds and Disk Access Times
Storage devices tend to publish their performance using data access
times. StorageReview.com defines access times as:
Access Times = Command Overhead Time + Seek Time + Settle Time + Latency
Of these factors, seek time and latency are the biggest variables in
measuring hard drive performance. Seek time refers to "the amount of
time required for the read/write heads to move between tracks over
the surfaces of the platters." Hard drive latency is the time spent
waiting for the disk to spin to the proper location once the read/write
head reaches the right place. The resulting access time is measured in
milliseconds. For even the fastest of hard disk drives, average data
access times exceed 5 milliseconds. Unfortunately, data access times
increase as powerful servers build up larger queues on the disk drives.
Figure 2: Hard Disk Drive and its Mechanical Components
High Access Times Result in I/O Wait Time
Servers today could complete millions of operations during a hard
disk's 5 milliseconds of data access time. As the processor waits on
data to return from the storage device, it does not go about other
calculations; it just waits for the data. This is called I/O wait time.
I/O wait time can be expressed as a percentage of available processor
time. In other words, a system that has an I/O bottleneck will show
high I/O wait percentages and, as a result, lower actual processor
utilization percentages (user time and system time). In UNIX operating
systems, I/O wait time is available through the SAR command. In
Windows operating systems, I/O wait time has to be intuited by
looking at processor utilization and physical disk queues.
Impact of the Performance Gap
As the performance gap between processors and hard disk drives
increases, I/O wait time increases as a percentage of processor
utilization. This trend is combined with trends toward sophisticated
applications and demanding users. The results of the performance gap
ripple through a business by creating:
Low I/Os per second and high I/O wait time
Based on high access times for storage access
Wasted server infrastructure
Caused by businesses responding to performance problems by
buying more processors and more servers.
Long application response times
As wait time builds up
Decreased user productivity and decreased customer satisfaction
Caused by application response time delays
Long running batch jobs
Caused by increasing I/O wait time
Shorter maintenance windows and failure to meet service level
Due to long running batch jobs
Caused by the previous factors
Approaches to Solving I/O Bottlenecks
When confronted with application performance problems, data center
managers often try a variety of techniques. The more common
approaches to solving I/O bottlenecks are:
Adding servers or processors
This is usually a knee-jerk reaction. It is natural to believe that buying a
faster processor will solve application performance problems.
Processor upgrades increase the number of operations the server can
perform per second. However, the new server with faster processors
simply waits on slow storage devices because attached storage is not
any faster. The costs of increasing processor power includes:
Equipment costs - new servers or new processors are expensive.
Support costs - new servers come with new support costs.
Licensing costs - many software products are licensed based on
the type of server and the number of processors.
Management costs - adding servers increases the number of
support personnel required to support an application.
Adding RAM is generally most effective in environments with a very
high percentage of reads from storage following writes of that data.
Adding RAM helps because these reads are cached in the RAM, thus
saving the processor from waiting on data from external hard disks. If
read requests are random and do not follow the writes quickly, the
RAM cache does not help much. Adding RAM does not significantly
affect performance of write-intensive applications. For many servers,
adding RAM is impossible because all of the expansion slots are already
taken. The cost of solving an I/O bottleneck by adding RAM includes:
RAM costs - the cost of RAM varies across server brands. Adding
small amounts of RAM is inexpensive. Adding large amounts of
RAM can be expensive due to the higher costs for higher density
Server costs - it is not unusual for a company looking to add
RAM to have to buy new server equipment to support it.
Licensing costs - in some environments, adding RAM could mean
moving to a different operating system, such as from Windows
Server to Windows Datacenter.
Buying more disk spindles
Adding more disk spindles improves I/O performance by spreading
data access across more disk drives. According to StorageReview.com,
the fastest hard disk drives provide as many as 400 I/Os per second
(compared to hundreds of thousands for solid state disk). Therefore, it
is theoretically possible to assemble a RAID solution with enough
spindles to solve a performance problem. In fact, a quick review of the
existing RAID Storage Performance Council (SPC-1 IOPST) results
shows how RAID manufacturers use high spindle counts to get high
performance. For example, the highest hard disk-based performance
numbers listed by the Storage Performance Council include hundreds
of small capacity disk drives taking up several racks of physical space
and costing millions of dollars. Two items are worth noting here. First,
for a RAID manufacturer to get maximum performance, they must use
low capacity disk drives. This is contrary to the trends of customers to
use high capacity drives in order to increase capacity. Second, even
these massively large and expensive systems only begin to approach
the same performance as a single solid state disk, which is physically
tiny and very cheap in comparison. The work of the Storage
Performance Council (storageperformance.org) really simplifies the
data center manager's job here by showing I/O performance and cost
for I/O across various vendors and configurations. The cost of adding
spindles for performance includes:
Equipment costs - buying disk drives from RAID vendors can be
expensive, and in order to add enough spindles, you might have to
add enclosures. In order to make use of those spindles, you might
need to add controller ports.
Support costs -adding hardware adds to support costs.
Management costs - having a large number of spindles of hard
disk results in extra management cost in order to replace failed
Space costs - large numbers of spindles just for performance uses
additional datacenter space including additional cooling and
Buying Monolithic RAID
Monolithic RAID solutions with large RAM caches help improve I/O
performance in a couple of ways. First, a large RAM cache can improve
read performance if reads tend to quickly follow writes or if a block is
frequently accessed. The effectiveness of the RAM cache for this
depends on the ratio of the cache to the disk. Second, a cached RAID
can speed up disk writes, depending on the write mode selected. RAID
manufacturers allow customers to select between different write-
modes. In a write-through mode (where the cache writes to disk
before notifying the server of a write), a RAID cache does not improve
write performance. In a write-back mode (where the cache notifies the
server that a write is completed as soon as it is written to cache), a
RAID can improve write performance up to a point; determined by a
mixture of the size of the cache and the speed of the back end disks. In
extremely write-intensive environments, even a write-back cache
slows down to the speed of the disks. The Storage Performance
Council test results provide guidance here by providing performance
levels of different storage devices with their cost/performance. Texas
Memory Systems recommends carefully considering the SPC results of
these RAID systems when evaluating this solution next to other
solutions. The cost of adding cached RAID to improve performance
Equipment costs - large cached RAID systems cost hundreds of
thousands to millions of dollars.
Support costs - with the equipment cost comes a hefty
maintenance obligation and added expenses from professional
Management costs -enterprise RAID products are typically
complex and require dedicated staff to manage.
Tuning makes SQL inquiries (that cause storage I/O) as efficient as
possible in their use of external storage. Some companies use internal
programmers and DBAs to tune application code; others supplement
their staff with contractors. Often, special tools are purchased to help
programmers find inefficient code. If an application is inefficiently
written, application tuning can eliminate I/O bottlenecks. An
unfortunate side effect of many tuning exercises is an actual decrease
in user functionality as code that makes too many disk accesses is
simply removed from the application. In addition, it is not unusual for
an organization to tune for too long. In other words, companies often
forget the cost of having programmers working on tuning projects.
Here are some of the costs associated with application tuning:
Staff costs - programming resources are costly in terms of salary
and overhead. Increase this cost if contractors are used.
Opportunity costs - programmers working on tuning could have
been adding functionality to a system.
Software licensing costs - many tuning efforts require or use
special software to aid their tuning efforts.
Summary of Approaches
||Solid State Disk
This section has exposed the difficulty in selecting the proper approach
to solve application performance problems caused by performance
bottlenecks. The next section discusses solid state disks as a preferred
choice for resolving I/O bottlenecks at a low cost-for-performance.
With the performance gap between processors and hard drive-based
storage systems widening, solid state storage is entering the limelight.
Because solid state systems rely on memory chips for data storage,
they offer unprecedented access times, which narrows the gap
between the processor speeds and storage speeds. Companies have
used solid state storage systems to resolve I/O bottlenecks for over
two decades. These systems have become increasingly sophisticated,
higher performing, and lower cost, which sends a clear message .
there is no better tool for improving I/O performance.
Reduce I/O Bottlenecks with Solid State Disks
Understanding Solid State Disks
Solid state disks are a proven technology. In fact, Texas Memory
Systems has designed and manufactured solid state disks for over 25
years. Solid state disks are non-volatile storage devices that use RAM
as the primary storage media. Solid state disks store and access data
directly on RAM chips, which results in storage speeds far greater than
conventional, magnetic storage devices. They achieve non-volatility
through integrated internal battery and disk backup systems.
Solid state disks have the following characteristics:
Lowest possible access times.
SDRAM-based solid state disks have access times below 20
microseconds, the best access times of any storage platform. As a
comparison, a hard disk drive at 5 milliseconds has 250 times higher
The enterprise solid state disk market includes products, such as Texas
Memory Systems RamSan-320, which can support over 1.5GB per
second of random data throughput.
High I/Os per second (IOPS).
Solid state disks offer extraordinarily high random I/O performance
because of their low access times and high bandwidth. For more
information, see the Texas Memory Systems SPC-1 IOPS results,
Low price for performance.
SDRAM based solid state disks provide the best possible
price/performance of all storage devices. For more information, see
the Texas Memory Systems SPC-1 IOPS results, below.
Solid state disks are inherently more reliable than hard disk drive based
systems because their data path does not require moving parts.
All enterprise solid state disk manufacturers offer non-volatile
solutions. These solid state disks have internal batteries that maintain
system power long enough to back-up data in memory to internal hard
Solid state disks are an excellent solution for I/O bottlenecks,
particularly those bottlenecks caused by the high access times of
traditional disk based storage systems.
Solid state disks offer incredible performance and an unbeatable
price:performance ratio. Two independent benchmarks demonstrate
the value and performance of the RamSan-320 Solid State Disk:
Results from IOMeter, a widely used freeware program used to
benchmark storage performance; Results from Texas Memory Systems
SPC-1 IOPST testing. The SPC-1 results, including the cost per SPC-1
IOPS, are audited and verified by the Storage Performance Council.
IOMeter test results
Storage manufacturers use IOMeter, a GNU program publicly available
from Sourceforge.net, to benchmark storage devices. The IOMeter
tool monitors cumulative results across servers. Texas Memory
Systems ran IOMeter on each of the servers shown in Figure 3, above.
The following figures show the resulting performance numbers for the
RamSan-320 with 100% random data traffic.
These IOMeter performance numbers demonstrate that solid state
disks offer the best possible performance levels for random data
accesses. The results confirm a peak I/O performance over 250,000
random I/Os per second at a 2k-block size. In addition, the results
show peak random bandwidth at over 1,500 Megabytes per second
starting with 64k block sizes.
Figure 4, below, shows peak random bandwidth. Note the "Total MBs
per Second" of 1651.87-greater than the Texas Memory Systems
advertised level of 1.5 GB per second.
Figure 4: Peak RamSan-320 Bandwidth Performance Using IOMeter
Figure 5, below, shows peak random I/Os per second (IOPS). Note the
"Total I/Os per Second" of 255508.45-greater than the Texas
Memory Systems advertised level of 250,000 IOPS.
Figure 5: Peak RamSan-320 IOPS Performance Using IOMeter
The results clearly display the value of solid state disk. Solid state disks
excel at both small and large block random data transfers. In
comparison, disk based storage systems truly excel at large block
sequential transfers only.
SPC-1 Benchmark Results with the RamSan-320 Solid State
About the Storage Performance Council
The Storage Performance Council (SPC) is a vendor-neutral
organization dedicated to the development and application of
performance-based storage benchmarks. To ensure the validity of all
performance claims, the SPC requires strict compliance to its
benchmark specifications and full auditing of all published results by an
official SPC auditor and full peer review following a successful audit.
SPC member companies include virtually every key vendor in the
industry, including Texas Memory Systems.
The Storage Performance Council is the first entity dedicated to
helping users understand the value of storage performance and
price:performance. Texas Memory Systems shares these objectives
with the Storage Performance Council. Many storage benchmarks fail
to replicate real world data access patterns. The SPC-1 performance
benchmark was designed to accurately replicate real-world online
transaction processing (OLTP), which is characterized by intense,
random I/O requests made on the storage appliance. This benchmark
is the first in a series of benchmarks to be established by the Storage
Initial, Audited Results of the RamSan-320 SPC-1 Benchmark
In March 2004, Texas Memory Systems and the Storage Performance
Council broke all current I/O performance records while executing the
first published SPC-1 measurement on a solid state disk-the
RamSan-320. The SPC auditor approved and submitted for peer review
the following results:
||RamSan-320 (8 ports)
|Total ASU capacity:
|Data protection level:
||Other Protection Level
|Audit Certification No.:
The Storage Performance Council SPC-1 results highlight that solid
state disks are the best solution when you want the best performance
and the best price:performance. When you look at cost per capacity
only, solid state disks appear expensive. However, when you evaluate
solutions to improve performance, solid state disks offer the best value
of any storage product. In fact, the RamSan-320 produced better SPC-
1 results than equipment priced nearly 10 times higher than the
RamSan-320. The SPC derives price from the total cost of the
implemented solution, including Fibre Channel infrastructure and
relevant support contracts. For details, see the SPC full disclosure
report. For details on other Storage Performance Council results,
please visit StoragePerformance.org.
The SPC-1 analysis produced some additional interesting findings. First,
the Texas Memory Systems test implementation is extremely simple.
Where other storage products tested under SPC-1 have included
dozens of host bus adapters and switched fabrics, the RamSan simply
included eight servers each with one host bus adapter connected to
the RamSan-320. Secondly, the response time graphs from the test
results show that, even under the heaviest loads, the RamSan never
took more than 70 microseconds to return data to the server. For
other storage devices, as load increased response times increased. For
test details, including the graphs mentioned here, see the full disclosure
report on StoragePerformance.org.
Results: Solid State Disk Storage Benefits
Replacing hard drive based storage with a RamSan solid state disk has
numerous benefits in improving I/O performance:
Improved Server Efficiency
When slow, conventional storage holds back the potential of expensive
processors and servers, efficiency is reduced and money is wasted.
Conversely, introducing a fast RamSan solid state disk fully utilizes
those servers, resulting in maximized ROI. If your data only travels as
fast as the slowest point in the network, then removing that bottleneck
results in efficiency gains throughout the system.
The drive towards server consolidation means squeezing every drop of
performance out of the remaining servers. When a RamSan can
improve server efficiency, then that efficiency increase can lead to
server consolidation without performance loss. This is especially true
in "server-bloated" environments, where adding additional servers or
processing power solved the problem of I/O wait time. Consolidating
servers and moving the hottest data to a RamSan Solid State Disk
reduce TCO (total cost of ownership) across the enterprise.
More Concurrent Users
When you install a RamSan solid state disk, it typically takes the
pressure off of whatever system was being thrashed in its place (a
RAID array, server system memory, etc.). This approach frees up
those resources for other applications and tasks. In the case of query-
based applications, adding a RamSan solid state disk can increase the
number of concurrent users receiving their data at higher speeds.
Conventional thinking suggests that adding concurrent users requires
more servers. With the RamSan, you scale concurrent users by
improving server efficiency.
Faster Response Times
RamSan solid state disks are famous for consistently decreasing the
response times of demanding applications. Without mechanical storage
devices to slow down performance, users and applications get data at
the speeds they demand. At the core of any enterprise is a critical
database. Whether employees, customers, or other servers query it,
anyone can benefit from faster response times.
Increased Customer Satisfaction
In many environments, particularly online transaction processing
(OLTP), increased customer satisfaction is the first priority. Eliminating
I/O bottlenecks with a RamSan solid state disk can improve the
performance of all hardware depending on that data. Whether the
application is e-commerce, OLTP, hot files storage, or any other use,
higher performance, faster response, and greater transactions, means
increased user satisfaction.
Financial, telecom, and e-commerce industries know the value of
increased transactions per second. To those industries, every
additional transaction that their hardware can carry out directly affects
the bottom line. In such a situation, it is easy to see how the RamSan
solid state disk quickly pays for itself. This logic, however, can be
applied to virtually any mission-critical application that requires a solid
state disk to reach its potential. When you compare the cost of a
RamSan to alternative solutions that increase application performance
from 2x to 10x, the choice becomes easy.
Diagnosing performance problems is easier when administrators can
prioritize their potential causes better. Hard disk drive access times
are a major cause of I/O bottlenecks. While hard disk drive capacities
are increasing, access times are improving at a snail's pace. Therefore,
while you may find some performance gains by buying new servers or
tuning application code, these solutions do not provide profound
results. Solid state disks, non-volatile systems that store data on RAM,
are a cost-effective solution to I/O bottlenecks.
Before the Storage Performance Council published independent
benchmarks, finding cost-effective performance solutions caused data
center managers to rely on marketing and gut instinct only. The
Storage Performance Council and their benchmarks are an excellent
source of unbiased information on storage performance and its costs.
The Texas Memory Systems results show that the RamSan-320 was
able to achieve both the highest SPC-1 IOPS result and the lowest cost
per SPC-1 IOPS. If you need performance, these results clarify that
solid state disks are the least expensive solution. Companies that have
implemented solid state disk have seen numerous benefits to the
bottom line by making better use of their existing servers, adding
concurrent users, reducing response time delays, and improving
RamSan Products Overview
Request a Price Quote
Oracle Performance Center
Solutions and Applications