When implementing an LSI Nytro WarpDrive (NWD) or Nytro MegaRAID (NMR) PCIe flash card in a Linux server, you need to modify quite a few variables to get the best performance out of these cards.

In the Linux server, device assignments sometimes change after reboots. Sometimes, the PCIe flash card can be assigned /dev/sda. Other times, it can be assigned /dev/sdd, or any device name. This variability can wreak havoc when modifying the Linux environment variables. To get around this issue, assignments by the SCSI address should be used so all of the Linux performance variables will persist properly across reboots. If using a filesystem, use the device UUID address in the mount statement in /etc/fstab to persist the mount command across reboots.

Cut and paste the script
The first step to solving is to cut and paste the following script, except the SCSI address (highlighted in yellow), into /etc/rc.local. You’ll need to enter the SCSI address of the PCIe card before executing the script.

nwd_getdevice.sh
ls -al /dev/disk/by-id |grep 'scsi-3600508e07e726177965e06849461a804 ' |grep /sd > nwddevice.txt
awk '{split($11,arr,"/"); print arr[3]}' nwddevice.txt > nwd1device.txt
variable1=$(cat nwd1device.txt)
echo "4096" > /sys/block/$variable1/queue/nr_requests
echo "512" > /sys/block/$variable1/device/queue_depth
echo "deadline" > /sys/block/$variable1/queue/scheduler
echo "2" > /sys/block/$variable1/queue/rq_affinity
echo 0 > /sys/block/$variable1/queue/rotational
echo 0 > /sys/block/$variable1/queue/add_random
echo 1024 > /sys/block/$variable1/queue/max_sectors_kb
echo 0 > /sys/block/$variable1/queue/nomerges
blockdev --setra 0 /dev/$variable1

The highlighted SCSI address above needs to be modified with the SCSI address of the PCIe flash card. To get the address, issue this command:

ls –al /dev/disk/by-id

When you install the Nytro PCIe flash card, Linux will assign a name to the device. For example, the device name can be listed as /dev/sdX, and X can be any letter. The output from the ls command above will show the SCSI address for this PCIe device. Don’t use the address containing “-partX” in it. Be sure to note this SCSI address since you will need it to create the script below. Include a single space between the SCSI address and the closing single quote in the script.

Create nwd_getdevice.sh file
Next, copy the code and create a file called “nwd_getdevice.sh” with the modified SCSI address.

After saving this file, change file permissions to “execute” and then place this command in the /etc/rc.local file:

/path/nwd_getdevice.sh

Test the script
To test this script, execute it on the command line exactly how you stated it in the rc.local file. The next time the system reboots, the settings will be set to the appropriate device.

Multiple PCIe flash cards
If you plan to deploy multiple LSI PCIe flash cards in the server, the easiest way is to duplicate all of commands in the nwd_getdevice.sh script and paste them at the end. Then change the SCSI address of the next card and overlay the SCSI address in the newly pasted area. You can follow this procedure for as many LSI PCIe flash cards as are installed in the server. For example:

nwd_getdevice.sh
ls -al /dev/disk/by-id |grep 'scsi-1stscsiaddr83333365e06849461a804 ' |grep /sd > nwddevice.txt
awk '{split($11,arr,"/"); print arr[3]}' nwddevice.txt > nwd1device.txt
variable1=$(cat nwd1device.txt)
echo "4096" > /sys/block/$variable1/queue/nr_requests
echo "512" > /sys/block/$variable1/device/queue_depth
echo "deadline" > /sys/block/$variable1/queue/scheduler
echo "2" > /sys/block/$variable1/queue/rq_affinity
echo 0 > /sys/block/$variable1/queue/rotational
echo 0 > /sys/block/$variable1/queue/add_random
echo 1024 > /sys/block/$variable1/queue/max_sectors_kb
echo 0 > /sys/block/$variable1/queue/nomerges
blockdev --setra 0 /dev/$variable1
ls -al /dev/disk/by-id |grep 'scsi-2ndscsiaddr1234566666654444444444 ' |grep /sd > nwddevice.txt
awk '{split($11,arr,"/"); print arr[3]}' nwddevice.txt > nwd1device.txt
variable1=$(cat nwd1device.txt)
echo "4096" > /sys/block/$variable1/queue/nr_requests
echo "512" > /sys/block/$variable1/device/queue_depth
echo "deadline" > /sys/block/$variable1/queue/scheduler
echo "2" > /sys/block/$variable1/queue/rq_affinity
echo 0 > /sys/block/$variable1/queue/rotational
echo 0 > /sys/block/$variable1/queue/add_random
echo 1024 > /sys/block/$variable1/queue/max_sectors_kb
echo 0 > /sys/block/$variable1/queue/nomerges
blockdev --setra 0 /dev/$variable1

Final thoughts
The most important step in implementing Nytro PCIe flash cards under Linux is aligning the card on a boundary, which I cover in Part 1 of this series.  This step alone can deliver a 3x performance gain or more based on our in-house tests as well as testing from some of our customers. The rest of this series walks you through the process of setting up these aligned flash cards using a file system, ASM or RAW device and, finally, persisting all the Linux performance variables to the card so these settings are persisted across reboots.

Links to the other posts in this series:

How to maximize PCIe flash performance under Linux

Part 1: Aligning PCIe flash devices
Part 2: Creating the RAW device or filesystem
Part 3: Oracle ASM

Tags: , , , , , , , , , , ,
Views: (1812)


During the past few years, the deployment of cloud architectures has accelerated to support various consumer and enterprise applications such as email, word processing, enterprise resource planning, customer relationship management and the like. Traditionally, co-located servers, storage and networking moved to the cloud en masse in the form of a service, with overlying applications that have been and remain very insensitive to delay and jitter.

But the fast-emerging next generation of business applications require much tighter service level agreements (SLA) from cloud providers. Applications such as Internet of Things, smart grids, immersive communications, hosted clients and gaming are some good examples. These use cases tend to be marked by periods of high interactivity, so delay and jitter for the network, computer and storage must be minimized.  During times of normal interactivity, the applications are in steady-state condition, requiring minimal SLAs from the infrastructure resources.

Emerging use cases drive demand for two-tier cloud architectures
These emerging use cases are driving the rise of two-tier cloud architectures. The key for these architectures to succeed is efficiency: they must be cost-effective to deploy and guarantee a tight SLA for applications while leaving the rest of the carrier and cloud infrastructure unchanged. What’s more, the application service needs to move closer to the end user, but only for the duration of the real-time interaction. These measures help ensure that the customer’s application-specific requirements for delay and jitter are met without requiring major upgrades to the carrier or cloud infrastructures.

In this two-tier cloud architecture, the first cloud tier, also referred to in the industry as centralized cloud, is where the applications typically reside. The second cloud tier is invoked on demand, and the application’s virtual machine along with its relevant network, application and storage data shift to this tier. Keep in mind that the second tier can be instantiated as part of an existing service provider network element or as a stand-alone infrastructure element closer to the end user.

A connected patient heart monitor provides a useful example. During most of its operational time, the device may be collecting data only periodically, and with no need for any interactions with medical staff. But when the heart monitor detects an abnormality, the application hosted in the cloud must instantly be moved closer to the user in order to provide interactivity. For this use case, the second tier cloud must host the application, assess the patient’s condition, retrieve relevant historical information and alert the medical staff for a possible medical response.

The key, then, is to move applications from tier one to tier two clouds seamlessly. LSI® Axxia® multi-core communication processors feature an architectural scalability for network acceleration and computer cluster capabilities that provide this seamless bridge between the two clouds. In order for the two-tier cloud architectures to thrive, they need three fundamental elements:

a.       On-demand resource provisioning
Many cloud datacenters are squarely focused on deploying end-to-end resource provisioning tools to improve efficiency. Not the least among these is the fast-growing end-to-end orchestration ecosystem for OpenStack® software, though there are many proprietary solutions. End-to-end orchestration tools need to be aware of all the second-tier cloud datacenter components. In some cases, OpenStack is even being deployed to boot up second tier cloud components. However, a big challenge remains – maintaining a steady state and full capabilities of various distributed second-tier cloud components.

b.       Efficient virtual machine movements
For tiered cloud architectures to thrive, they must also transfer enough network, application and storage data to sustain continuing operations of the application at the second tier. However, many of today’s virtual machine migration solutions are not geared to moving datacenter resources efficiently. In a two-tier cloud architecture, the virtual machine migration may traverse many hops of carrier infrastructure, increasing total cost of ownership (TCO). In addition, complete virtual machine images must be transferred before the destination station can start the machines, extending the time it takes for the second tier to take control. The upshot is that optimized solutions need to be developed to enable seamless virtual machine migrations.

c.       Network and storage acceleration of resource-constrained tier-two clouds
Unlike the first cloud tier, the second cloud tier is bound to be resource-constrained, requiring significant data acceleration for both the networking and storage layers. A 16-core full SMP ARM®-based processor like the LSI Axxia 5500 processor, with its processor cores and, more importantly, its fully programmable acceleration engines for offloading security, deep packet inspection, traffic management and other functions is well-suited for network acceleration of the second cloud tier. Keep in mind that specific acceleration needs vary based on the location of the second tier cloud. For example, the acceleration requirements of the second cloud tier would differ depending on whether it is part of a service provider access aggregation router or located on a remote lamp post. The need for security acceleration, in particular, increases tremendously in cases where data associated with particular data events must be authenticated before further processing. To support these various acceleration needs, the second cloud tier can be built out of fairly homogeneous and scalable ARM-based hardware components with differing acceleration builds tuned to specific tasks running on it.

Momentum for greater connectivity builds
Momentum behind billions of connected things/machines across industrial and consumer applications to create a more connected, interactive world is building. Two-tier clouds and other innovative architectures are emerging at an accelerated pace to meet demand for this higher order of connectivity. And it is solutions like the LSI Axxia processor that promise to enable the scalable, flexible acceleration required for these emerging two-cloud architectures.

 

Tags: , , , , , , , , , , , , ,
Views: (1291)


With the much anticipated launch of 12gb/s SAS MegaRAID and 12Gb/s SAS expanders featuring DataBolt™ SAS bandwidth-aggregation technologies, LSI is taking the bold step of moving beyond traditional IO performance benchmarks like IOMeter to benchmarks that simulate real-world workloads.

In order to fully illustrate the actual benefit many IT administrators can realize using 12Gb/s SAS MegaRAID products on their new server platforms, LSI is demonstrating application benchmarks on top of actual enterprise applications at AIS.

For our end-to-end 12Gb/s SAS MegaRAID demonstration, we chose Benchmark Factory® for Databases running on a MySQL Database. Benchmark Factor, a database performance testing tool that allows you to conduct database workload replay, industry-standard benchmark testing and scalability testing, uses real database application workloads such as TPC-C, TPC-E and TPC-H. We chose the TPC-H benchmark, a decision-support benchmark, because of its large streaming query profile. TPC-H shows the performance of decision support systems – which examine large volumes of data to simplify the analysis of information for business decisions – making it an excellent benchmark to showcase 12Gb/s SAS MegaRAID technology and its ability to maximize the bandwidth of PCIe 3.0 platforms, compared to 6Gb/s SAS.

LSI MegaRAID SAS 9361-8i storage performance on display using Spotlight® on MySQL, which is monitoring the data traffic across Intel’s new R2216GZ4GC server based on the new Intel® Xeon® processor E5-2600 v2.

The demo uses the latest Intel R2216GZ4GC servers based on the new Intel® Xeon® processor E5-2600 v2 product family to illustrate how 12Gb/s SAS MegaRAID solutions are needed to take advantage of the bandwidth capabilities of PCIe® 3.0 bus technology.

When the benchmarks are run side-by-side on the two platforms, you can quickly see how much faster data transfer rates are executed, and how much more efficiently the Intel servers handles data traffic. We used Quest Software’s Spotlight® on MySQL tool to monitor and measure data traffic from end storage devices to the clients running the database queries. More importantly, the test also illustrates how many more user queries the 12Gb/s SAS-based system can handle before complete resource saturation – 60 percent more than 6Gb/s SAS in this demonstration.

What does this mean to IT administrators? Clearly, resourse utilization is much higher,  improving total cost of ownership (TCO) with their database server. Or, conversley, 12Gb/s SAS can reduce cost per IO since fewer drives can be used with the server to deliver the same performance as the previous 6Gb/s SAS generation of storage infrastructure.

Tags: , , , , , , , , , , , , ,
Views: (769)