My first blog in this series, “How to maximize performance of PCIe flash for enterprise applications running on Linux,” describes the steps for aligning PCIe® flash devices. This blog covers the next stage of setting up the PCIe flash device when using the Linux® operating system: creating a RAW device or a file system.

At this point, one or more PCIe flash cards have been partitioned on a sector boundary. Depending on their use, these partitioned devices are either set up as a single RAW device or as part of a logical volume or RAID array.

Next step is to determine how these devices will be used. Most administrators will create file systems on these partitions. Some Oracle administrators will use them as RAW devices and assign them to Automatic Storage Management (ASM). Still others, those looking for the best performance possible from the device will stick to a RAW device. For many years, the recommendation was not to use RAW devices because the complexity of managing them outweighed their small potential gains in performance.

ASM uses RAW devices but makes administration of these devices much easier. More on ASM in Part 3 of this series.

Building a file system
Next is to build a file system on the RAW device, LVM or RAID. But first we need to determine the best type of file system to use. There are many to choose from including:

  • EXT-2
  • EXT-3
  • EXT-4
  • XFS
  • BTRFS
  • ZFS

To keep this brief, I will only go over EXT-4. This type of file system is the most current and provides the latest enhancements for increasing capacity, disabling journaling and many other capabilities, though XFS can be a higher performance alternative.

To create an EXT-4 file system, use this command:

mkfs.ext4 /dev/sdX

You can now turn off or on certain features of the EXT-4 file system by using “tune2fs. Here are a couple of examples of using tune2fs:

  • To list all file system features for /dev/sdX1, use this tune2fs command:

tune2fs –l /dev/sdX1 | grep ‘Filesystem features’

  • To disable journaling on /dev/sdX1, use this tune2fs command:

tune2fs -O ^has_journal /dev/sdX1

Mounting the file system
The next step is to mount the file system and assign the owner:group to the mount point. There are also many tuning options that can be added to the mount command when using PCIe flash cards. The mount options I use are:

  • NOATIME
  • NODIRATIME
  • MAX_BATCH_TIME=0
  • NOBARRIER
  • DISCARD

The mount command for /dev/sda1 to /u01 would be:

mount –o noatime,nodiratime,max_batch_time=0, nobarrier,discard /dev/sda1 /u01

To make these mount points persistent over reboots, add them to the mount entries in ‘/etc/fstab’. Finally, you need to give a user rights for reading and writing to this mount point, and to assign ownership to /u01 – for example, assigning ownership of /u01 to the oracle userid and to the dba group. To do this, use the “chown” command:

chown oracle:dba /u01

The PCIe flash device is now ready to be used.

Part 3 of this series will describe how to use Oracle ASM when deploying PCIe flash cards.

Part 4 of this series will describe how to persist assignment to dynamically changing NWD/NMR devices.

Rick Stehno has 36 years of IT industry experience and for the past 24 years has worked with Oracle relational databases. He has been at LSI for six years, benchmarking Oracle databases for the company’s SAN storage and flash technology divisions.... Read more

Tags: , , , , , , , , , ,
Views: (861)

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>