Imagine a bathtub full of water and asking someone to empty the tub while you turn your back for a moment. When you look again and see the water is gone, do you just assume someone pulled the drain plug?
I think most people would, but what about the other methods of removing the water like with a siphon, or using buckets to bail out the water? In a typical bathroom you are not likely to see these other methods used, but that does not mean they do not exist. The point is that just because you see a certain result does not necessarily mean the obvious solution was used.
I see a lot of confusion in forum posts from SandForce Driven™ SSD users and reviewers over how the LSI® DuraWrite™ data reduction and advanced garbage collection technology relates to the SATA TRIM command. In my earlier blog on TRIM I covered this topic in great detail, but in simple terms the operating system uses the TRIM command to inform an SSD what information is outdated and invalid. Without the TRIM command the SSD assumes all of the user capacity is valid data. I explained in my blog Gassing up your SSD that creating more free space through over provisioning or using less of the total capacity enables the SSD to operate more efficiently by reducing the write amplification, which leads to increased performance and flash memory endurance. So without TRIM the SSD operates at its lowest level of efficiency for a particular level of over provisioning.
Will you drown in invalid data without TRIM?
TRIM is a way to increase the free space on an SSD – what we call “dynamic over provisioning” – and DuraWrite technology is another method to increase the free space. Since DuraWrite technology is dependent upon the entropy (randomness) of the data, some users will get more free space than others depending on what data they store. Since the technology works on the basis of the aggregate of all data stored, boot SSDs with operating systems can still achieve some level of dynamic over provisioning even when all other files are at the highest entropy, e.g., encrypted or compressed files.
With an older operating system or in an environment that does not support TRIM (most RAID configurations), DuraWrite technology can provide enough free space to offer the same benefits as having TRIM fully operational. In cases where both TRIM and DuraWrite technology are operating, the combined result may not be as noticeable as when they’re working independently since there are diminishing returns when the free space grows to greater than half of the SSD storage capacity.
So the next time you fill your bathtub, think about all the ways you can get the water out of the tub without using the drain. That will help you remember that both TRIM and DuraWrite technology can improve SSD performance using different approaches to the same problem. If that analogy does not work for you, consider the different ways to produce a furless feline, and think about what opening graphic image I might have used for a more jolting effect. Although in that case you might not have seen this blog since that image would likely have gotten us banned from Google® “safe for work” searches.
I presented on this topic in detail at the Flash Memory Summit in 2011. You can read it in full here: http://www.lsi.com/downloads/Public/Flash%20Storage%20Processors/LSI_PRS_FMS2011_T1A_Smith.pdf
Tags: bathtub drain, controller, data reduction technology, DuraWrite, flash, flash controller, flash memory, Flash Memory Summit, NAND, over-provisioning, SandForce, SandForce Driven SSD, SATA, Serial ATA, solid state drive, TRIM
It may sound crazy, but hard disk drives (HDDs) do not have a delete command. Now we all know HDDs have a fixed capacity, so over time the older data must somehow get removed, right? Actually it is not removed, but overwritten. The operating system (OS) uses a reference table to track the locations (addresses) of all data on the HDD. This table tells the OS which spots on the HDD are used and which are free. When the OS or a user deletes a file from the system, the OS simply marks the corresponding spot in the table as free, making it available to store new data.
The HDD is told nothing about this change, and it does not need to know since it would not do anything with that information. When the OS is ready to store new data in that location, it just sends the data to the HDD and tells it to write to that spot, directly overwriting the prior data. It is simple and efficient, and no delete command is required.
However, with the advent of NAND flash-based solid state drives (SSDs) a new problem emerged. In my blog, Gassing up your SSD, I explain how NAND flash memory pages cannot be directly overwritten with new data, but must first be erased at the block level through a process called garbage collection (GC). I further describe how the SSD uses non-user space in the flash memory (over provisioning or OP) to improve performance and longevity of the SSD. In addition, any user space not consumed by the user becomes what we call dynamic over provisioning – dynamic because it changes as the amount of stored data changes.
When less data is stored by the user, the amount of dynamic OP increases, further improving performance and endurance. The problem I alluded to earlier is caused by the lack of a delete command. Without a delete command, every SSD will eventually fill up with data, both valid and invalid, eliminating any dynamic OP. The result would be the lowest possible performance at that factory OP level. So unlike HDDs, SSDs need to know what data is invalid in order to provide optimum performance and endurance.
Keeping your SSD TRIM
A number of years ago, the storage industry got together and developed a solution between the OS and the SSD by creating a new SATA command called TRIM. It is not a command that forces the SSD to immediately erase data like some people believe. Actually the TRIM command can be thought of as a message from the OS about what previously used addresses on the SSD are no longer holding valid data. The SSD takes those addresses and updates its own internal map of its flash memory to mark those locations as invalid. With this information, the SSD no longer moves that invalid data during the GC process, eliminating wasted time rewriting invalid data to new flash pages. It also reduces the number of write cycles on the flash, increasing the SSD’s endurance. Another benefit of the TRIM command is that more space is available for dynamic OP.
Today, most current operating systems and SSDs support TRIM, and all SandForce Driven™ member SSDs have always supported TRIM. Note that most RAID environments do not support TRIM, although some RAID 0 configurations have claimed to support it. I have presented on this topic in detail previously. You can view the presentation in full here. In my next blog I will explain how there may be an alternate solution using SandForce Driven member SSDs.
Most people fully understand that electronics are useless without power, but what happens when devices lose power in the middle of operating? That answer is highly dependent on a number of variables including what type of electronic device is in question.
For solid state drives (SSDs) the answer depends on factors such as whether an uninterruptable power supply (UPS) is connected, what controller or flash processor is used, the design of the power circuit of the SSD, and the type of memory. If an SSD is in the middle of a write operation to the flash memory and power to the SSD is disconnected, many bad things could happen if the right safety measures are not in place. Many users do not think about all the non-user initiated operations the SSDs may be performing like background garbage collection that could be under way when the power fails. Without the correct protection, in most cases data will be corrupted.
According to the Nielsen company, 108.4 million viewers were tuned into the 2013 Super Bowl in New Orleans only to be shocked to witness the power go down for 34 minutes in the middle of the game. If power can be lost during such an incredibly high profile event such as this, it can happen just about anywhere.
Inside the New Orleans Superdome stadium operations and broadcast server rooms
Enterprise computing environments typically have multiple servers with data connections and lots of storage. Over the past few years, a larger percentage of the storage is kept on SSDs for the very active or “hot” data. This greatly improves data access time and reduces overall latency of the system. Enterprise servers are often connected to UPS systems that supply the connected devices with temporary power during a power failure.
Usually this is enough power to support uninterrupted system operations until power is restored, or at least until technicians can complete their current work and shut down safely. However there are many times when UPS systems are not deployed or fail to operate properly themselves. In those cases, the server will experience a power failure as abrupt as if someone had yanked the power cord from the wall socket.
The LSI® SandForce® flash controllers are at the heart of many popular SSDs sold today. The flash controller connects the host computer with the flash memory to store user data in fast non-volatile memory. The SandForce flash controllers are specifically engineered to operate in different environments, and the SF-2500/2600 FSPs are designed to provide the high level of data integrity required for enterprise applications. In the area of power failure protection, they include a combination of firmware (FW) and hardware circuitry that monitors the power coming into the SSD. In the event of a power failure or even a brown-out, the flash controller is alerted to the situation and hold-up capacitors in the SSD provide the necessary power and time for the controller to complete pending writes to the flash memory. This same circuit is also designed to prevent the risk of lower page corruption with Multi-level Cell (MLC) memory.
Watch out for SSD solutions that provide backup capacitors, but lack the necessary support circuitry and special firmware to ensure the data is fully committed to the flash memory before the power runs out. Even if these other special circuits are present, only truly enterprise SSDs that are meticulously designed and tested to withstand power failures are up to the task of storing and protecting highly critical data.
In the control room and down on the field
The usage patterns of non-enterprise systems like notebooks and ultrabooks call for a different power failure support mechanism. Realize that when you have a notebook or ultrabook system, you have a built in mini-UPS system. A power outage from the wall socket has no impact to the system until the battery gets low. At that point the operating system will tell the computer to shut down and that will be ample warning for the SSD to safely shut down and ensure the integrity of the data. But what if the operating system locks up and does not warn the SSD or the system is an A/C-powered desktop without a battery?
The LSI SF-2100/2200 FSPs are purpose-built for these client environments and operate with the assumption that power could disappear at any point in time. They use special FW techniques so that even without a battery present, as is the case with desktop systems, they greatly limit the potential for data loss.
The naked facts
It should be clear that the answer to the original question is highly dependent upon the flash controller at the heart of the SSD. Without having the critical features discussed above and designed into the LSI SandForce flash controllers, it is very possible to lose data during a power failure. The LSI SandForce flash controllers are engineered to withstand power failures like the one that hit New Orleans at the Super Bowl, but don’t expect them to fix wardrobe malfunctions.
Merging the working cultures of two different companies can be a very complex task. In my past experience with these situations I have not typically found the result to be highly positive for the employees of the incoming company.
As one example, the acquiring company may tell the employees they will be able to keep their startup environment and mentality, but within one or two quarters nearly all of those attributes are eliminated and the employees just become another cog in the bigger machine. After that the drive and creativity that was fostered with that startup mentality often disappears.
Year two and still happily married
In the first week of 2012, LSI completed the acquisition of SandForce, further expanding its growing coverage of flash memory technology IP. SandForce had grown significantly since its emergence from stealth mode in 2009 to become a leading provider of flash controllers for enterprise, cloud and client solid state storage solutions.
The SandForce team was kept in whole and created the Flash Components Division of LSI. The team even continues to reside in the same building it had occupied the prior year, further supporting the internal feeling of their original startup mentality. Most of the original culture of SandForce was kept intact and that enabled the transition into the larger LSI Corporation much easier for most people. Because those changes were spread out over a longer period of time it was easier for the team to digest with minimal disruption to their daily flow.
For the business side of the acquisition, there has been significant upside as a result of the merger. Over the last 14 months, both companies have invested significant time and resources to leverage the SandForce® flash controller technology across the company. LSI had already designed a high-end enterprise solution using the SandForce technology in their PCIe based Nytro™ product line. With both companies now under one umbrella, the engineering teams are free to develop advanced capabilities between the products to enable deeper integration that will result in greater customer benefits.
Greater efficiency … for the sake of customers
As a single larger company, LSI is now able to redistribute engineering and support resources as needed to better align with the quick expansion of flash memory storage solutions for its customers. It is also much easier to ensure a higher level of interoperability between related products and solutions. Many of the customers already purchasing LSI products can use the same sales and support teams already in place to access and incorporate a larger set of solutions from LSI.
Enterprise storage manufacturers have millions if not billions of dollars of revenue and their reputation at stake when they select new and emerging technologies like flash memory to provide storage for their customers. There is always a level of concern when these companies work with smaller startup organizations. The high-technology industry is full of company names that had a great technology, but for one reason or another they could not sustain the business and fell into oblivion. The acquisition of SandForce by LSI adds the support and confidence of a multi-billion dollar company to help assuage any possible concerns of those enterprise manufacturers.
Personally, as one of the early SandForce employees, I have found the acquisition to be very beneficial to everyone involved including employees, customers, and end users. I look forward to further advancements we will make to the flash industry as we continue to drive flash memory into the storage industry.