SNIA has created a set of nomenclatures used to describe the lifecycle of an SSD. A new SSD is called FOB, for "fresh out of the box."
After an SSD's initial use, it settles into a stage that SNIA terms the "Steady State," which is when performance levels out and can be accurately measured. "In terms of performance, reads are fastest, writes [are] slower and erases are the slowest yet," Wassenberg said.
Handy and Tom Coughlin, founder of consultancy Coughlin Associates, teamed up with Calypso to compile a study on SSD performance that involved 18 different drives.
"We found that there was no performance consistency between any two SSDs. They vary all over the map," Handy said, adding that some single-level cell (SLC) SSDs perform worse than less expensive multi-level cell (MLC) SSDs.
"And some MLC-based SSDs are slower than an enterprise hard-disk drives once they have entered their steady state," Handy said.
Handy and Coughlin tested 18 drives using the PTS specification. No two were alike.
How long it takes an SSD to move into a steady state that's reliable for testing varies from product to product, but the new spec requires that an SSD go through five separate performance tests prior to it being benchmarked.
"The key thing with the PTS spec is it tells you what to do and how to prepare the drive. Is this the only way to test performance? No. But, over time, we found it to be very efficient and the most dependable way. You can run this test multiple times and get the same result," Wassenberg said.
The PTS Test Sequence is as follows:
- Purge: put SSD into a near FOB condition, by erasing data
- Workload Independent Precondition: Write prescribed data to entire SSD to facilitate reaching Steady State
- Workload Based Precondition: Run the Test loop itself until SSD is in Steady State
- Test: Take measurements when SSD is in Steady State
The PTS describes a reference test hardware and software platform used to validate the specification itself. The reference test platform was developed by SSSI member Calypso Systems.
Calypso built a hardware platform that has multiple bays to test drives in parallel and developed the software that adheres to the specification.
"You pretty much plug in a drive and it does the test," Wassenberg said. "If you want to test a drive, they'll test it for a fee. This reference test platform is the gold standard."
"You just need to ensure that you use a hardware platform that doesn't bottleneck the SSDs. We recommend a server motherboard," he said. "You must also be knowledgeable enough to write [a] script for it."
The SNIA is currently also working on application-specific specifications that will allow SSDs to be tested under loads for specific tasks. For example, SSDs could be tested for their performance in PC environments running Windows 7 or in server environments running Oracle software.
"But, that's a ways off. The important thing for us was to get something out there to test drives with and compare performance," Wassenberg said.
Lucas Mearian covers storage, disaster recovery and business continuity, financial services infrastructure and health care IT for Computerworld. Follow Lucas on Twitter at @lucasmearian or subscribe to Lucas's RSS feed. His email address is firstname.lastname@example.org.
Read more about storage hardware in Computerworld's Storage Hardware Topic Center.