|Introduction and Specifications|
It's rare we get genuinely excited around here about a product. To be honest, likely we're spoiled by all the bleeding-edge technology we have the great opportunity to look at week in and week out. Or perhaps it's just that we're jaded a bit and have heard one too many marketing pitches, but truth be told it takes a lot to get us fired up about a new product or technology. Sure we can appreciate and will occasionally hand out high praise for a major evolutionary advancement but we honestly can't remember the last time a product really officially "changed the game" for us, or so to speak.
The first time we looked at Fusion-io's ioDrive product, we offered the notion that it was a "disruptive technology", something that had the potential to set the industry on its ear. Of course the ioDrive is an enterprise-class product that showed the significant potential of PCI Express direct-attached SSD storage but its cost structure was such that the mainstream market couldn't possibly even begin to justify it, no matter what the upside performance looked like. And then of course we heard of Fusion-io's more consumer-targeted play, the ioXtreme, which debuted at E3 this past summer.
Obviously, E3 is a consumer electronics entertainment venue so it became abundantly clear that Fusion-io wasn't only productizing their technology for the enterprise space but for the enthusiast, workstation professional and power user as well. Today we've got a full deep-dive look at Fusion-io's ioXtreme PCI Express Solid State Drive. Weighing in at a pricey $899 for 80GB (standard card), it's definitely still a high ticket item but it's at least approachable now, for those of you that have the need for speed as they say. Just how much speed? And what about RAIDing a couple of these bad boys together? We aim to quantify that for you, as well as a couple of the product's early release caveats, in the pages ahead.
In terms of its high level specification the ioXtreme reads a lot like previous Fusion-io products with a half-height PCI Express X4 card design and a total 80GB capacity for the current product offering. Bandwidth-wise the ioXtreme claims a mind-boggling 700MB/sec of read throughput and a more modest 280MB/sec of write throughput. That said, in the pages ahead you'll notice that write number is a tad on the conservative side perhaps, depending on what you're doing with it. You can also configure multiple ioXtreme Pro cards together with what Fusion-io calls their Xlink technology that works over a PCI Express link. Three cards can offer an outrageous 2.1GB/sec of read throughput, as is shown in the diagram above.
The ioXtreme is also built on less expensive MLC NAND flash, unlike its big brother the ioDrive, which is an SLC-based product. Finally, as you can see, the ioXtreme supports multiple operating systems including Windows 7. Unfortunately, a bit of a let-down for some might be, that the product still currently can't be utilized as a boot volume. Fusion-io assures us that this feature will be supported in future driver and/or firmware revisions but also didn't commit to a schedule for that roll-out just yet.
|Under The Hood Of The ioXtreme|
Fusion-io sent us two cards for testing, an ioXtreme and their ioXtreme Pro. We're told the the Pro variant is identical to the standard card, in terms of performance, except for Fusion-io's "Xlink" technology which allows two cards to be ganged together in a RAID setup over PCI Express.
The cards themselves are a case study in simplicity and design elegance. The current ioXtreme design packs 80GB of MLC NAND flash in a half-height PCI Express X4 footprint and requires no external power beyond the PCIe slot power provided to the card.
The ioXtremes are also passively cooled with a rather small heasink mounted to the top of the board's controller ASIC. The ioXtreme's flash controller ASIC (Application Specific IC) is also supported by various other chips on the board, like Samsung DRAM cache and an Intel configuration Flash chip that is responsible for setting up the controller chip, since it is in fact a programmable device.
In reality, the proprietary controller ASICs on all ioXtreme boards currently are FPGAs (Field Programmable Gate Array) from Xilinx. The Xilinx Vertex 5 chip that Fusion-io chose to build their controller out of is a rather large 1136 ball grid array device that comes with 110,000 logic cells for programming as well as several blocks of configurable memory on board and multiple other serial IO blocks like PCI Express. These Xilinx FPGAs likely are few hundred dollars each and are easily the most costly component on the ioXtreme's bill of materials. In the future, Fusion-io can cost reduce the board significantly, once their controller design is stable, by going to a full custom, hard-wired ASIC, versus the programmable devices they're currently using now.
The controller design of the ioXtreme implements a 25 parallel channel (X4 bank) memory architecture, with one channel dedicated to error detection and correction, as well as self-healing (data recovery and remapping) capabilities for the flash memory at the cell level. By way of comparison, Intel's X25-M SSD implements a 10 channel design.
|The Setup, Methodology and SANDRA|
|Our Test Methodologies: Under each test condition, the Solid State Drives tested here were installed as secondary volumes in our testbed, with a standard spinning hard disk for the OS and benchmark installations. All SSDs were left blank without partitions wherever possible, unless a test required them to be partitioned and formatted, as was the case with our ATTO benchmark tests and a few of our Iometer runs. Windows firewall, automatic updates and screen savers were all disabled before testing. In all test runs, we rebooted the system and waited several minutes for drive activity to settle before invoking a test.
Also, you'll note that we performed all of our individual and RAID SSD testing on an Intel X58 chipset-based motherboard via its ICH10R Southbridge SATA controller. The OCZ Z-Drive and ioXtreme PCI Express cards were tested in PCIe X16 slots on the same test bed. For a few of our file transfer tests specifically, we utilized Fusion-io's ioDrive as a source drive to read files from, for our write performance testing, or write files to, for our read performance testing of all products we show in the following performance comparisons.
In our SiSoft SANDRA testing, we used the Physical Disk test suite. We ran the tests without formatting the drives and both read and write performance metrics are detailed below. Please forgive the use of these screen captures and thumbnails, which will require a few more clicks on your part. However, we felt it was important to show you the graph lines in each of the SANDRA test runs, so you are able to see how the drives perform over time and memory location and not just an average rated result.
We've posted 2 x Intel X25-M (Gen 1 80GB) RAID 0 160GB scores for a quick comparison to the ioXtreme's performance. This RAID array offered in excess of 500MB/sec read bandwidth and 160MB/sec+ of write bandwidth. The ioXtreme shows a flat, very robust performance graph here with read throughput in excess of 800MB/sec. On the write side of the equation, the ioXtreme shoes some oscillation but not much, at 286MB/sec or so.
The ioXtreme Pro offers, as expected, nearly identical performance to the standard ioXtreme product. You'll recall earlier that we noted both ioXtreme variants offer identical performance, with the only difference between the two cards being that Pro cards are able to pair with another ioXtreme Pro or standard card, for RAID array configurations like RAID 0 for even higher performance. Unfortunately, SANDRA's test suite did not work properly with a RAID 0 setup with the ioXtremes, since they need to be configured with Windows software RAID and SANDRA saw the disks as two separate drives rather than a single volume. We were able to run other benchmarks with the ioXtremes in RAID 0 for a few other tests however and we'll show you some of the results next.
ATTO is a more straight-forward type of disk benchmark that measures transfer speeds across a specific volume length. It measures raw transfer rates for both reads and writes and graphs them out in an easily interpreted chart. We chose .5kb through 8192kb transfer sizes over a total max volume length of 256MB. This test was performed on blank, formatted drives with default NTFS partitions in Windows Vista 64-bit with SP2. We've also provided test results from an Intel single and RAID 0 SSD setup as well as OCZ's Z-Drive, for reference.
The ioXtremes show solid small file transfer performance, all the way down to even 2KB and the cards are as fast as the Intel RAID 0 setup at this level. With larger transfer sizes, the ioXtreme drives show clearly the fastest read performance from any SSD solution on the market today, with respectable write performance, at 300MB/sec or so. Though the OCZ Z-Drive also began to stretch its legs with writes in this test showing 2X the performance of the ioXtreme in some cases.
What, no RAID benchmarks again?
That's correct, unfortunately ATTO is unable to accurately measure performance in this test, above the 999MB/sec mark. With some help from Fusion-io and Windows Perfmon utility, we determined that ATTO was actually wrapping performance readings back around, for transfer rates hitting 1GB/sec and higher. In fact the ioXtreme RAID 0 pair chalked up what looked like 300MB/sec in some of the larger file transfers of the test, but in reality was pushing 1.3GB/sec (or 1300MB/sec) though the bar graphs and numbers were only showing the 300MB/sec or so, for read throughput. For write performance, we did observe in excess of 600MB/sec of available throughput in RAID 0 mode with a pair of ioXtreme drives.
CrystalDiskMark is a new synthetic test we've started looking at that tests both sequential as well as random small and large file transfers. It does a nice job of providing a quick look at best and worst case scenarios with SSD performance, best case being large sequential transfers and worse case being small, random 4K transfers.
Once again we've provided some tests on various Intel X25-M SSD configurations, OCZ's Z-Drive, and a Fusion-io ioDrive just for reference as a baseline.
Once again, we observed that both the ioXtreme cards perform virtually identical, offering in excess of 750MB/sec read throughput and 300MB/sec of write throughput for sequential transfers and random transfers at 512K. The 4K random transfer test of course takes all the SSDs we tested to their knees. Fusion-io's hugely expensive ioDrive enterprise solution offers the best single drive performance of the bunch however, under all conditions. Finally, this time we were able to run the ioXtremes cards in RAID 0 and the numbers speak for themselves - a peak of 1348MB/sec read and 586MB/sec write bandwidth is certainly something to covet, at least from a geek's perspective.
|HD Tach Testing|
Simpli Software's HD Tach is described on the company's web site as such:
The first quick notable we'd like to make here is that, yes, again we can't provide you with two-drive ioXtreme RAID 0 numbers in this test because HDTach doesn't support the configuration on a formatted, Windows software RAID striped volume. Though we could run the test, HDTach would still see two physical drives at the hardware level and bypass the software RAID stripe. With that out of the way, the single drive performance of the ioXtreme is moderately impressive here, besting an Intel X25-M G1 RAID 0 array with two SSDs. The OCZ Z-Drive never seemed to fair very well in HDTach however.
|PCMark Vantage HDD Test Module|
|Next we ran the Intel RAID 0 arrays, our 256GB OCZ Z-Drive and the Fusion-io ioXtremes through a battery of tests in PCMark Vantage from Futuremark Corp. We specifically used only the HDD Test module of this benchmark suite to evaluate all of the drives we tested. Feel free to consult Futuremark's white paper on PCMark Vantage for an understanding of what each test component entails and how it calculates its measurements. For specific information on how the HDD Test module arrives at its performance measurements, we'd encourage you to read pages 35 and 36 of the white paper.
We like PCMark Vantage's HDD Performance for its real-world application measurement approach to testing. From simple Windows Vista start-up performance to data streaming from a disk drive in a game engine and video editing with Windows Movie Maker, we feel that these tests illustrate more of a real-world performance profile of an SSDs in an end user/consumer PC or Workstation usage model.
Once again the ioXtreme single card setup shows its prowess and even excels past the likes of a 4-drive Intel SSD RAID array, save for the Photo Gallery test. Interestingly, our ioXtreme/ioXtreme Pro Windows software RAID 0 setup offered less performance versus a single ioXtreme drive in these tests. As you'll note in our upcoming Iometer tests, the ioXtreme drives due consume a fair bit of CPU resources, so perhaps this is why we're seeing lower scores here. We've reached out to Fusion-io with respect to this anomaly and will offer an update with insight here, should more details become available.
|PCMark Vantage HDD (cont.)|
|Our next series of Vantage tests will stress the current weakness of most NAND Flash, that being random write performance. Applications like video editing, streaming and recording are not what we would call a strong suit for the average SSD, due to their high mix of random write transactions.
Once again our ioXtreme RAID 0 volume tested slower than a single ioXtreme configuration though it was much less pronounced versus the more read-intensive functions of the previous set of Vantage HDD tests. In addition, there are some wild swings with respect to the Windows Media Center, as you can see in the Intel RAID numbers. Regardless, the ioXtreme puts up impressive performance in a single card setup and really shines in the Application Loading test but doesn't quite keep pace with an Intel-based RAID array, in some of the tests like Movie Maker or Media Player.
|Iometer Workstation Results|
|The IOMeter Question:
As we noted in a previous SSD round-up article, though IOMeter is clearly thought of as a well respected industry standard drive benchmark, we're not completely comfortable with it for testing SSDs. The fact of the matter is, though our actual results with IOMeter appear to scale properly, it is debatable whether or not certain access patterns, as they are presented to and measured on an SSD, actually provide a valid example of real world performance for the average end user. That said, we do think Iometer is a solid gauge for relative available bandwidth with a given storage solution.
We've also considered two schools of thought with respect to proper Iometer testing from an unpartitioned, clean drive test setup and also a full, formatted volume that is obviously more representative of how an end user would setup the product. However, the latter case also fills the drive up with a large test file and then reads and writes to that file, which is less than a real-world usage model as well.
And so, as a result of all the back-and-forth discussion we've had around the lab lately with respect to this benchmark, we've decided to show you all the angles Here's a sampling of our test runs with Iometer version 2006.07.27 with the ioXtreme and the various competitive solutions we tested both in unpartitioned/formatted states and also with a full NTFS partition.
* Note: Clean, unformatted drive performance is represented in these tests
The above test condition with Iometer is representative of our "Workstation" test pattern performed on blank SSDs with no partition. Thinks of this test as a "best case" scenario in terms of performance for the test pattern since higher bandwidth is available on a completely clean SSD with a fresh low-level format, though again this is hardly how an end user would run the product in a real usage model. We also can't test the ioXtreme in RAID 0 here since the product relies on Windows software RAID with a formatted NTFS partition. Regardless, you can see the ioXtreme is posting available throughput that is some 4X faster than its closest competitor, the OCZ Z-Drive and about 5X the speed of a single Intel Gen 2 SSD.
Next we have the same test, only this time with a formatted NTFS partition which means we can also offer RAID performance numbers with the ioXtreme Pro and ioXtreme in RAID 0. You'll have to forgive us that we don't have RAID arrays with Intel's SSDs in these tests due to time constraints but you can still get a sense of relative performance.
Partitioned and Formatted NTFS Volumes Tested
Here a single ioXtreme, in the same test but on a formatted NTFS volume, offers about 28K IOPS, which is significantly faster than any other single drive solution in the test, as you can see. However, versus the peak of 41K IOPS we saw on the unformatted, clean ioXtreme, this is a decidedly different picture, though just as impressive perhaps. As a frame of reference, the Intel SSD dropped off about 1/2 of its bandwidth, though the OCZ Z-Drive managed to maintain much of its performance and even scaled up as we added outstanding IO requests. What's interesting is how much of a hit the ioXtreme took, though still dominant performance-wise. With the subsequent performance runs over time, perhaps the garbage collection algorithm of the ioXtreme, needs more time to recover versus the OCZ product, which is made up of Vertex series SSDs (and the new garbage collection algorithm that OCZ worked on with Indilinx) along with an LSI RAID controller.
|Iometer Workstation and Database Results (cont.)|
The next series of Iometer benchmarks we're going to offer you are single-point, 8 thread tests that show the various SSDs we tested in a simple bar graph spread. These tests were all performed on formatted NTFS partitions on each drive.
Looking at the smaller file workloads we've organize here, the ioXtreme drives, offer over 2X the performance of the OCZ Z-Drive and 5X the performance of a single Intel SSD. Also, the RAID 0 setup with the ioXtreme Pro scaled very nicely here, offering roughly double the performance of a single ioXtreme drive.
Here the spread is decidedly tighter with larger file transfers at 64K and it's obvious that sequential, larger file performance is easier on the average SSD in general, as is apparent in the Intel SSD number. Also, the RAID 0 volume with the pair of ioXtremes offers almost 2X the performance of a single ioXtreme drive. So, in short, where the ioXtreme really excels is with smaller, more random access patterns, which you could make a case for being more like an average desktop usage model. However, we'll just let the numbers speak here and you can determine for yourself as to your own personal usage model.
|File Transfer Tests|
Our final series of tests are what you might call more "crude measurements" in that we simply fired up our trusty stop-watch and measured the time it took to complete a copy and paste command of a single large file or a bunch of large files from one storage volume in our test system to another.
** Please note that we utilized a Fusion-io ioDrive card as our source drive in some of the following tests, to read files from or copy files to, for our read and write measurements of all reference products and the ioXtreme cards. This affords us the luxury of much higher available bandwidth from the source or target drive, such that it would not be the limiting factor in a given test condition. We've also included some measurements with a standard WD Raptor 150GB hard drive as our source or target drive, to offer a more practical usage model and benchmark measurement.
Here the ioXtreme drives offer a sizable performance gain over a standard SSD but actually trade victories with OCZ's Z-Drive. The ioXtremes offering slightly faster read performance, while the Z-Drive took the write file transfer test by a few seconds. Interestingly, in RAID 0, though write performance was significantly improved, read performance fell back slightly with the ioXtreme drives in this test. Next, we performed the same test with the ioXtreme drives and a WD Raptor as the source or target drive for read/write transfers.
Here the field was quickly equalized, with all product tested being held back by the slower spinning hard drive, when either reading from or writing to it. However, the Z-Drive managed to edge out the ioXtreme product in this test, with the exception of the ioXtreme's RAID 0 performance which offered an advantage of about 10% or so.
Though still a little on the large side, we think our custom multi-file transfer test is a bit more representative of a day to day workload that and end user may put upon the product. Here we've transferred the same 3.5GB of data to and from the drives we tested, only this time it was comprised of a bunch of 100MB+ sized files (video files to be specific). In this test, the ioXtreme drives still can't catch the Z-Drive with writes, unless they are striped together in RAID 0. For read performance, the ioXtreme single card setups beat all competitors with a couple of seconds to spare versus the Z-Drive.
|Game Level Load Tests|
Our final test case couldn't get any more straightforward. Here we'll show you how the ioXtreme performs under a real-world application loading condition like a game level load-up. Here we simply installed Left 4 Dead on each of the drives we've tested and then proceeded to launch single player games on either the No Mercy or Dead Air levels. Also, as you'll note, we've included the Fusion-io card in this test, for a relative measure of performance.
If you recall from our PCMark Vantage testing, this is an area that the ioXtreme drives excel - application loading. The ioXtreme Pro RAID combination didn't offer much in the way of a performance boost but the single drive performance was more than twice as fast as the Z-Drive and the Intel SSD in one test. We tossed in Fusion-io's ioDrive for good measure here as well and it too offered similar performance. Application level loading like this is comprised of lot of fast smaller file transfers and this is where the ioXtreme and ioDrives really shine versus a standard SSD.
|Our Summary and Conclusion|
The ioXtreme drives performed a lot like their higher-end brethren, the ioDrive, from Fusion-io. These diminutive, half-height cards offered peak bandwidth that was often times three to five times faster than a standard SSD and in some cases, as with our Iometer testing, even versus higher-end solutions like OCZ's Z-Drive. The ioXtremes excel in random, fast read operations like application loading, where the bulk of the end user experience in desktop environments lies. Where it lagged behind higher-end solutions like the Z-Drive occasionally, was with write operations where the product's peak throughput tops out at 300MB/sec.
In reality, the new ioXtreme from Fusion-io is a high-end niche' product that caters to the performance enthusiast and high-end workstation user looking to maximize system performance for one of the age-old bottlenecks that will continue to be the single-most limiting factor in system performance in the foreseeable future. Storage subsystems have traditionally been orders of magnitude slower than other system components with nanosecond access times and several GB/sec of bandwidth. SSD technology has come a long way to closing that performance gap but even existing bridging requirements, from Serial ATA storage interfaces to PCI Express native system interfaces, still hamper performance. Fusion-io's direct-attach PCI Express Flash SSD technology is quite literally a ground-breaking advancement. Though its base cost structure is still very high, even compared to the current $2.75 - $3.50 per GB cost range of standard SATA SSDs, Fusion-io's ioXtreme version of their product makes solid strides towards driving their technology down from the enterprise datacenter to the end user market.
As we noted early on in this product showcase, it takes a lot to get us excited in the lab these days. From its design elegance to its blisteringly fast performance, even for the most jaded tech journalist among us, it's hard not to be thoroughly impressed with Fusion-io's technology. Whether or not you can personally justify the cost of the product for your own use, is another question all together. One thing is for sure however, you'll really want to be clear on your intended usage model for this device. We've noted before that Fusion-io likens their product to a "new memory tier" in the desktop or workstation architecture. These devices are not bulk storage products obviously, though loading them up with your most-used demanding applications for things like content creation, 3D animation, CAD and gaming, will bring you relatively huge dividends in performance. Even a few business class applications like a overly huge Outlook PST file for example, will show significant benefit from the ultra-high bandwidth and low latency of the ioXtreme.
That's not to say the product isn't without its caveats. For one thing, the value proposition of the ioXtreme Pro is a little fuzzy in that unfortunately you have to rely on a Windows software RAID to stripe or mirroring cards together, not to mention it costs a stiff $600 more. A full-fledged, configurable RAID BIOS would do wonders for this product, affording a RAID configuration pre-boot, and while we're at it, let's get these cards bootable! Again, Fusion-io has noted that boot functionality is coming though they were non-committal with respect to when exactly. What really excites us, is the opportunity Fusion-io has to cost reduce these cards in the future. Those Xilinx FPGAs cost big bucks and they're not really a full-production vehicle for a product that could sell in the kind of volumes these cards should be selling in. Drop in a custom ASIC at a fraction of the price and now the the ioXtreme starts tracking a more traditional SATA SSD cost curve. It would make sense this is on the roadmap for Fusion-io but we've gotten not so much as a wink or a nod in this direction. Still, it seems only natural, right Woz?
The other thing that hampers some of the joy here is retail availability. To date, the only place we've been able to find the product is over at Amazon. All that said, we tip our collective hats to the team at Fusion-io for impressing us with something new and truly innovative. The ioXtreme paves the way for direct-attach PCI Express storage in the mainstream. Let's hope the Fusion-io team can continue to offer products so wildly appealing to our inner-geek. We have to admit, the ioXtreme is just plain <HOT>.