what's so very different about the design of Fusion-io's PCIe SSDs? - and rackmounts too! - analysis by StorageSearch.com
by Zsolt Kerekes, editor - December 2011 - a classic article
If you ask most people who know about the SSD market - which company do you think of when you think about the concept of PCIe SSDs? - the answer is Fusion-io - the company which evangelized the concept of using MLC flash in enterprise PCIe SSD accelerators - in new ways which break away from decades of performance sapping legacy hard disk software.
Fusion-io ignited the interest of the world's leading server makers too - transforming accelerator SSDs from a hard to find after-thought into a "must have" factory shipped option.
Fusion-io has lots of different models which span the range from products which accelerate graphics effects workstations, scalable value based accelerators which accelerate some of the world's largest online websites and high-end products for supercomputer modeling and analysis apps.
In this this article - which was prompted by many questions from readers - I try to answer the question - what's so very different about the design of Fusion-io's ioDrives / PCIe SSDs?
Let's begin with taking a step back and asking - Why would you want to install an SSD accelerator?
The obvious answer is - to make your storage and applications run faster.
But if you have a preference - which should it be?
- Speeding up the storage? - or
- Speeding up the app?
Fusion-io's SSDs are unique in today's enterprise flash accelerator market in that they don't include a microprocessor on-board the SSD card to implement the main functions of the SSD controller. Doing the flash management isn't as big a load as you might suppose for the ioDrive host - because it's still less than the traditional packeting and handshaking workload of talking to legacy storage interfaces. In other respects FIO's SSDs fit into in my classified groupings of SSD architecture types as follows:-
- ioDrives are - skinny (RAM cache flash)
- ioDrives are - big architecture.
- ioDrives are - efficient SSD designs
- ioDrives have - good roadmap symmetry
This was a bold move by the fledgling company when it started. Throughout all past SSD history - the designers of SSDs had gone to great lengths to hide their memory handling intellectual property inside chips like FPGAs and ASICs. One business result of this was that in the early days Fusion-io had to be careful about picking its customers - so they made sure they chose big name reputable companies - whether they were end users (like Facebook) or server oems (like HP etc) - who would not reverse engineer the algorithms - or if they did - would be worth suing. Of course nowadays Fusion-io is a big enough company itself that it has volume based cost and brand advantages - so even if another company were to copy some of their SSD technology - why would anyone buy it. But when Fusion-io's founders started down this track it was a very risky move - so why do it?
The consequences of this architectural decison are:-
- less hardware in the SSD card - so much better intrinsic MTBF and potentially the lowest cost to build product
- easier to control end to end data integrity
- faster access to flash data by wrap around software APIs
- greater stickiness with customers - once customers start seeing the benefits from leveraging the features - competing products with bland raw speed don't look so attractive
- speed of the SSD family is predictably scalable over future product
generations.
That's important for customers who don't want the risk of being locked into a design which looks fast this year - but might not be able to keep up with future CPU advances. If you look at the philosophy of the Fusion-io designs - one view you can take is that the SSD makes the host CPU work faster - because it speeds up access to the data. Another view - equally valid - is that because the host CPU is doing work for the flash memory - the faster the user's CPU - the faster the SSD.
A practical consequence for server oems is this. If they have a bunch of ioDrives in stock and they put a faster CPU in their new server - some of that speedup also makes the ioDrives work a bit faster too - in a way which is different to products which use on-board microprocessors. And unlike SSD companies which use microprocessors - as long as host CPUs get faster and flash gets faster - then it's easy to predict that future ioDrives can be faster too (this is what I call roadmap symmetry).
But if you look an application speed - and if the app uses Fusion-io's APIs - which they call VSL - you see a different story.
Suppose your software does a lot of twiddly bits very fast and very often. No matter how fast or clever the offload processor is on another type of PCIe SSD card - there is still a level of inefficiency that comes in - because your software is talking to other software which has to go and look at the flash somewhere else. But in the ioDrive - the APIs are inside the flash controller loop. You can access (cleaned up and endurance managed) data almost at the same latency as if you had designed the SSD yourself for this particular app.
That's why with the ioMemory architecture the application can speed up at a faster rate than you might think by simply looking at comparative raw storage benchmarks. But you only have this level of control and software gain factor in a new installation - or a new design of storage appliance. That's something I discussed in my article Legacy vs New Dynasty a new way of looking at enterprise SSDs. You can probably guess - that Fusion-io is New Dynasty - and that's a description they liked when they read my article. (And their competitors on the Legacy side of the fence liked their own appellation too.)
If you're already in a distressed environment with servers overloaded and legacy apps which are running slow - adding an SSD accelerator may help - but the advantages and disadvantages of the ioMemory compared to alternative products are difficult or impossible to predict. I always say in that situation - it's best to try before you buy. Cautious minded users may even think about looking at much higher-cost add-on rackmount SSDs in those distressed situations - to avoid the risk of doing bad things to their already suffering old servers.
Another thing to think about in the legacy scenario is that - simply adding a fast SSD to a server running heavy duty apps isn't guaranteed to speed up your app.
That's something I learned in 1990 - when I had a customer with an Oracle app I wanted to speed up with an SSD. (The data set wasn't I/O bound - it was already small enough to be sitting entirely in main memory.)
More recently an architect from a leading software company told me he was surprised to discover that their core software didn't speed up as much as he expected when they evaluated the impact of ioDrive PCIe SSDs. When he analyzed the cause - it lay in their own source code and data architecture. Their software had been written for hard disk arrays and had inbuilt assumptions which meant they weren't running enough parallel threads to take advantage of the greater bandwidth and lower latency of SSDs. They would have got the same result with any other brand of PCIe SSD - but it made him realize that there was a lot more to being ready for the solid state storage future. Of course when they rewrote that software it ran faster.
Getting to success in the SSD market is complicated. I see my job here at the mouse site as trying to understanding all this complex stuff to help the SSD industry get better, faster. I hope you've found this helped you too. It's just one of thousands of SSD related peices I've written in the past 20 years. You can see the top 100 most popular SSD articles here.
editor's footnotes
I've spoken to the founders of Fusion-io many times in the past 4 years and also to systems integrators and ISVs who have used these products. I also talk to all their main SSD competitors too to learn how competing products and businesses work. My notes above are designed to help you quickly understand why this product from Fusion-io is different - and whether those are differences which matter to you.
Later
I noticed that Fusion-io has written a white paper re the Fusion-io Difference (SSD Differentiator). It has a different perspective - but has nice pictures. For some of you that may be worth seeing too.
Later still or should it be PPS?
Take a look at this quote - "Hard drive emulation appeals to silicon providers because it offers the shortest route to market and requires the least innovation. By emulating disks, SSDs can be wedged into existing software stacks. Problem is - the native features of solid state are hidden by emulation. Performance gains are blunted, protection at the silicon level is ignored, and software interfaces at the cell level are simply not considered."
It's from a white paper Going Beyond SSD - to Software Defined Flash - which Fusion-io probably wrote a long time ago - but which I only saw recently (in April 2013). The article usefully encapsulates the key principles which make FIO's SSDs different. (I'm still going to call them SSDs.)
and now? - rackmounts too
Fusion-io was one of the companies which I discussed in my home page blog - (June 2013) - thinking inside the box will lead to better enterprise flash arraysIn a news story (September 2013) I thought it would be useful to recall the history of FIO's SSDs and technologies inside the rackmount SSD market. For the sake of speed and simplicity I've cut and pasted it here below. I may tidy it up later into a separate article.
Fusion-io - upcoming webcast on rackmount related technologies
Editor:- September 11, 2013 - The other thing which Fusion-io would like you to recognize them for - is rackmount storage. The company yesterday announced that it will be talking about this in a webcast tomorrow (Thursday 9am PT).
Editor's comments:- Using PCIe SSDs as components within rackmount SSDs is already a well established concept in SSD's recent past - and expected to remain one of the key uses in enterprise SSD's future too.
A company called Dolphin launched the first such systems back in March 2009.
And Fusion-io's own ioDrives have been integrated by various companies within rackmount storage systems in the past starting with :- NextIO - as fast but software-less storage (April 2010), and notably followed in a very different way (as the raw flash in ultrafast high availability FC SAN SSDs) by Kaminario (September 2011).
But for anyone who thought this might be a good idea - but didn't see why they should have to buy this kind of solution from another new vendor - they only had to wait till August 2012 - which is when Fusion-io launched its ION product - a software bundle which enabled any user to build their own legacy software compatible fast FC SAN compatible SSD rack using a bunch of iodrives and almost any customer preferred standard server.
That created the possibility of a new competitive choice for those in the Violin performance category.
But for those interested in SSD acceleration whose needs for performance and cost were more modest - there would soon be another way they could use FIO's PCIe SSDs in a different way (as the flash cache in iSCSI hybrid HDD racks).
An early example of this in March 2011 - was iSCSI hybrid systems by StoneFly and then later we heard about iSCSI systems by NexGen Storage (which Fusion-io acquired in April 2013).
Nevertheless - for most people in the enterprise SSD market - the mere mortals who haven't already got inside Fusion-io's priviledged big customer gatekeeper orbit - it has been exceedingly difficult to get a coherent picture of whether these systems products are relevant - and if so - how to buy them.
Delays in getting that information were undoubtedly not helped by the need to assimilate the NexGen products into a business culture in which hard drives had previously been anathema - coupled with an ultra competitive market outlook (the quality and diversity of competing options facing users in this market is very high) and a reorganization due to the precipitous change in leadership 4 months ago.
So despite the title (Flash First for Hybrid and All Flash Storage) I'm sure that even more of you than usual - will be interested to learn what FIO has to say on this particular subject.
more links to FIO rackmount info - added later
re ioControl - FIO's aggressively priced hybrid rackmount for the iSCSI SSD market
ioControl (short intro) on SlideShare.net
re ION - FIO's Violin-smasher class fast rackmount offering for the FC SAN SSD market
| the fastest SSDs latency loving reasons for fading out DRAM where are we heading with memory intensive systems? | ||
| .. | ||
| ||
| .......... | ||
.......... | ||
| .. | ||
| .. | ||
| ||
.. | ||
| ||
. | ||
. | ||
| ||
| . | ||
| . | ||
| PCIe SSDs what changed in SSD
year 2015? | ||
| . | ||
| ||
| .. | ||
Shown above - an early example of a software focused banner ad from Fusion-io from around 2010. Fusion advertised its PCIe SSDs on StorageSearch.com from 2008 upto the acquisition by SanDisk in 2014. | ||
| .. | ||
| ||
| .. | ||
| ||
| .. | ||
| .. | ||
| .. | ||
| . | ||
| ||
| . |