Storage Networking

Storage Networking focuses upon virtualizing the storage and the SAN while collapsing and simplifying the management and performance of storage. This includes the role of storage virtualization in the Software Defined Data Center, and the comparison of pooled local storage options with traditional network attached storage (NAS) and fiber channel attached storage. Covered vendors include VMware, EMC, IBM, Dell, HP, Cisco, Tintri, and Nutanix.

The Big Data Back End for the SDDC Management Stack

SDDC.Management.Stack.Reference.ArchitectureIn, Building a Management Stack for Your Software Defined Data Center, we proposed a reference architecture for how one might assemble the suite of management components that will be needed to manage a Software Defined Data Center (SDDC). In this post we take a look at the need for that management suite to be supported by a multi-vendor big data datastore, and take a look at who might provide such a data store.

The Need for a Multi-Vendor Management Data Store in Your SDDC Management Stack

So why you ask, will a SDDC require a set of management products that will in turn require a multi-vendor big data back end. The reasons are as follows:

  1. The whole point of moving the configuration and management of compute, memory, networking and storage out of the respective hardware and into a software layer abstracted from the hardware is to allow for configuration and management changes to happen both more quickly and more automatically (which means very quickly). Each configuration change or policy change is going to create a blizzard of management data.
  2. If you look at all of the horizontal boxes in our Reference Architecture (below) each one of them along with the vertical IT Automation box will be generating data.
  3. The rate of change in the SDDC will be high enough so as to require fine grained and very frequent monitoring at every layer of the infrastructure.
  4. Just combining the number of layers with the rate of change with the need for fine grained and high frequency monitoring (5 minutes is an eternity) creates a big data problem.
  5. Finally, the need to be able to to cross layer root cause analytics (where in the software or hardware infrastructure is the cause of that application response time problem) means that the root cause analysis process has to cross domains and layers. This in and of itself calls for a common data store across management products.

The Software Defined Data Center Management Stack Reference Architecture



Who Could Provide the Multi-Vendor Big Data Repository?

There are two basic criteria for being able to provide such a repository. The first is that you have to have one, or have the intention to build one. The second is that since it is multi-vendor, you have to have the technical capability to, and the business model to partner with the vendors whose products will feed this datastore. The rest of this post is entirely speculative in nature as it is based upon who could do what, not upon who is doing what. To be clear, no vendor listed below has told us anything about what they intend to do in this regard. The rest of this post is based entirely upon what people are shipping today and the author’s speculation as to what might be possible.


If there is one vendor who has an early lead in filling this role it would be Splunk. Splunk is in fact the only vendor on the planet from whom you can purchase an on-premise big data datastore, which is today based upon shipping and available products being populated by data from management products from other vendors.  In fact if you go to SplunkBase do searches on things like APM, monitoring, security, and operations you will find a wide variety of Splunk written and vendor written applications that feed data into Splunk. Now it is important to point that today, when a vendor like ExtraHop Networks or AppDynamics feeds their data into Splunk they are not making Splunk in THE back end datastore for their products. They are just feeding a subset and a copy of their data into Splunk. But this is a start, and it puts Splunk further down this road than anyone else. Needless to say, if the vision of the multi-vendor datastore is correct, and Splunk is to become the or one of the vendors who provides this, then Splunk is going to have to entice a considerable number of software vendors to trust Splunk to perform a role that no vendor today trusts any other vendor to perform.


In, VMware Fleshes Out SDN Strategy with NSX, we went through how VMware is combining Nicira into its Network Virtualization offering, NSX. In the VMware announcement there was a link to a blog post, “Open Source, Open Interfaces, and Open Networking” contained the following fascinating statement:

Statistics Collection & Telemetry

“Another area of focus for an open networking ecosystem should be defining a framework for common storage and query of real time and historical performance data and statistics gathered from all devices and functional blocks participating in the network. This is an area that doesn’t exist today. Similar to Quantum, the framework should provide for vendor specific extensions and plug-ins. For example, a fabric vendor might be able to provide telemetry for fabric link utilization, failure events and the hosts affected, and supply a plug-in for a Tool vendor to query that data and subscribe to network events”.

Needless to say, it is highly unlikely that VMware would choose to make the current datastore for vCenter Operations into the “framework for the common storage and query of real time performance data“. Rather it is much more likely that VMware would build its own big data datastore with the people and the assets that VMware acquired when VMware acquired the Log Insight technology and team from Pattern Insight. VMware therefore clearly has the technology building blocks and the people to pull this off. You could also argue that they would not have make this acquisition if there were not intentions to go at least somewhat in this direction. The key challenge for VMware will then be the multi-vendor part. VMware has no relationship of technical cooperation with any other management software companies other than Puppet Labs, so this is clearly an area where VMware has a long way to go.

New Relic

New Relic is the hands down market leader for monitoring Java, .NET, Ruby, Python, and PHP applications in the cloud. New Relic offers cloud hosted APM as a Service and has gone in four years from a brand new company to now having more organizations using its product than the rest of the APM industry combined. New Relic recently raised $75M from top tier investors and is rumored to be positioning itself for an IPO in the 2014-2015 timeframe. New Relic already makes its data available to third parties in its partner program via a REST API. It is not much of a stretch for New Relic to consider becoming the management platform in the cloud, partnering with adjacent vendors and becoming a vendor of the multi-vendor cloud datastore. Again all of this is pure speculation at this point.

The Pivotal Initiative

The Pivotal Initiative is a new company formed with assets and people from EMC and VMware lead by former VMware CEO Paul Maritz. These assets consist of the application platform PaaS products from VMware (Gemfire, Spring, vFabric and CloudFoundry), and the big data assets fromEMC (GreenPlum). The stated ambition is to deliver a way to build and deploy big data applications that is radically better than the incumbent method and tackle giants like IBM, Microsoft, and Oracle in the process. This means that the focus of both the application development assets and the big data assets is most likely to be upon solving business problems for customers, not IT management problems for customers. However, it would not be inconceivable for a third party company to license these technologies from Pivotal and build an offering targeting the multi-vendor management stack use case.


Consider the possibility that this multi-vendor big data datastore is in fact non on-premise, but in the cloud. If you are willing to consider that possibility, then it is not much of a stretch to consider that CloudPhysics a vendor with cloud hosted (delivered as a service) operations management solutions might step into this fray. One of the key reasons that CloudPhysics may be able to provide something of extraordinary value is that the company has a strategy of applying Google quality analytics to Google size data sets. The analytics come from a world class team of people some of whom previously worked at Google. The data today is collected by virtual appliances installed at CloudPhysic’s customer sites (in their respective VMware environments). If CloudPhysics is already collecting data across customers and putting it in its cloud, it is not too huge a stretch to consider the possibility that other vendors who also deliver their value as a service could partner up with CloudPhysics, combine their respective sets of data, and produce a 1+1=3 scenario for joint customers.


AppNeta is today a market leading vendor of a cloud hosted service, PathView Cloud, that measures the performance of the wide area network in between the users and branch offices of an enterprise and the enterprises back end data center. The back end is a true big data back end, built around true big data technologies. AppNeta is branching out into APM with its TraceView offering.  But network performance data and application performance data are just parts of the compete set of data that will get generated by the SDDC and about the SDDC by various management products. AppNeta does not today have a partner program to attract third party data to its management data cloud, but who knows what the future holds.


Boundary is an APM vendor with a cloud hosted big data back end that today focuses upon collecting the statistics from the network layer of the operating system that support applications running in clouds. If you think of New Relic as the vendor who is monitoring your application in the cloud, you can think of Boundary as the vendor who should be monitoring the interaction of your operation system underlying your application with the cloud.  Boundary has no partner program today, and no ability to add third party vendor data to its cloud datastore today, but again who knows what the future might hold.


The SDDC and the Cloud are going to require a new SDDC Management Stack that will need to be based upon a multi-vendor big data datastore. There will likely be on-premise and cloud hosted version of these datastores. Splunk, VMware, New Relic, The Pivotal Initiative, CloudPhysics, AppNeta, and Boundary are all excellent hypothetical suppliers of such a datastore.

Hard Disk Drives (HDD) for virtual environments (Part IV) from power to warranties

StorageNetworkingBy Greg Schulz, Server and StorageIO @storageio

Let us pick up where we left off in part III in our look beyond the covers to help answer the question of which is the best HDD to use.

Power and energy

Power consumption will vary based on size and type of HDD, along with different usage. For example, during power-up there is a larger amount of energy being used vs. when the drive is idle (not reading or writing) yet still spinning, or actively reading and writing. With intelligent power management (IPM), inactive drives can go into lower power usage modes with variable performance. IPM includes the ability to vary the amount of power used to level of performance with different steps or levels. This is different from some first generation MAID solutions based on desktop class drives that were either on or off with subsequent performance issues. While an HDD requires power to spin the platters, once those are in motion, less power is required; however, energy is needed for the read write heads and associated electronics.

This leads to a common myth or misperception that HDDs consume a lot of energy because they are spinning. There is energy being used to keep the platters moving, however power is also required for the electronics to manage the drives interface, read write heads and other functions. With IPM leaving the drive spinning or reducing the rotational speed can help save power, so to can disabling or putting into low power mode the associated processors and control electronics.

As a comparison SSD, drives are often touted as not drawing as much energy compared to an HDD, which is true. However, SSDs do in fact consume electricity and get warm as they also have electronics and control processors similar to HDDs. If you do not believe this, put an SSD to work and feel it over time as it heats up. Granted that is an apple to oranges comparisons, however my point is that there is more to energy savings with HDDs than simply focusing on the rotational speeds. Speaking of energy savings, typical enterprise class drives are in the 4 to 8 watts or a fraction of what they were only a few years ago. Notebook, laptop and workstation drives can be in the single watt to a few watts in power usage range. Note that these numbers may be less than what some will talk about when comparing SSD and HDDs, or trying to make a point about HDDs and power consumption. The reason is this is a reduction from where just a few years ago when drives were in the “teens” in terms of watts per drive. For performance or active drives, compare those on a cost per activity per watt such as cost per IOP per watt, for inactive data then cost per capacity per watt can be relevant.


Given the large amount of data that can be stored on an HDD along with compliance and other concerns, drive level security is becoming more common. There are different types of drive level encryption including self-encrypting devices (SEDs) with some supporting FIPS level requirements. Drive level encryption depending on implementation can be used to off-load servers, workstations or storage systems from performing encrypt and decrypt functions.

Space capacity

The space capacity of the drives is determined by the aerial density (how many bits in a given amount of space) per platter, the size of the platter (3.5” are larger than 2.5”) and number of platters. For example at the same aerial density, more bits and bytes exist on a 3.5” vs. 2.5” device, and by adding more platters (along with read/write heads) the resulting taller height drive has even more space capacity. Drive space capacities include 4TB and smaller for 3.5” devices and TB plus sized for various 2.5” form factors. Watch out for “packaging” games where for example a drive is offered as say 4TB that are actually two separate 2TB drives in a common enclosure (no RAID or NAS or anything else).

The super parametric barrier effects keeps being delayed, first with perpendicular recording, now with shingled magnetic recording (SMR) and heat assisted magnetic recording (HAMR) all in the works. The super parametric barrier is the point where data bits can no longer safely (with data integrity) be stored and later used without introducing instability. Watch for more on SMR and HAMR in a later post when we look at new and emerging trends.

Speaking of space capacity, ever wonder where those missing bits and bytes disappeared on a HDD or SSD? First there is how it is measured, meaning decimal or base10 vs. binary base 2 for example one Gigabyte (GB) being one billion bytes vs.  1,024,000,000.00 bytes. These space capacities are before RAID or hypervisor or operating system and file system formatting overhead are added. There is also reserved space for bad block re vectoring which can be thought of as hot spare blocks for when the drive (HDD or SSD) detects something going bad. In addition to the bad block areas, there are also some reserved space that you will not be able to access that is kept for drive management, read/write head alignment and other things.

Speaking of large capacity drives, as mentioned earlier, rebuild operations with RAID configurations can take longer given more data to move. Good news is that some RAID systems or solutions can rebuild a 1TB or 2TB drive as fast as or faster than a 9GB drive from a decade ago. The catch is that there are more drives and they are getting larger with 3TB and 4TB shipping and larger ones in the works. Things you can do to minimize the impact of long rebuild times; include selecting the right type of drive that has better endurance, reliability and availability. This could mean that selecting a lower priced drive up front that is not as reliable could cost you down the road. Configuration including RAID level, number of parity drivers, and software, adapter, controller or storage system with ability to accelerate rebuilds can also make a difference.

Another impact of large capacity drives or large numbers of HDDs in general is how to securely erase them when decommissioning. That is assuming you are securely erasing them or taking other safeguards disposition vs. throwing in the garbage or giving them away. Self-encrypting devices (SEDs) normally associated with security can be part of a solution for some environments. Since SEDs can effectively erase the data stored on those by, removing the enablement key, instead of hours or days, for some environments secure erase can be in minutes or less.


There are various warranties on HDDs, those from the manufacture that may be the same as what an OEM or system integrator passes on to their customers. Some HDDs have a manufactures limited warranty of five years while others have shorter terms. Thus while a manufacture may offer a five year warranty, it can be up to the OEM or integrator to pass that warranty on, or in turn provider a shorter duration with different terms or price. Something to think about in terms of HDD warranties is that replacing them can mean sending your old device back in exchange for a new one. If you have sensitive or secure data on those devices, how will they be disposed of? An option is to not leverage return to vendor or manufacture warranties opting for self-disposition, or using self-encrypting devices (SEDs).

This wraps up this post, coming up next in part V we will look at what to use when, where along with other options and some trends.

Ok, nuff said (for now).

Cheers gs

Hard Disk Drives (HDD) for virtual environments (Part III) from form factor to power

StorageNetworkingBy Greg Schulz, Server and StorageIO @storageio

In part II of this series we covered some of the differences between various Hard Disk Drive (HDD) including looking beyond the covers at availability, cache and cost. Let us pick up where we left off on our look beyond the covers to help answer the question of which is the best HDD to use.

Form factor (physical attributes)

Physical dimensions including 2.5” small form factor (SFF) and 3.5” large form factor (LFF) HDDs. 2.5” HDDs are available in 7mm, 9mm and larger 14mm height form factors. Note that taller drives tend to have more platters for capacity. In the following image note that the bottom HDDs is taller than the others are.

Hard Disk Drive Sizes
Top thin 7mm, middle 9mm, and bottom 15mm (thick)

The above tall or “thick” (not to be confused with thick or thin provisioned) is a SFF 5.4K RPM 1.5TB drive that I use as an on-site backup or data protection target and buffer. The speed is good enough for what I use it for, and provides good capacity per cost in a given footprint.

Also, note that there is a low profile 7mm device (e.g. middle) that for example can fit into my Lenovo X1 laptop as a backup replacement for the Samsung SSD that normally resides there. Also shown on the top is a standard 9mm height 7.2K Momentus XT HHDD with 4GB of slc nand flash and 500GB of regular storage capacity.


Functionality include rebuild assist, secure erase, self-encrypting device (SEDs) without or without FIPS, RAID assist, support for large file copy (e.g. for cloud, object storage and dispersal or erasure code protection). Other features include intelligent power management (beyond first generation MAID), native command queue (NCQ), and Advanced Format (AF) 4Kbyte block and 512 byte emulation). Features also include those for high-density deployments such as virtualization and cloud such as vibration management in addition to SMART (Self-Monitoring, Analysis, and Reporting Technology) reporting and analysis.

Drives can also depending on vendor, make and model support various block or sector sizes including standard 512, 520, 524 and 528 for  different operating systems, hypervisors or controllers. Another feature mentioned above is the amount of volatile (DRAM) or persistent (nand flash) cache for read and read-ahead. Some drives are optimized for standalone or JBOD (Just a Bunch of Disks) and others for use with RAID controllers. By the way, put several SSD drives into an enclosure without a controller and you have Just a Bunch Of SSDs or JBOS.  What this means is that some drives are optimized to work with RAID arrays and how they chunk or shard data while others are for non-RAID use.

Speaking of RAID and HDDs, have you thought about your configuration settings, particular if working with big data or big bandwidth and large files or objects? If not you should including paying attention to stripe, chunk or shard size of how much data gets written to each device. With larger IO sizes, revisit what the default settings are to determine if you need to make some adjustments. Just as some drives are optimized for working with RAID controllers or software, there are some drives being optimized for cloud and object storage along with big data applications. The differences is that these drives are optimized for moving larger chunks or amounts of data usually associated with distributed data dispersal, erasure coding and enhanced RAID solutions. An example of a cloud storage optimized HDD is the Seagate Constellation CS (Cloud Storage).

Moving on, some drives are designed to be spinning or in constant use while others for starting and stopping such as with a notebook or desktop. Other features appearing in HDDs support high-density, along with hot and humid environments for cloud and managed service provider or big data needs. The various features and functionality can be part of the firmware enabled for a particular device along with hard features built into the device.

Interface type and speed

The industry trend is moving towards 6Gb SAS for HDDs similar to that for SSD drives. However, there is also plenty of 6Gb SATA activity, along with continued 4Gb Fibre Channel (4GFC) that eventually will transition to SAS. There is also prior generation 3Gb SAS and 3Gb SATA and you might even have some older 1.5Gb SAS or SATA devices around, maybe even some Parallel ATA (PATA) or Ultra320 (Parallel SCSI). Note that SATA devices can plug into and work with SAS adapters and controllers, however not the other way around.

Note that if you see or hear about a storage system or controller with back-end 8Gb Fibre Channel, chances are the HDD would auto-throttle negotiate down to 4GFC. In addition to the current 6Gb speed of SAS, there are improvements in the works for 12Gb and beyond, along with many different topology or configuration options. If you are interested in learning more about SAS, check out SAS SANs for Dummies sponsored by LSI that I helped write.

Notice I did not mention iSCSI, USB, Thunderbolt or other interfaces and protocols? Some integrators and vendors offer drives with those among other interfaces, they are usually SAS or SATA with a bridge, router or converter interface attached to them externally or as part of their packaging (See following image).

Performance of the device

A common high-level gauge of drive performance is the platter rotational speed.  However there is other metrics including seek time, transfer rate and latency. These in turn vary based on peak and sustained, read or write, random or sequential, large or small IOPS or transfer requests. There are many different numbers floating around as to how many IOPS a HDD can do based on its rotational speed among other factors. The challenge with these numbers or using them is putting into context of what size the IOP is, was it a read or write, large or small, random or sequential relative to your needs. Another challenge is how those IOPs are measured, for example were the measured below a file system to negate buffering, or via a file system.

Rotational speed such as 5,400 (5.4K) revolutions per minute (RPM), 7.2K, 10K and 15K RPMs. Note that while a general indicator of relative speed, some of the newer 10K SFF (e.g. 2.5”) HDDs provide the same or better performance of earlier generation 3.5” 15K devices. This is accomplished with a combination of smaller form factor (spiral transfer rate) and improvements in read/write electronics and firmware. The benefit is that in the same or smaller footprint, more devices, performance and capacity can be packaged as well as the devices individually using less power. Granted if you pack more devices into a given footprint, the aggregate power might increase, however so too does the potential performance, availability, capacity and economics depending on implementation. You can see the differences in performance using various HDDs including an HHDD in this post here that looked at Windows impact for VDI planning.

This wraps up this post, up next part IV, we continue our look beyond the covers to determine the differences and what HDD is best for your virtual or other data storage needs.

Ok, nuff said (for now).

Cheers gs


Hard Disk Drives (HDD) for virtual environments (Part II) how drives differ

StorageNetworkingBy Greg Schulz, Server and StorageIO @storageio

In part I of this series we looked at basic Hard Disk Drive (HDD) characteristics and wrapped up with the question of what is the best type of HDD to use?

I often get asked why there needs to be different types or tiers of data storage devices including HDD and Solid State Devices (SSDs), along with interfaces, why not just one or a few? Continue reading Hard Disk Drives (HDD) for virtual environments (Part II) how drives differ

Building a Management Stack for Your Software-Defined Data Center

SDDC.Management.Stack.Reference.ArchitectureData Center Virtualization has spawned several entirely new categories and variants of management software. This is largely because data center virtualization alone was a large enough change to create new requirements that legacy management products could not meet. This created a new constituency for management solutions—the virtualization team—which proceeded to purchase management solutions that met their needs. This trend was facilitated by the “easy to try and easy to buy” business model that many of the new vendors of virtualization management solutions adopted. Out of this a new management software industry arose. Continue reading Building a Management Stack for Your Software-Defined Data Center

Virsto: Software Defined Data Center: Tip of the Iceberg

CloudComputingVMware buying Virsto is a big move and after considerable discussion a logical step for VMware in many technical areas as well. We previously mentioned that Virsto would add to VMware’s existing in Software Defined Data Center (SDDC), but there is more to this than just SDDC, which I believe is the end goal. Getting there absolutely requires a storage abstraction layer. So what does VMware gain other than SDDC with Virsto. Continue reading Virsto: Software Defined Data Center: Tip of the Iceberg