Knowledge

• Identify storage provisioning methods
  
• Identify available storage monitoring tools, metrics and alarms

Skills and Abilities

• Apply space utilization data to manage storage resources
  
• Provision and manage storage resources according to Virtual Machine requirements
• Understand interactions between virtual storage provisioning and physical storage provisioning
• Apply VMware storage best practices
• Configure Datastore Alarms
• Analyze Datastore Alarms and errors to determine space availability
• Configure Datastore Clusters

Identify storage provisioning methods

Official Documentation: –
There are two types of storage that can be provisioned through vSphere; block storage and NAS.

• Block Storage
  
  • Local – any local storage attached to the host; uses VMFS
  • iSCSI – IP storage using a hardware or software iSCSI initiator; uses VMFS
  • FCoE – Fibre Channel over Ethernet using a hardware of software HBA; uses VMFS
  • FC – Fibre Channel using a hardware HBA; uses VMFNAS Storage
• NAS Storage
  • NFS – currently using NFSv3 to mount NFS shares as datastores; uses NFS instead of VMFS

GUI Provisioning Method

The easiest way to provision storage is using the vSphere client. From the vSphere client you can create VMFS 3 or VMFS 5 datastores, you can create Raw Device Mappings or create a Network File System. You can do all this through the Add Storage wizard from within the client

• Log into the vSphere client
  
• Select a host > click the Configuration Tab
• Click the Storage hyperlink
• Click the Add Storage. . . hyperlink to launch the Add Storage wizard

From the Add Storage wizard you can provision block or NAS storage into the vSphere environment
Command-line Provisioning Methods

To provision storage through the command-line you can use vmkfstools. See http://pubs.vmware.com/vsphere-50/index.jsp?topic=%2Fcom.vmware.vcli.ref.doc_50%2Fvcli-right.html

There aren’t a WHOLE lot of options for this command as it relates to creating file systems. you can also use vmkfstools to provision virtual disks. Here are the options:

• You can specify whether it will be VMFS 3 or VMFS 5
  
• You can set a block size (VMFS 3 ONLY)
• You can set the volume name
• You can also choose to span or grow an existing file system

For an example see VMware KB1009829
Identify available storage monitoring tools, metrics and alarms

Official Documentation: –
Storage can be monitored with a few tools from VMware, there are also a few nice ones from 3rd party software companies like Veaam or Vizoncore.

The VMware tools are:

• ESXTOP and RESXTOP
  
• vscsiStats
• VMware vCenter
• VMware vSphere Client.

Eric Siebert has written an article about VMware vSphere’s built-in performance monitoring tools, see his article at http://searchvmware.techtarget.com/tip/VMware-vSpheres-built-in-performance-monitoring-tools

VMware Knowledgebase article 1008205 describes how to use ESXTOP and or RESXTOP to identify performance issues. See http://kb.vmware.com/kb/1008205

For some more info about the vscsiStats see Yellow Bricks article http://www.yellow-bricks.com/2009/12/17/vscsistats/

There is also a special VMware training available, this is called: VMware vSphere: Skills for Storage Administrators. See Course datasheet at http://mylearn.vmware.com/mgrreg/courses.cfm?ui=www_edu&a=det&id_course=85301

Scott Sauer has created an article about Performance Troubleshooting on Storage, see http://www.virtualinsanity.com/index.php/2010/03/16/performance-troubleshooting-vmware-vsphere-storage/ He also discusses some tools used on different levels in the VMware infrastructure.
Apply space utilization data to manage storage resources

Official Documentation:

vSphere Monitoring and Performance, Chapter 4,”Monitoring Storage Resources”, page 31.
If you use vCenter Server to manage your hosts, you can review information on storage usage and visually map relationships between all storage entities that are available in vCenter Server.

In the vSphere Client, for any inventory object except networking, the storage usage data appears in the Storage Views tab. To view this tab, you must have the vCenter Storage Monitoring plug-in, which is generally installed

and enabled by default.

You can display storage information as reports or storage topology maps.

• Reports Reports display relationship tables that provide insight about how an inventory object is associated with storage entities. They also offer summarized storage usage data for the object’s virtual and physical storage resources. Use the Reports view to analyze storage space utilization and availability, multipathing status, and other storage properties of the selected object and items related to it.
  If you use arrays that support storage vendor providers developed through Storage APIs – Storage Awareness, the Reports view offers additional information about storage arrays, storage processors, ports, LUNs or file systems, and so on. For more information about vendor providers, see the vSphere Storage documentation.
  
• Maps Storage topology maps visually represent relationships between the selected object and its associated virtual and physical storage entities.

Working with Storage Reports

You monitor storage information through the Reports view on the vSphere Client Storage Views tab.

For the object you select in the Inventory, a list of categories associated with the object is available. You can display and review statistics for each category of items on the list depending on the inventory object. For example, if the inventory object is a datastore, you can display information for all virtual machines that reside on the datastore, all hosts that have access to the datastore, a LUN or LUNs, on which the datastore is deployed, and so on. In addition, if your storage supports vendor providers developed through the Storage APIs – Storage Awareness, you can also see information about physical arrays. For information on vendor providers, see the vSphere Storage documentation.

When you display the reports tables, the default column headings depend on the inventory object you select. You can customize the tables by adding or removing columns. Reports are automatically updated every 30 minutes. You can manually update the reports by clicking the Update link. You can search for specific information you need to see by filtering reports tables based on storage attributes and keywords.
Working with Storage Maps

The Maps view on the vSphere Client Storage Views tab helps you visually represent and understand the relationships between an inventory object and all virtual and physical storage resources available for this object.

Maps are object-centric and display only items relevant to the specific object.

Maps are automatically updated every 30 minutes. You can manually update the maps by clicking the Update link.

You can customize a map view by selecting or deselecting options in the Show area, or by hiding specific items or changing their position on the map.

You can reposition the map by dragging it, and zoom in or out of the map or a section.
Provision and manage storage resources according to Virtual Machine requirements

Official Documentation:

vSphere Virtual Machine Administration, Chapter 4 “Creating a Virtual Machine in the vSphere Client”, Section “Selecting a Virtual Disk Type”, page 38. More useful information in Chapter 8 “Configuring Virtual Machines”
When you are provisioning storage resources to a VM, you make several decisions like:

• Type of Storage Controller. Today for a virtual SCSI controller, four controller types exist: (Buslogic Parallel, LSI Logic Parallel, LSI Logic SAS and VMware paravirtual)
  
• Type of Disk, RDM or Virtual disk (Tick Provision Lazy Zeroed, Thick Provision Eager Zeroed or Thin Provision)

But when it comes to selecting the Datastore that will store your newly created virtual disk, you are probably making the most important decision. By choosing a Datastore, you are selecting your type of Physical Storage and making decisions concerning:

• Local Storage
  
• Networked Storage
  • Fibre Channel (FC)
  • Intermet SCSI (iSCSI)
  • Network-attached Storage (NAS, aka NFS)
  • Shared Serial Attached SCSI (SAS)
  • RAID levels
  • Number of physical Disks in the Volume
  • Path Selection Policies

When placing a virtual disk on a Datastore, besides capacity, be aware of the requested disk performance in terms of R/W speed and IOPS. Listen to end-users and monitor the performance with use of the vSphere Client and/or ESXtop.
RDM Disk.

You can store virtual machine data directly on a SAN LUN instead of storing it in a virtual disk file. This ability is useful if you are running applications in your virtual machines that must detect the physical characteristics of the storage device. Mapping a SAN LUN allows you to use existing SAN commands to manage storage for the disk.

When you map a LUN to a VMFS volume, vCenter Server creates a Raw Device Mapping (RDM) file that points to the raw LUN. Encapsulating disk information in a file allows vCenter Server to lock the LUN so that only one virtual machine can write to it at a time. For details about RDM, see the vSphere Storage documentation.

The RDM file has a .vmdk extension, but the file contains only disk information that describes the mapping to the LUN on the ESXi host. The actual data is stored on the LUN.

You can create the RDM as an initial disk for a new virtual machine or add it to an existing virtual machine.

When you create the RDM, you specify the LUN to be mapped and the datastore on which to put the RDM.
Virtual Disk.

When you create a virtual disk, you can specify disk properties such as size, format, clustering features, and more.

Different formats for the virtual machine’s disks.

• Thick Provision Lazy Zeroed Create a virtual disk in a default thick format. Space required for the virtual disk is allocated during creation. Any data remaining on the physical device is not erased during creation, but is zeroed out on demand at a later time on first write from the virtual machine.
  
• Thick Provision Eager Zeroed Create a thick disk that supports clustering features such as Fault Tolerance. Space required for the virtual disk is allocated at creation time. In contrast to the flat format, the data remaining on the physical device is zeroed out during creation. It might take much longer to create disks in this format than to create other types of disks.
• Thin Provision Use the thin provisioned format. At first, a thin provisioned disk uses only as much datastore space as the disk initially needs. If the thin disk needs more space later, it can grow to the maximum capacity allocated to it.

Other references:

• The blog Simple-talk.com did an article about Virtual Machine Storage Provisioning and best practices. See http://www.simple-talk.com/sysadmin/virtualization/virtual-machine-storage-provisioning-and-best-practises/
  
• VMware has also created an document called StorageWorkload Characterization and Consolidation in Virtualized Environments. You can find it here: http://www.vmware.com/files/pdf/partners/academic/vpact-workloads.pdf
• Microsoft has a document to analyse storage performance, see http://blogs.technet.com/b/cotw/archive/2009/03/18/analyzing-storage-performance.aspx

Understand interactions between virtual storage provisioning and physical storage provisioning

Official Documentation: –

See previous point and objective 1.1
Apply VMware storage best practices

Official Documentation: –
Many of the best practices for physical storage environments also apply to virtual storage environments. It is best to keep in mind the following rules of thumb when configuring your virtual storage infrastructure:

Configure and size storage resources for optimal I/O performance first, then for storage capacity.

This means that you should consider throughput capability and not just capacity. Imagine a very large parking lot with only one lane of traffic for an exit. Regardless of capacity, throughput is affected. It’s critical to take into consideration the size and storage resources necessary to handle your volume of traffic—as well as the total capacity.

Aggregate application I/O requirements for the environment and size them accordingly.

As you consolidate multiple workloads onto a set of ESX servers that have a shared pool of storage, don’t exceed the total throughput capacity of that storage resource. Looking at the throughput characterization of physical environment prior to virtualization can help you predict what throughput each workload will generate in the virtual environment.

Base your storage choices on your I/O workload.

Use an aggregation of the measured workload to determine what protocol, redundancy protection and array features to use, rather than using an estimate. The best results come from measuring your applications I/O throughput and capacity for a period of several days prior to moving them to a virtualized environment.

Remember that pooling storage resources increases utilization and simplifies management, but can lead to contention.

There are significant benefits to pooling storage resources, including increased storage resource utilization and ease of management. However, at times, heavy workloads can have an impact on performance. It’s a good idea to use a shared VMFS volume for most virtual disks, but consider placing heavy I/O virtual disks on a dedicated VMFS volume or an RDM to reduce the effects of contention.
Other references:

VMware Presentation: VMware Storage Best Practices

VMware document: Dynamic Storage Provisioning. Considerations and Best Practices for UsingVirtual Disk Thin Provisioning

VMware document: Best Practices for Running vSphere on NFS Storage
Configure Datastore Alarms

Official Documentation:

vSphere Examples and Scenarios, Chapter 10,”Alarm Example: Setting an Alarm Action for Datastore Usage on a Disk”, page 91.
Dozens of default alarm definitions are provided by the vSphere Client, which you can locate in the Alarms tab for an inventory object. One of these default alarms is Datastore usage on disk, which you use to monitor the percentage of disk usage. This kind of alarm is important if the virtual machines in the datastore have virtual disks in the thin provisioned format. With thin provisioning, at first a thin provisioned disk uses only as much datastore space as the disk initially needs. However, if the thin disk needs more space later, it can grow to the maximum capacity allocated to it. With thin provisioning, it is possible to oversubscribe storage space if the virtual machines grow unattended.

An alarm set on the datastore can notify you when the space issues threaten to become critical.
Required Privileges for Configuring and Acknowledging an Alarm

Someone who is assigned the Administrator role can configure alarms. Alarm privileges do not appear in other default sample roles accessible with the vSphere Client. To create a unique role for this process that does not include all the privileges of an Administrator, the following list contains the minimum privileges necessary.

• Alarms.Create alarm
  
• Alarms.Modify alarm
• Alarms.Acknowledge alarm

For information about roles, permissions, and privileges, see the vSphere Security documentation.
Configure and Act on an Alarm in a Scenario

Change alarm settings and specify what actions are taken when the alarm is triggered. Take ownership of an issue by acknowledging the alarm.

The Datastore usage on disk alarm is one of the many default alarms included with vCenter Server. The alarm is enabled at installation. If the used disk space on the datastore exceeds 75 percent, the alarm triggers a Warning. If the used disk space on the datastore then exceeds 85 percent, the alarm triggers an Alert. These warnings and alerts appear in the user interface in the form of modified icons for the elements affected. To have the alarm perform actions beyond this visual notification, change the configuration of the alarm. Follow the workflow in the sample tasks in this example scenario to become familiar with changing the settings of an alarm and responding to a triggered alarm.

Changes that you can make to this alarm are altering the percentage values for a Warning and for an Alert, modifying the frequency of an alarm, and defining the action the system takes at an alarm. For this example, you lower the percentage values, but keep the default frequency. In addition, on the Actions tab, you configure the action so that the system sends an email to specified addresses when a condition triggers the alarm.
In the procedures for acknowledging and resetting triggered alarms, follow the steps described in the vSphere Client option for both.

Procedure

1. Access Alarm Settings So You Can Make Changes in an Example Scenario on page 93. You can view alarm settings from any object, but you can modify settings only through the object on which the alarm is defined. You create and modify alarms in the Alarm Settings dialog box.
2. Specify How the Alarm is Triggered (Condition or State-based) on page 94. You can specify the events, states, or conditions that triggers the alarm in the Triggers tab of the Alarm Settings dialog box. The options you choose under the General tab of the Alarm Settings dialog box determine the options available under the Triggers tab. An alarm definition must contain at least one trigger before it can be saved.
3. Specify Which Actions to Perform When Triggered on page 95. You can specify actions that the system performs when the alarm is triggered or changes status. You can enable and disable alarms and alarm actions independently of each other.
4. Acknowledge Triggered Alarms on page 96. Acknowledging an alarm lets other users know that you are taking ownership of the issue. After an alarm is acknowledged, its alarm actions are discontinued. For example, a host has an alarm set on it that monitors CPU usage and that sends an email to an administrator when the alarm is triggered. The host CPU usage spikes, triggering the alarm which sends an email to the host’s administrator. The administrator acknowledges the triggered alarm to let other administrators know he is working on the problem, and to prevent the alarm from sending more email messages. The alarm, however, is still visible in the system. Alarms are neither cleared, nor reset when acknowledged.
5. (Optional) Reset Triggered Event Alarms on page 97. An alarm triggered by an event might not reset to a normal state if vCenter Server does not retrieve the event that identifies the normal condition. In such cases, reset the alarm manually to return it to a normal state.

Other references:

See the vSphere Datacenter administration guide chapter 13, Working with Alarms. http://www.vmware.com/pdf/vsphere4/r41/vsp_41_dc_admin_guide.pdf

Jeremy Waldrop has an blog article written about the new alarms that are available in vCenter. See http://jeremywaldrop.wordpress.com/2010/01/24/vmware-vsphere-vcenter-storage-alarms/
Analyze Datastore Alarms and errors to determine space availability

Official Documentation:

see the previous point and vSphere Monitoring and Performance, Chapter 4,”Monitoring Events, Alarms and Automated Actions”, page 38.
There are different posts in the VMware community site of VMware. See the following.

• Datastore Free Space Best Practice
  
• vCenter DataStore alarms

Configure Datastore Clusters

Official Documentation:

vSphere Resource Management Guide, Chapter 11,”Creating a Datastore Cluster” , page 77. Also Chapter 12 “Using Datastore Clusters to Manage Storage resources”, page 83.
A datastore cluster is a collection of datastores with shared resources and a shared management interface. Datastore clusters are to datastores what clusters are to hosts. When you create a datastore cluster, you can use vSphere Storage DRS to manage storage resources.

NOTE Datastore clusters are referred to as storage pods in the vSphere API.

When you add a datastore to a datastore cluster, the datastore’s resources become part of the datastore cluster’s resources. As with clusters of hosts, you use datastore clusters to aggregate storage resources, which enables you to support resource allocation policies at the datastore cluster level. The following resource management capabilities are also available per datastore cluster.

• Space utilization load balancing.You can set a threshold for space use. When space use on a datastore exceeds the threshold, Storage DRS generates recommendations or performs Storage vMotion migrations to balance space use across the datastore cluster.
  
• I/O latency load balancing.You can set an I/O latency threshold for bottleneck avoidance. When I/O latency on a datastore exceeds the threshold, Storage DRS generates recommendations or performs Storage vMotion migrations to help alleviate high I/O load.
• Anti-affinity rules.You can create anti-affinity rules for virtual machine disks. For example, the virtual disks of a certain virtual machine must be kept on different datastores. By default, all virtual disks for a virtual machine are placed on the same datastore.

Initial Placement and Ongoing Balancing

Storage DRS provides initial placement and ongoing balancing recommendations to datastores in a Storage DRS-enabled datastore cluster.

Initial placement occurs when Storage DRS selects a datastore within a datastore cluster on which to place a virtual machine disk. This happens when the virtual machine is being created or cloned, when a virtual machine disk is being migrated to another datastore cluster, or when you add a disk to an existing virtual machine.

Initial placement recommendations are made in accordance with space constraints and with respect to the goals of space and I/O load balancing. These goals aim to minimize the risk of over-provisioning one datastore,storage I/O bottlenecks, and performance impact on virtual machines.

Storage DRS is invoked at the configured frequency (by default, every eight hours) or when one or more datastores in a datastore cluster exceeds the user-configurable space utilization thresholds. When Storage DRS is invoked, it checks each datastore’s space utilization and I/O latency values against the threshold. For I/O latency, Storage DRS uses the 90th percentile I/O latency measured over the course of a day to compare against the threshold.
Storage Migration Recommendations

vCenter Server displays migration recommendations on the Storage DRS Recommendations page for datastore clusters that have manual automation mode.

The system provides as many recommendations as necessary to enforce Storage DRS rules and to balance the space and I/O resources of the datastore cluster. Each recommendation includes the virtual machine name, the virtual disk name, the name of the datastore cluster, the source datastore, the destination datastore, and a reason for the recommendation.

• Balance datastore space use
  
• Balance datastore I/O load

Storage DRS makes mandatory recommendations for migration in the following situations:

• The datastore is out of space.
  
• Anti-affinity or affinity rules are being violated.
• The datastore is entering maintenance mode and must be evacuated.

In addition, optional recommendations are made when a datastore is close to running out of space or when adjustments should be made for space and I/O load balancing.

Storage DRS considers moving virtual machines that are powered off or powered on for space balancing. Storage DRS includes powered-off virtual machines with snapshots in these considerations.
Datastore Cluster Requirements

Datastores and hosts that are associated with a datastore cluster must meet certain requirements to use datastore cluster features successfully.

Follow these guidelines when you create a datastore cluster.

• Datastore clusters must contain similar or interchangeable datastores. A datastore cluster can contain a mix of datastores with different sizes and I/O capacities, and can be from different arrays and vendors. However, the following types of datastores cannot coexist in a datastore cluster.
  
  • NFS and VMFS datastores cannot be combined in the same datastore cluster.
  • Replicated datastores cannot be combined with non-replicated datastores in the same Storage-DRSenabled datastore cluster.
• All hosts attached to the datastores in a datastore cluster must be ESXi 5.0 and later. If datastores in the datastore cluster are connected to ESX/ESXi 4.x and earlier hosts, Storage DRS does not run.
• Datastores shared across multiple datacenters cannot be included in a datastore cluster.
• As a best practice, do not include datastores that have hardware acceleration enabled in the same datastore cluster as datastores that do not have hardware acceleration enabled. Datastores in a datastore cluster must be homogeneous to guarantee hardware acceleration-supported behavior.

Other exam notes

• The Saffageek VCAP5-DCA Objectives http://thesaffageek.co.uk/vcap5-dca-objectives/
  
• Paul Grevink The VCAP5-DCA diaries http://paulgrevink.wordpress.com/the-vcap5-dca-diaries/
• Edward Grigson VCAP5-DCA notes http://www.vexperienced.co.uk/vcap5-dca/
• Jason Langer VCAP-DCA notes http://www.virtuallanger.com/vcap-dca-5/
• The Foglite VCAP5-DCA notes http://thefoglite.com/vcap-dca5-objective/

VMware vSphere official documentation

Disclaimer.
The information in this article is provided “AS IS” with no warranties, and confers no rights. This article does not represent the thoughts, intentions, plans or strategies of my employer. It is solely my opinion.