NetApp Partition Alignment

Partition Alignment

Virtual machines store their data on virtual disks. As with physical disks, these virtual disks contain storage partitions and file systems, which are created by the VM’s guest operating system. In order to ensure optimal disk I/O within the VM one must align the partitions of the virtual disks to the block boundaries of VMFS and the block boundaries of the storage array. Failure to align all three of these items will result in a

dramatic increase of I/O load on a storage array and will negatively impact the performance of all Virtual Machines being served on the array.

The impact of mis-aligned partitions

Failure to properly align the file systems within Virtual Machines has a negative impact on many aspects of a Virtual Infrastructure. Customers may first notice the impact of misalignment with virtual machines running high performance applications. The reason for this is every I/O operation executed within the VM will require multiple I/O operations on the storage array.

In addition to the negative performance impact storage savings with NetApp Data Deduplication will be negatively impacted, reducing the total amount of storage savings. Finally, storage arrays will be over taxed and as the Virtual Infrastructure grows the storage array will require hardware upgrades in order to meet the additional I/O load generated by this misalignment. Simply put, one can save their company a significant amount of money by optimizing the I/O of their VMs.

Verifying partition alignment with windows operating systems

To verify the starting partition offset for a windows based virtual machine log onto the VM and run the System Information utility (or msinfo32). There you will be able to find this setting). To run msinfo32, select Start > All Programs > Accessories > System Tools > System Information.

NetApp volume Language

Volume Language

Every volume has a language. The storage system uses a character set appropriate to the language for the following items on that volume:

•  File names
• File access

The language of the root volume is used for the following items:

•  System name
•  CIFS share names
•  NFS user and group names
•  CIFS user account names
•  Domain name
•  Console commands and command output
•  Access from CIFS clients that don’t support Unicode
•  Reading the following files:
•  /etc/quotas
•  /etc/usermap.cfg
• the home directory definition file

You are strongly advised to set all volumes to have the same language as the root volume, and to set the volume language at volume creation time. Changing the language of an existing volume can cause some files to become inaccessible.

NetApp root volume

Recommended root volume (vol0) sizes

Root Volume

The storage system's root volume contains special directories and configuration files that help you administer your storage system. The storage system contains a root volume that was created when the storage system was initially setup at the factory. The root volume can be contained in its own aggregate which is the best practice for performance and scalability, if you are restrained by the amount of disks then the root volume can also exist in a larger aggregate. If the root volume exists in a larger aggregate then as it is a flexible volume you have the opportunity to resize the volume to reclaim more usable space for other volumes. Below is a table showing Data ONTAP Recommended Root FlexVol Volume Size by Platform.

Platform	Root Volume size 7.3.x		Root Volume Size 8.0.x
FAS2040		16GiB		160GiB
FAS/V3040		16GiB		160GiB
FAS/V3140		16GiB		160GiB
FAS/V3070		23GiB		230GiB
FAS/V3160		23GiB		230GiB
FAS/V3170		25GiB		250GiB
FAS/V3210		10GiB		100GiB
FAS/V3240		15GiB		150GiB
FAS/V3270		30GiB		300GiB
FAS/V6000 series		25GiB		250GiB
FAS/V6200 series		Unsupported		300GiB

NetApp Best Practices

Below is a compiled list of the top recomendations and best practices when deploying a NetApp Filer.

• Use RAID-DP™, the NetApp high-performance implementation of RAID 6, for better data protection.
• Use multipath HA with active-active storage configurations to improve overall system availability as well as promote higher performance consistency.
• Use the default RAID group size when creating aggregates or traditional volumes.
• Allow Data ONTAP to select disks automatically when creating aggregates or volumes.
• Use the latest Data ONTAP general deployment release available on the NOW™ (NetApp on the Web) site.
• Use the latest storage controller, shelf, and disk firmware available on the NOW site.
• Maintain at least one hot spare for each type of disk drive in the storage system:
• Disk drive differences are FC, SAS, SATA disk drive types, disk size, and rotational speed (RPM).
• Maintain two hot spares for each type of disk drive in the storage system to take advantage of Maintenance Center.
• Reserve space for at least one week of Snapshot™ backups.
• Do not put user data into the root volume.
• Replicate data with SnapMirror® or SnapVault® for disaster recovery (DR) protection.
• Replicate to remote locations to increase data protection levels.
• Use an active-active storage controller configuration (clustered failover) to eliminate single points of failure (SPOFs).
• Deploy SyncMirror® and RAID-DP for the highest level of storage resiliency. 

Export NFS Shares From Command Line

Exporting NFS shares can be painful when you are trying to export the shares through the command line as the syntax's have to be 100% correct and i think altogether confusing so here is what i think is the simplest way to export NFS shares via the command line.

run rdfile /vol/vol0/etc/exports

The output you get should look something like this


#Auto-generated by setup Wed Oct 14 09:35:11 GMT 2012
/vol/vol0 -sec=sys,rw,anon=0,nosuid
/vol/vol0/home -sec=sys,rw,nosuid

To add in an export the easiest way is to write it directly to the file then run exportfs -a

So to add an NFS share for /vol/nfs_vol1 with Read, Write and Root access for host 1, host 2 and host 3 then add the following lines to the exports file

/vol/nfs_vol1     -sec=sys,rw

/vol/nfs_vol1      -sec=sys,rw,root=host1:host2:host3

So now you will need to write this to the exports file you can do this on the command line or by opening the exports file with wordpad to do this from the command line copy the old and new exports contents and  run wrfile /vol/vol0/etc/exports like below

wrfile /vol/vol0/etc/exports

#Auto-generated by setup Wed Oct 14 09:35:11 GMT 2012

/vol/vol0 -sec=sys,rw,anon=0,nosuid

/vol/vol0/home -sec=sys,rw,nosuid

/vol/nfs_vol1     -sec=sys,rw

/vol/nfs_vol1      -sec=sys,rw,root=host1:host2:host3

hit enter

then ctrl-c

the file will now have been updated so running wrfile /vol/vol0/etc/exports will now show

#Auto-generated by setup Wed Oct 14 09:35:11 GMT 2012

/vol/vol0 -sec=sys,rw,anon=0,nosuid

/vol/vol0/home -sec=sys,rw,nosuid

/vol/nfs_vol1     -sec=sys,rw

/vol/nfs_vol1      -sec=sys,rw,root=host1:host2:host3

Now all you will have to do is run the command exportfs -a and the share will have been exported and you will be able to mount the share on the host.

How does Snapmanager for Virtual infrastructure work

SMVI is now integrated into VSC which is good as most of the VMware integrated products are now centralised hopefully the rest of the snapmanger suite will soon be added. Here is how snapmanager for virtual infrastructure works

1. VSC initiates a backup
2. VSC triggers VMware snapshots of VMs
3. vCenter creates VM snapshots and captures active transactions in delta files
4. VSC triggers instant NetApp snapshot
5. VSC triggers VMware snapshot removal
6. vCenter removes consistent snapshots and reapplies delta files

SnapManager for SQL Sizing

SnapManager for SQL Server gives extended capabilities for SQL backups which include backing up and restoring SQL Server databases with minimum disruption to the database and minimising backup and restore times. These capabilities are provided by using the NetApp underlying snapshot technology which will quiecse the database and snapshot the data at a volume level so no data is actually copied the process is completed in a timely.

SnapManager for SQL Server includes a client-based graphical user interface (GUI) program to perform various activities on SQL Server. These activities include:

Setting options for SnapManager for SQL Server operations.

Backing up, restoring and cloning databases and database components.

Monitoring and Reporting SnapManager for SQL Server operations.

Limitations & Known issues

All SQL Server 2000, 2005, & 2008 Editions are supported, except Microsoft Desktop Edition (MDSE) 2000. And all versions of MYSQL

Pre-Requisites

• System database files cannot reside on the same LUN as user databases
• Each user database log files and data files need to be on separate LUNs
• Tempdb must have its own LUN or stay on Local disk
• A separate LUN needs to be created to store the Snapshots
• SQL must be installed on the server before installing Snapdrive and SMSQL
• ONLY database files can exist on the LUN. You’ll need to put all application binaries and non-database files on different LUNs.
• The client machine must be in DNS (whichever is appropriate), and be part of a Domain.
• A reboot of the client will be required, so a downtime slot must be arranged. This does not have to be at the same time as the installation, if necessary.
• Verify that the follow requirements are met for the database software.
• SQL Server is installed on Windows 2000 or later.
• Snapdrive must be installed prior to the install of Snap Manager for SQL

For further requirements please refer to the new SnapManager for SQL server 5.2 best practices - http://media.netapp.com/documents/tr-4003.pdf

Drive layouts

Mount points will be used instead of drive letters as using drive letters gives you a limit of 26 LUNs per server.

Each SQL server will be provisioned with the following LUN and volume layout:

System Databases will be migrated onto a separate LUN contained in a single volume

LUN Name –  systemdb_”server name”.lun

Volume Name – v_systemdb_”server name”

Qtree Name – q_systemdb_”server name”

Tempdb will be migrated onto a separate LUN contained in a single volume

LUN Name – tempdb_”server name”.lun

Volume Name – v_tempdb_”server name”

Qtree name – q_tempdb_”server name”

Each Database that is in need of a granular restore or is an intensively used database will need to placed on its own Data LUN along with its own Log LUN both of these LUNs will be contained in separate volumes

Database

LUN Name – “database name”_servername”.lun

Volume Name – v_”database name”_”server name”

Qtree Name – q_”databse name”_”server name”

Logs

LUN Name – “log name”_”server name”.lun

Volume Name – v_”log name”_”server name”

Qtree Name – q_”log name”_”server name”

For small databases that are not heavily utilised and are not in need of a granular restore, these can be grouped together and will be backed up together. NOTE – When a restore is performed on a single database in this configuration all databases will be restored, a restore should only be performed in a DR scenario.

Database

LUN Name – “db1”_servername”.lun

Volume Name – v_”db1”_”server name”

Qtree Name – q_”db1”_”server name”

Logs

LUN Name – “log1”_”server name”.lun

Volume Name – v_”log1”_”server name”

Qtree Name – q_”log1”_”server name”

A snapinfo LUN will also need to be created as this will hold a copy of the trans logs that have been backed up sizing for this LUN is shown in section 6.5.

LUN Name – snapinfo_”servername”.lun

Volume Name – v_snapinfo_”server name”

Qtree Name – q_snapinfo_”server name”

Database Volume Sizing

To find out what the size of the Database volume should be follow the procedure below

Database LUN size = Check with application owner or look on server

Database LUN change rate = 5%

Number of online snaps = 3

Database LUN Size x 5% = N

LUN Size + (N x 3) = volume size

For example - Note this is an example and will not ascertain the size of the volume you are creating

Database LUN size = 100Gb

100 x 5% = 5Gb = N

5Gb x 3 = 15Gb

Volume Size = 100Gb + (5gb x 3) = 115Gb

Log Volume Sizing

To find out what the size of the Log volume should be follow the procedure below

Log LUN size = Check with application owner or look on server

Log LUN change rate = 5%

Number of online snaps = 3

Log LUN Size x 5% = N

LUN Size + (N x 3) = volume size

For example - Note this is an example and will not ascertain the size of the volume you are creating

Log LUN size = 100Gb

100 x 5% = 5Gb = N

5Gb x 3 = 15Gb

Volume Size = 100Gb + (5gb x 3) = 115Gb

Snapinfo LUN Sizing

To make sure that we have adequate space available for our SLQ backups we must size the snapinfo volume and LUN. To do this run through the following steps for each SQL server:

Step 1

Run the below script on the database that is being moved

-- Create the table to accept the results

create table #output (dbname char(30),log_size real, usage real, status int)

-- execute the command, putting the results in the table

insert into #output

exec ('dbcc sqlperf(logspace)')

-- display the results

select *

from #output

go

--open cursor

declare output_cur cursor read_only for

select log_size

from #output

--make space computations

declare @v1 as real

declare @base_val as real

set @base_val = 0

open output_cur

FETCH NEXT FROM output_cur

INTO @v1

WHILE @@FETCH_STATUS = 0

BEGIN

set @base_val = @base_val + @v1

FETCH NEXT FROM output_cur

INTO @v1

END

set @base_val = @base_val + 15

PRINT 'BASE_VAL = ' + cast(@base_val as nvarchar) + ' MB'

--clean-up

close output_cur

drop table #output

BASE_VAL =  21.46906 MB

Step 2

Total number of transaction log files to be kept in the snapinfo LUN = 3

LUN Size = 21.4906 * 3 = 64MB

Snap info Volume Sizing

To calculate the size of the snapinfo volume follow the procedure below

Volume size = LUN Size + (Largest Trans log * 3)

64MB + (21.46906 * 3) = 128MB

How to create an aggregate via the command line

An aggregate is a collection of one or two plexes and it supports RAID-level mirroring. If the aggregate is un-mirrored, it contains a single plex. If the SyncMirror feature is licensed and enabled, a second plex can be added to any aggregate, which serves as a RAID-level mirror for the first plex in the aggregate.

aggr create <aggr_name> [-m] –r <raid_size> –t <raid_type> <total # disks>
snap reserve –A <aggr_name> <snap_reserve_pct>
snap sched –A <aggr_name> <snap_sched>

create a multi ifgroup

An Interface Group (ifgrp) is a mechanism that supports aggregation of network interfaces into one logical interface unit.  Once created, an ifgrp is indistinguishable from a physical network interface.  Ifgrps are used to provide fault tolerance of the network connection and in some cases higher throughput to the storage device.

ifgrp create multi ifgrp0 –b ip e5a e5b e5c e5d

ifconfig ifgroup0 192.168.0.25 netmask 255.255.255.0 partner ifgroup0

ifconfig e0M `hostname`-e0M flowcontrol full netmask 255.255.255.0

route add default 192.168.0.1 1

NSO-502 Brain dump

NSO-502 Brain dump

ISCSI

ISCSI security show – will show the current chap security settings

Igroups are mapped to portsets

Iscsi security show command shows current chap settings

Command to show iscsi digests enabled (iscsi session show –v)

To support an iscsi multipath config you need to have 2 vmkernels

Vlans are recommended for iscsi and nfs and cifs traffic

Command to check if interface is enabled for iscsi – iscsi interface show

Iscsi interface show shows interfaces enabled for iscsi

Switch/Ethernet

E_Port = Inter Switch link

L_Port is a fc hub to a fc switch

Cisco

Show version – tells you the firmware version on cisco switch

Show zoneset – will show the zone config information

The Web Tools GUI is used to manage CISCO MDS servers

Show zone active will show the active zone on a cisco switch

Showversion on a ciso switch will tell you firmware versions and other information

Cisco switch – show firmware (Shows the firmware of a cisco switch)

Show cisco switch zone config (show zoneset)

Brocade

Fabricshow on a brocade fc switch will show you what devices are connected to the fabric

Web tools manages brocade switches

Supportshow will show diags on a brocade switch

Brocade fabricshow

Brocade supportshow

Zoning

Netapp recommends wwpn zoning

FC

Single_image clusters are only supported with Ontap 7.1 and up.

An FCP license is needed for Snapdrive

Hard Zoning is done at the ASIC level

Cfgshow displays the config on a cisco switch

Fcp stat –I 5

Having a single_image cfmode will increase the amount paths to a lun and will also change the wwpn on a host (As they share a wwpn)

HBYAnywhere is used to find information about emulux HBA’s

Standby mode is only supported by windows and solaris hosts

Highly available fc switch config (2 fc switches minimum)

Highly available switch config (2 hba’s minimum)

Fcp connectivity diagram port targets in it, targets, fc switch ports, wwn’s, hosts wwn’s

Advantage of software zoning is that you can plug a cable anywhere in the switch and is the same between vendors

Current 4gb fibre channel switch and want to upgrade to 8gb just need to upgrade the 4gb switches to 8gb

Fas6030 and 6070 have 8 x 2gb onboard fc ports

Fas6040 and 6080 have 8 x 4gb onboard fc ports

Snapdrive

An FCP license is needed for Snapdrive

31.5Mb is the smallest lun that can be created using Snapdrive

Fcp license needed for Snapdrive

Best practice for Snapdrive is to create a user account with admin privileges

Volumes

Luns

Boot volumes need persistant bindings

Sanlun show –p will show you on a solaris machine how many paths are available

Lun stats –I 5 will show you the throughput of a netapp lun

To share the same lun between hosts you need a host based clustered file system like VMFS

31.5Mb is the smallest lun that can be created using Snapdrive

Igroups are mapped to portsets

Solaris error messages are logged in the following location /var/adm/messages

There is no difference between an fc lun and an iscsi lun

Copy on write – 1 read 2 writes

Lun stats –I 5

Portsets bound to igroups

DSM for windows controls fc and iscsi paths at the same time

Cfgshow

Show zone set active

With an existing emc setup check the lun layout

Things to document with an existing netapp setup

• Snapshot config
• Protocols in use
• Igroups being used
• Disk ownership settings

Which doc would show the cab diagram (cabinet diagram)

100gb lun

10% data change a day

Snap every night

7 snaps

10% of 100gb = 10gb x 7 = 70gb

100% fractional reserve = 100gb

100gb lun = 100gb + fractional reserve  + 70gb = 270gb

Fractional reserve = 0%

Snap reserve = 0%

3 x 100gb luns

No snaps

3 x 100gb = 300gb

Gurantee = volume

3 x 100gb luns = 300gb

50% fractional reserve = 150gb

2 snaps per week keep 2 weeks

10% data change = 30gb x 2 = 60gb x 2 = 120gb

300gb + 150gb + 120gb = 570gb

The native gui to manage brocade switches is web tools

The best way to test a storage failover is to perform a storage takeover on each controller

RDM’s can be used with NPIV

Tools to show  wwn;s and wwpn’s – sanlun fcp show adapter, esxcfg-info

When using dsm diable alua on igroups

Storage type plugins – VM_stap_ALUA, MVW_SATP_Default_AA

Fc supported hop count

Brocade 6

Cisco 5

McData 3

Qlogic 4

With aix to discover newly created luns use – cfgmgr

Lesdev verify hdisk mpio

Lsattr info about setup

Sanlun lun show –p info about setup

DSM Device specific module

TimeOutValue = how long i/o requests are held at iscsi layer before time out

Request windows hot fixes from the microsft website

Snapdrive needs to be installed before DSM

MaxRequestHoldTime

Disable Microsoft mpio if Snapdrive mpio is enabled

Host utilities kit must be enabled in a cluster

Initiator node names cannot have _ in the name

DSM set path verify enabled set to 0

Qlogic host bus adapterLinkDownTimeOut set to 5

MaxRequestHoldTime = time the initiator retries failed i/o requests when a path to a target is lost

Initiator – disk shelf, host

Target – tape drive, netapp controller

You would find an initiator on an esx host

LUN ID is not a HBA characteristic

SCSI command is encapsulated into FC, ISCSI and FCoE protocols.

Target is connected via onboard fc ports and add in target HBA cards

Lun masking = igroups

Isns provides a list of target initiators

Sessions are related to target portal groups connections are related to interfaces

Lun show –m shows lun maps to igroups

Lossless Ethernet DCB data center bridging alias CEE converged enhanced Ethernet

FCoE includes pfc priority based flow control, ETS Enhanced Transmission Selection, CN Congestion Notification.

802.1Qbb (PFC)

8021Qaz (ETS)

802.1Qau (CN)

FCoE card is called a UTA Unified Target Adapter

OM-3 multimode 62.5 micron fc cables with SFP connectors

DSM controllers fc & Iscsi

Layer 1 of ISO stack = Physical Layer

Jumbo frames can increase performance at 1gb but essential for 10Gb

Switching functions are at the layer 2 level

You can have nics at different speed in a VIF

Switching functions are at layer 2

Training for Exam NS0-501:

• Data ONTAP Fundamentals (DOTF)
• NetApp Protection Software Administration (NPSA)
• Data ONTAP SAN Administration (SAN)
• NetApp SnapManager 5.0 for Microsoft Exchange (SME)
• Microsoft SQL Server 2005 on NetApp Storage Systems (MSSQL)

Training for Exam NSO-510:

• Data ONTAP Fundamentals (DOTF)
• NetApp Protection Software Administration (NPSA)

Training for Exam NSO-520:

• Data ONTAP Fundamentals (DOTF)
• NetApp Protection Software Administration (NPSA)

Training for Exam NSO-530:

• Data ONTAP Fundamentals (DOTF)
• DataFort Storage Security Administration (DSSA)