You might need to tune
your FC queue depth on the host to achieve the maximum values for hosts per
controller and FC port fan-in. The maximum number of LUNs and the number of
HBAs that can connect to an FC port are limited by the available queue depth on
the FC target ports.
About this task
Queue depth is the
number of I/O requests (SCSI commands) that can be queued at one time on a
storage controller. Each I/O request from the host's initiator HBA to the
storage controller's target adapter consumes a queue entry. Typically, a higher
queue depth equates to better performance. However, if the storage controller's
maximum queue depth is reached, that storage controller rejects incoming
commands by returning a QFULL response to them. If a large number of hosts are
accessing a storage controller, plan carefully to avoid QFULL conditions, which
significantly degrade system performance and can lead to errors on some
systems.
In a configuration with
multiple initiators (hosts), all hosts should have similar queue depths. This
prevents hosts with small queue depths from being starved by hosts with large
queue depths.
The following general
recommendations can be made about "tuning" queue depths.
- For small to mid-size systems, use a HBA queue depth of
32.
- For large systems, use a HBA queue depth of 128.
- For exception cases or performance testing, use a queue
depth of 256 to avoid possible queuing problems.
- All hosts should have the queue depths set to similar
values to give equal access to all hosts.
- Ensure
that the storage controller target FC port queue depth is not exceeded to
avoid performance penalties or errors.
Steps
- Count the total number of FC initiators in all the
hosts that connect to one FC target port.
- Multiply by 128.
- If the result is less than 2,048, set the queue depth
for all initiators to 128.
Example
You
have 15 hosts with one initiator connected to each of two target ports on the
storage controller. 15 x 128 = 1,920. 1,920 is less than the total queue depth
limit of 2,048, so you can set the queue depth for all your initiators to 128.
- If the result is greater than 2,048, go to step 3.
Example
You
have 30 hosts with one initiator connected to each of two target ports on the
storage controller. 30 x 128 = 3,840. 3,840 is greater than the total queue
depth limit of 2,048, so you should choose one of the options under step 3 for
remediation.
- Choose one of the following options.
- Option 1:
- Add more FC target ports.
- Redistribute your FC initiators.
- Repeat steps 1 and 2.
Example
The
desired queue depth of 3,840 exceeds the available queue depth per port. To
remedy this, you can add a two-port FC target adapter to each controller, then
rezone your FC switches so that 15 of your 30 hosts connect to one set of
ports, and the remaining 15 hosts connect to a second set of ports. The queue
depth per port is then reduced to 15 x 128 = 1,920.
- Option 2:
- Designate each host as "large" or
"small" based on its expected I/O need.
- Multiply the number of large initiators by 128.
- Multiply the number of small initiators by 32.
- Add the two results together.
- If the result is less than 2,048, set the queue depth
for "large" host to 128 and the queue depth for
"small" host to 32.
- If the result is still greater than 2,048 per port,
reduce the queue depth per initiator until the total queue depth is less
than or equal to 2,048.
Note: To estimate the
queue depth needed to achieve a certain I/O per second throughput, use this
formula.
Needed queue depth = (Number of I/O per second) x (Response time)
For example, if you need 40,000 I/O per second with a response
time of 3 milliseconds, the needed queue depth = 40,000 x (.003) = 120.
Example
The
desired queue depth of 3,840 exceeds the available queue depth per port. You
have 10 "large" hosts that have high storage I/O needs, and 20
"small" hosts that have low I/O needs. Set the initiator queue depth
on the "large" hosts to 128 and the initiator queue depth on the
"small" hosts to 32.
Your resulting total queue depth is (10 x 128) + (20 x 32) =
1,920.
Example
You
can spread the available queue depth equally across each initiator.
The
Packet VLAN is used by protocols such as CDP, LACP, and IGMP.
VSM
configuration commands to each VEM, and their responses