Manual intervention required when adding new node to cluster with QDSKVOTES gt 1
(too old to reply)
Jon Pinkley
2021-04-22 00:19:26 UTC
This is not a question, only a comment and hint for other people adding
a node to a cluster with a quorum disk with more than a single vote.

If you are using a quorum disk, and QDSKVOTES is greater than 1,
CLUSTER_CONFIG_LAN.COM will not preserve the value when it creates its
initial IA64VMSSYS.PAR file for the new system. Instead it will use the
value 1.

This will result in the new node not being allowed into the cluster, and
you will be forced to crash the node attempting to boot.

The easiest solution is to use a conversational boot for the first boot
and change QDSKVOTES to the correct value, and do not set WRITESYSPARAMS
to 0 after the change, so the new value will be written back to the
CURRENT params.

In case there are questions about why use a quorum disk or why use
QDSKVOTES greater than 1, the short answer is so you can have the
cluster survive with a single node up, without the need to manually
adjust quorum.

So, if you have x active boot nodes, you set VOTES to 1 on all active
nodes, QDSKVOTES to x-1, and EXPECTED_VOTES to 2x-1. Then the cluster
will maintain quorum and not hang, while the quorum disk is available
and at least one active boot node is running.

Volker Halle
2021-04-22 06:42:43 UTC

the same is true for CLUSTER_CONFIG.COM

There are 3 lines in both procedures, which explicity set QDSKVOTES:

$WRITE tempfile1 "QDSKVOTES=1"
$WRITE tempfile1 "QDSKVOTES=1"
$WRITE tempfile1 "SET QDSKVOTES 1"

A cluster with QDSKVOTES > 1 may be a rare configuration, but it still should be handled correctly. Also watch out for QDSKVOTES in PARAMS.DAT or MODPARAMS.DAT, which could 'bite' you after the next AUTOGEN.

If you have a support contract with VSI, please let them know. If you don't have a support contract, sending a mail to VSI support wouldn't hurt - remember SPRs ?

Volker Halle
2021-04-22 09:02:36 UTC

an even easier workaround than fiddling with a conversational boot would be (after executing @CLUSTER_CONFIG[_LAN]):

SYSGEN> USE dev:<SYSx.SYSEXE>IA64VMSSYS.PAR ! newly created params in new system root
SYSGEN> SET . <correct-value-of-QDSKVOTES>

CLUSTER_CONFIG[_LAN] already asks for the quorum disk and uses the current value of DISK_QUORUM as the default value. It should do the SAME with QDSKVOTES, i.e.

What is the value of the Quorum Disk votes [<current_value_of_QDSKVOTES>] ?

Stephen Hoffman
2021-04-22 18:03:48 UTC
Post by Jon Pinkley
So, if you have x active boot nodes, you set VOTES to 1 on all active
nodes, QDSKVOTES to x-1, and EXPECTED_VOTES to 2x-1.
The running value of quorum is dynamically determined.

Irrespective of EXPECTED_VOTES settings, the quorum value will float to
the detected number of VOTES and QDSKVOTES present.

Setting EXPECTED_VOTES low can open opportunities for corruptions
particularly with multi-path storage present, though.
Pure Personal Opinion | HoffmanLabs LLC