Discussion:
OpenVMS STARTUP Whitepaper
(too old to reply)
geze...@rlgsc.com
2020-12-03 15:56:42 UTC
Permalink
I have posted a whitepaper on OpenVMS STARTUP, discussing the capabilities of STARTUP.COM and its related files.

The preliminary copy of the paper is at http://www.rlgsc.com/publications/openvmsstartupunderappreciatedflexibility.html.

- Bob Gezelter, http://www.rlgsc.com
geze...@rlgsc.com
2020-12-03 18:09:25 UTC
Permalink
Abstract:

The OpenVMS STARTUP process is documented, but the information is spread across several different manuals and files. This leads to a fragmentation of knowledge and the propagation of out-of-date and apocryphal information. A close reading of the documentation and associated files yields a coherent picture of the STARTUP facility and how it can be used to improve time-to restart.

- Bob Gezelter, http://www.rlgsc.com
Stephen Hoffman
2020-12-04 16:12:34 UTC
Permalink
Post by ***@rlgsc.com
The OpenVMS STARTUP process is documented, but the information is
spread across several different manuals and files...
Welcome to OpenVMS, where the doc is commonly and unfortunately quite
scattershot and in need of an overhaul, and where the cookbook-style
presentation is lacking.

Section 11 is missing the "why" for the items listed. That detail might
be obvious to the writer, but not to the reader. Some New Features
documentation is notorious for this—here's this feature, but omitting
why you might want to adopt it.

Not much on SYSMAN in that whitepaper, that being an approach you've
oft preferred per our previous discussions, and an approach I've become
fond of when seeking to "hide" startups.

And as for the previous discussions here, this document seems thin in
the relevant section for that discussion; section 11. What can be fixed
or updated, what's missing. Better app packaging and app isolation and
automated means to add and remove app startups has been an ongoing
mess. Most apps scatter at least part of themselves into various spots
on an OpenVMS boot device, often requiring manual steps by the system
manager / app developer / app support team / Jada, the person on staff
that keeps that old server going. And the creation of documentation for
the end-user, since this presently can't be reliably automated within
OpenVMS.

As I've commented previously from the earlier thread, hand-editing a
file to add a startup is archaic and error-prone and somewhere between
silly and stupid, but that's on OpenVMS and not on you. And manually
invoking SYSMAN to add a startup isn't appreciably better, and that
approach is documented and used used in approximately no layered
products. And manually removing those same entries, if/when necessary.
And whatever else the installer added. PCSI tries, but there are holes
there.

And as for cookbook documentation more generally, background and
internals are prolly best left in an appendix at most (e.g. first 10
sections, here), with a discussion of options and alternatives for
developers as the focus. How an end-user or a layered product might use
the startups, and what options and alternatives exist. Because I'd
really rather not have to know how the OpenVMS startups work, outside
of the minimum necessary for an app developer or installation kit
author or suchlike to add and to remove the app and the startup.

And as for passive voice in technical documentation, I prefer to avoid
passive voice in cases where active voice fits. Which is usually.
Passive voice is best left for assuaging for or deflecting after some
screw-up, for cover-ups and related backside-covering, or variously as
part of any efforts toward avoiding having or avoiding stating an
opinion. Put differently, encountering passive voice can be inferred to
indicate you're dealing with somebody trying to hide something. One can
of course have other opinions for oneself, as one often does.
--
Pure Personal Opinion | HoffmanLabs LLC
Phillip Helbig (undress to reply)
2020-12-04 16:43:13 UTC
Permalink
Post by Stephen Hoffman
Passive voice is best left for assuaging for or deflecting after some
screw-up, for cover-ups and related backside-covering, or variously as
part of any efforts toward avoiding having or avoiding stating an
opinion.
Or for writing sentences like that above. :-D
geze...@rlgsc.com
2020-12-05 20:29:25 UTC
Permalink
Post by Stephen Hoffman
Post by ***@rlgsc.com
The OpenVMS STARTUP process is documented, but the information is
spread across several different manuals and files...
Welcome to OpenVMS, where the doc is commonly and unfortunately quite
scattershot and in need of an overhaul, and where the cookbook-style
presentation is lacking.
Section 11 is missing the "why" for the items listed. That detail might
be obvious to the writer, but not to the reader. Some New Features
documentation is notorious for this—here's this feature, but omitting
why you might want to adopt it.
Not much on SYSMAN in that whitepaper, that being an approach you've
oft preferred per our previous discussions, and an approach I've become
fond of when seeking to "hide" startups.
And as for the previous discussions here, this document seems thin in
the relevant section for that discussion; section 11. What can be fixed
or updated, what's missing. Better app packaging and app isolation and
automated means to add and remove app startups has been an ongoing
mess. Most apps scatter at least part of themselves into various spots
on an OpenVMS boot device, often requiring manual steps by the system
manager / app developer / app support team / Jada, the person on staff
that keeps that old server going. And the creation of documentation for
the end-user, since this presently can't be reliably automated within
OpenVMS.
As I've commented previously from the earlier thread, hand-editing a
file to add a startup is archaic and error-prone and somewhere between
silly and stupid, but that's on OpenVMS and not on you. And manually
invoking SYSMAN to add a startup isn't appreciably better, and that
approach is documented and used used in approximately no layered
products. And manually removing those same entries, if/when necessary.
And whatever else the installer added. PCSI tries, but there are holes
there.
And as for cookbook documentation more generally, background and
internals are prolly best left in an appendix at most (e.g. first 10
sections, here), with a discussion of options and alternatives for
developers as the focus. How an end-user or a layered product might use
the startups, and what options and alternatives exist. Because I'd
really rather not have to know how the OpenVMS startups work, outside
of the minimum necessary for an app developer or installation kit
author or suchlike to add and to remove the app and the startup.
And as for passive voice in technical documentation, I prefer to avoid
passive voice in cases where active voice fits. Which is usually.
Passive voice is best left for assuaging for or deflecting after some
screw-up, for cover-ups and related backside-covering, or variously as
part of any efforts toward avoiding having or avoiding stating an
opinion. Put differently, encountering passive voice can be inferred to
indicate you're dealing with somebody trying to hide something. One can
of course have other opinions for oneself, as one often does.
--
Pure Personal Opinion | HoffmanLabs LLC
Hoff,

Thank you for taking the time to read my whitepaper.

I appreciate your comments. However, they appear to misconstrue my intent.

The whitepaper was written to be descriptive of the present implementation. Since it is descriptive, passive voice seemed most appropriate, which is admittedly a matter of taste. Section 11 (Improvements), is not meant to be comprehensive. It is a list of quirks in the current implementation which limit the existing implementation, IMHO, needlessly. The first item listed, the requirement that STARTUP.COM be run from SYSTEM, is a complete non-sequitur. Nothing done by STARTUP.COM itself is specific to SYSTEM. Some of the actions in the standard STARTUP databases require privileges, but any account with suitable privileges/quotas would be acceptable. The same appears to apply to the use of Executive mode logical names. These two items appear to be privilege requirements in excess of that which is actually required.

I have received comments privately questioning why I did not include a discussion of the bootstrap and activities done by SWAPPER. Once again, that was not the purpose of this whitepaper. The bootstrap sequence and creation of the initial processes is adequately covered by the various versions of the IDSM, and need not be repeated.

It was not my intent to redesign the STARTUP.COM processing. That is a project with a far wider scope, with many subtleties.

As I have said in the past, STARTUP.COM can be used to more effect than is common. This is a multi-level failure, from unneeded implementation restrictions to less than adequate documentation. However, even modest changes in code and usage would have beneficial effects. The shortcomings I mentioned in Section 11 are all correctable with a few lines of changes, not a major engineering effort.

- Bob Gezelter, http://www.rlgsc.com
Stephen Hoffman
2020-12-05 21:19:09 UTC
Permalink
Post by ***@rlgsc.com
Section 11 (Improvements), is not meant to be comprehensive.
I clearly misinterpreted your earlier replies about this whitepaper,
then. This material is not what I was expecting.

From that earlier thread... App startup-related processing, PCSI and
VMSINSTAL installations and the ever-problematic app removal
requirements, app security and app signing and app isolation, the whole
area needs an overhaul and substantial updates. There's no good way for
an app developer to offload any of this from the end-user, and the
expectations that somebody will need to manually complete an app
install is archaic at best. Each layered product ends up with a
different and generic sequence describing how to edit
SYSTARTUP_VMS.COM, which may or may not be applicable locally. And the
SYSMAN STARTUP hooks for installers got nowhere. It's all just
hilarious. And sad.

Avoid passive voice in tech doc.
--
Pure Personal Opinion | HoffmanLabs LLC
Edgar Ulloa
2020-12-08 02:49:19 UTC
Permalink
Post by ***@rlgsc.com
I have posted a whitepaper on OpenVMS STARTUP, discussing the capabilities of STARTUP.COM and its related files.
The preliminary copy of the paper is at http://www.rlgsc.com/publications/openvmsstartupunderappreciatedflexibility.html.
- Bob Gezelter, http://www.rlgsc.com
Bob it is a very complete writing, I thank you very much for the time you invest in this type of research and share it with the community. I appreciate it.

very thankful
geze...@rlgsc.com
2020-12-08 16:10:10 UTC
Permalink
Post by ***@rlgsc.com
I have posted a whitepaper on OpenVMS STARTUP, discussing the capabilities of STARTUP.COM and its related files.
The preliminary copy of the paper is at http://www.rlgsc.com/publications/openvmsstartupunderappreciatedflexibility.html.
- Bob Gezelter, http://www.rlgsc.com
Bob it is a very complete writing, I thank you very much for the time you invest in this type of research and share it with the community. I appreciate it.
very thankful
Edgar,

You are welcome. My pleasure.

Since I posted the paper. I identified several errata. I corrected them and posted a revised copy (Revision 2020-12-06).

- Bob Gezelter, http://www.rlgsc.com
Dave Froble
2020-12-08 22:45:45 UTC
Permalink
Post by ***@rlgsc.com
Post by ***@rlgsc.com
I have posted a whitepaper on OpenVMS STARTUP, discussing the capabilities of STARTUP.COM and its related files.
The preliminary copy of the paper is at http://www.rlgsc.com/publications/openvmsstartupunderappreciatedflexibility.html.
- Bob Gezelter, http://www.rlgsc.com
Bob it is a very complete writing, I thank you very much for the time you invest in this type of research and share it with the community. I appreciate it.
very thankful
Edgar,
You are welcome. My pleasure.
Since I posted the paper. I identified several errata. I corrected them and posted a revised copy (Revision 2020-12-06).
- Bob Gezelter, http://www.rlgsc.com
"Errata"?

Is that the replacement word for "mistake"?

:-)
--
David Froble Tel: 724-529-0450
Dave Froble Enterprises, Inc. E-Mail: ***@tsoft-inc.com
DFE Ultralights, Inc.
170 Grimplin Road
Vanderbilt, PA 15486
geze...@rlgsc.com
2020-12-09 00:34:27 UTC
Permalink
Post by Dave Froble
Post by ***@rlgsc.com
Post by ***@rlgsc.com
I have posted a whitepaper on OpenVMS STARTUP, discussing the capabilities of STARTUP.COM and its related files.
The preliminary copy of the paper is at http://www.rlgsc.com/publications/openvmsstartupunderappreciatedflexibility.html.
- Bob Gezelter, http://www.rlgsc.com
Bob it is a very complete writing, I thank you very much for the time you invest in this type of research and share it with the community. I appreciate it.
very thankful
Edgar,
You are welcome. My pleasure.
Since I posted the paper. I identified several errata. I corrected them and posted a revised copy (Revision 2020-12-06).
- Bob Gezelter, http://www.rlgsc.com
"Errata"?
Is that the replacement word for "mistake"?
:-)
--
David Froble Tel: 724-529-0450
DFE Ultralights, Inc.
170 Grimplin Road
Vanderbilt, PA 15486
David,

"Mistake" is singular. Errata is the plural of erratum.

- Bob Gezelter, http://www.rlgsc.com
Dave Froble
2020-12-09 05:06:37 UTC
Permalink
Post by ***@rlgsc.com
Post by Dave Froble
Post by ***@rlgsc.com
Post by ***@rlgsc.com
I have posted a whitepaper on OpenVMS STARTUP, discussing the capabilities of STARTUP.COM and its related files.
The preliminary copy of the paper is at http://www.rlgsc.com/publications/openvmsstartupunderappreciatedflexibility.html.
- Bob Gezelter, http://www.rlgsc.com
Bob it is a very complete writing, I thank you very much for the time you invest in this type of research and share it with the community. I appreciate it.
very thankful
Edgar,
You are welcome. My pleasure.
Since I posted the paper. I identified several errata. I corrected them and posted a revised copy (Revision 2020-12-06).
- Bob Gezelter, http://www.rlgsc.com
"Errata"?
Is that the replacement word for "mistake"?
:-)
--
David Froble Tel: 724-529-0450
DFE Ultralights, Inc.
170 Grimplin Road
Vanderbilt, PA 15486
David,
"Mistake" is singular. Errata is the plural of erratum.
- Bob Gezelter, http://www.rlgsc.com
Ok, then, "mistakes" ...

Since you took the time to write, I decided I should at least devote the
time to read. It is not something that most users will need to know
about, but, knowledge is usually a good thing.

Overall, a nice description of the processing.

One thing I noticed by it's omission. That is the admonishment to
strictly stay out of the VMS startup procedures as much as possible. It
is my firm belief that applications and users should have a different
place than the OS startup. At most the system startup could invoke user
and application startups.

As an example, I use a separate batch job to start up DECnet, LAT, and
TCP/IP. I also use separate command files to startup applications.
Prior to this, it is possible for an operator login on OPA0, should that
be required.

Just my opinion, namely, "stay out of the OS stuff".
--
David Froble Tel: 724-529-0450
Dave Froble Enterprises, Inc. E-Mail: ***@tsoft-inc.com
DFE Ultralights, Inc.
170 Grimplin Road
Vanderbilt, PA 15486
geze...@rlgsc.com
2020-12-09 16:44:55 UTC
Permalink
Post by Dave Froble
Post by ***@rlgsc.com
Post by Dave Froble
Post by ***@rlgsc.com
Post by ***@rlgsc.com
I have posted a whitepaper on OpenVMS STARTUP, discussing the capabilities of STARTUP.COM and its related files.
The preliminary copy of the paper is at http://www.rlgsc.com/publications/openvmsstartupunderappreciatedflexibility.html.
- Bob Gezelter, http://www.rlgsc.com
Bob it is a very complete writing, I thank you very much for the time you invest in this type of research and share it with the community. I appreciate it.
very thankful
Edgar,
You are welcome. My pleasure.
Since I posted the paper. I identified several errata. I corrected them and posted a revised copy (Revision 2020-12-06).
- Bob Gezelter, http://www.rlgsc.com
"Errata"?
Is that the replacement word for "mistake"?
:-)
--
David Froble Tel: 724-529-0450
DFE Ultralights, Inc.
170 Grimplin Road
Vanderbilt, PA 15486
David,
"Mistake" is singular. Errata is the plural of erratum.
- Bob Gezelter, http://www.rlgsc.com
Ok, then, "mistakes" ...
Since you took the time to write, I decided I should at least devote the
time to read. It is not something that most users will need to know
about, but, knowledge is usually a good thing.
Overall, a nice description of the processing.
One thing I noticed by it's omission. That is the admonishment to
strictly stay out of the VMS startup procedures as much as possible. It
is my firm belief that applications and users should have a different
place than the OS startup. At most the system startup could invoke user
and application startups.
As an example, I use a separate batch job to start up DECnet, LAT, and
TCP/IP. I also use separate command files to startup applications.
Prior to this, it is possible for an operator login on OPA0, should that
be required.
Just my opinion, namely, "stay out of the OS stuff".
--
David Froble Tel: 724-529-0450
DFE Ultralights, Inc.
170 Grimplin Road
Vanderbilt, PA 15486
David,

Thank you for taking the time to read the whitepaper.

Your question is a good one. The "admonishment to strictly stay out of the VMS startup procedures as much as possible" is omitted because with the exception of VMS$VMS.DAT, the facilities are deliberately designed and implemented to be used by layered product, ISVs, and end-user administrators. Were that not so, file names would be hardwired and SYSMAN would not have the commands to manipulate the data contained in the databases. As the paper noted, adding components to be part of the startup sequence is documented, although the documentation is not as clear as it could be.

Documented facilities used within their design intent are always permissible. The facilities of STARTUP.COM and SYSMAN are as much a part of supported OpenVMS as SYS$QIO, SYS$WAIT, and other system services.

Using LAT and TCP/IP as examples, the appropriate startup files are easily added to VMS$LAYERED.DAT, so long as the prerequisites are satisfied. In this particular case, since DECnet changes the MAC address, DECnet (Phase IV or Phase V) must be started before LAT and TCP/IP. There is no dependency relationship between LAT and TCP/IP, so they can be started during the same phase. Thus, I add LAT$STARTUP.COM and the appropriate TCP startup file as SPAWN mode after DECnet has been started. NET$STARTUP.COM is invoked fairly early in the STARTUP sequence.

STARTUP.COM could automatically submit batch jobs under SYSTEM, but I generally prefer SPAWN, since a batch job could get hung.

Similarly, ENABLING/DISABLING entries in the Startup database is simple, using the SYSMAN STARTUP ENABLE/DISABLE commands. Far less error prone than manual editing.

My preference is to avoid manual intervention during system/application startup. Manual intervention is far too often a source of problems.

- Bob Gezelter, http://www.rlgsc.com
David Jones
2020-12-09 00:37:16 UTC
Permalink
Post by ***@rlgsc.com
I have posted a whitepaper on OpenVMS STARTUP, discussing the capabilities of STARTUP.COM and its related files.
I've been gradually migrating my hobbyist cluster towards having a sequencer, as
the whitepaper calls it, control the startup. It's driven by a database with two
main tables and an ancillary table from which I define 2 views for each node
in the cluster. The schemas for node 'PSXP1' are thus:

sqlite> .schema PSXP1_boot*
CREATE VIEW PSXP1_boot AS
SELECT name,step,inv_type,command,param,description FROM component
WHERE name IN (SELECT component FROM startup
WHERE node_group IN ('*','PSXP1'))
/* PSXP1_boot(name,step,inv_type,command,param,description) */;
CREATE VIEW PSXP1_boot_dep as SELECT component,prereq FROM dependencies
WHERE component in (SELECT name FROM PSXP1_boot);
/* PSXP1_boot_dep(component,prereq) */

The view rows are used by the startup driver (sequencer) to build a boot list
of commands to spawn. The sequencing of the spawns then happens at 2 levels:

1. All components with a higher step priority must complete before a component
may start ('step' is equivalent to 'phase' in the OpenVMS startup driver).
Components with the same step value may run concurrently, subject to a
limit parameter imposed by the sequencer (default 2).

2. Components with rows in the dependencies view are blocked from starting
until all their prerequisite components have completed.

I haven't decided the best way to handle the case where the prereq component is in
the master component table but inadvertantly not in the view for the node. Do you
omit the component, ignore the dependency, or automatically pull in the needed
component at the same step priority?
Stephen Hoffman
2020-12-11 01:57:12 UTC
Permalink
... haven't decided the best way to handle the case where the prereq
component is in the master component table but inadvertantly not in the
view for the node. Do you omit the component, ignore the dependency, or
automatically pull in the needed component at the same step priority?
For those interested in the topic of startups and startup sequencing
and dependency analysis, here's a technical introduction to resolving
dependencies while also detecting dependency deadlocks:

https://www.electricmonk.nl/docs/dependency_resolving_algorithm/dependency_resolving_algorithm.html


SQLite can be useful for storing the data, particularly for those
without Oracle licenses and that for whatever reason.
--
Pure Personal Opinion | HoffmanLabs LLC
David Jones
2020-12-11 05:28:30 UTC
Permalink
Post by Stephen Hoffman
SQLite can be useful for storing the data, particularly for those
without Oracle licenses and that for whatever reason.
Using Oracle in your startup sequencer means you have to start it before you can run
the thing that starts it -- somewhat problematic. A cloud-based database still needs the
network running. I link the startup driver application statically with the sqlite3 library
and run it with the data file flagged as immutable (readonly, no locking).
Craig A. Berry
2020-12-11 12:55:04 UTC
Permalink
Post by David Jones
Post by Stephen Hoffman
SQLite can be useful for storing the data, particularly for those
without Oracle licenses and that for whatever reason.
Using Oracle in your startup sequencer means you have to start it before you can run
the thing that starts it -- somewhat problematic. A cloud-based database still needs the
network running. I link the startup driver application statically with the sqlite3 library
and run it with the data file flagged as immutable (readonly, no locking).
I believe in the case of Rdb there is no start-up because there is no
service -- you just link your code against the libraries and run it in
the context of whatever process. In other words, the same way you are
running SQLite.
Jan-Erik Söderholm
2020-12-11 16:12:18 UTC
Permalink
Post by Craig A. Berry
Post by David Jones
Post by Stephen Hoffman
SQLite can be useful for storing the data, particularly for those
without Oracle licenses and that for whatever reason.
Using Oracle in your startup sequencer means you have to start it before you can run
the thing that starts it -- somewhat problematic. A cloud-based database
still needs the
network running. I link the startup driver application statically with
the sqlite3 library
and run it with the data file flagged as immutable (readonly, no locking).
I believe in the case of Rdb there is no start-up because there is no
service -- you just link your code against the libraries and run it in
the context of whatever process.  In other words, the same way you are
running SQLite.
Rdb use file access (unless using the using one of the remote network
protocols that rely SQL service or a thin JDBC driver server).
SQL/Services is for the SQL/Services API (as used by ODBC drivers),
JDBC from Java based clients or OCI from clients that thinks they are
accessing an Oracle Classic database.

Then you can run "RDBSERVER" using it's own protocoll. That s mainly for
VMS-2-VMS access such as when you want the single-node optimazitions
within a cluster and having the other cluster nodes access Rdb remotely
instead of direct shared access (involves more cross-cluster locking).
Doesn't have to be a cluster, of course.

If you are running DECnet, you can specify a remote file (just as when
using FAL) in the attach and the Rdb code will detect that and connect
the the remote RDBSERVER object without the application having to care.
(I thinkthat also works for TCPIP, do not use this).

$ tcpip sh service rdbserver

Service Port Proto Process Address State
RDBSERVER 611 TCP RDB 0.0.0.0 Enabled

$ ncp show object rdbserver

Object Number File/PID User Id Password
RDBSERVER 35 RDBSERVER.COM RDB$REMOTE <pw>
$

Anyway, RDBSERVER has nothing to do with SQL/Services that uses
a different TCPIP port and another DECnet obbject.

But yes, you can call RDBREMOTE for "file access", just as you can
with FAL, since the application doesn't care.
But will Rdb work properly without the RDMS_MONITOR and RMU processes running?
No, your application will not do anything with the data without having
an "OK" from the monitor. You send an "attach request" (well, the code
in the sharable images does) to the monitor and the monitor says "OK".

It also logs this like:


3-AUG-2020 09:46:57.56 - Received user attach request from 0018F182:1
- process name <process>, user <user>
- image name "$1$DGA3610:<dir.dir><EXE file>"
- database name "$1$DGA3620:[dir]<*.RDB file"
- 20 global buffers allocated; 14918 free out of 20000
- sending normal user attach reply to 0018F182:1

Then the attach either ends normaly like:

3-AUG-2020 12:46:48.37 - Received user attach request from 0018F182:1
- process name <process>, user <user>
- image name "$1$DGA3610:<EXE file>"
- database name "$1$DGA3620:[dir]<.RDB file>
- 20 global buffers allocated; 14558 free out of 20000
- sending normal user attach reply to 0018F182:1

3-AUG-2020 13:01:24.30 - Received user image termination from 0018F182:1
- database name "$1$DGA3620:[dir]<.RDB file>
- 20 global buffers freed; 14578 free out of 20000


Or abnormaly like this attach:

3-AUG-2020 13:01:50.39 - Received user attach request from 0018F182:1
- process name <process>, user <user>
- image name "$1$DGA3610:<EXE file>"
- database name "$1$DGA3620:[dir]<.RDB file>
- 20 global buffers allocated; 14558 free out of 20000
- sending normal user attach reply to 0018F182:1

Then some hours later (process terminated or crashed).
One recovery process started (RDM_RB73_15FA5 in this example).
Total of 0.04 sec from process crash to recovery done.

3-AUG-2020 20:56:39.47 - Received user image termination from 0018F182:1
- database name "$1$DGA3620:[dir]<.RDB file>
- abnormal user termination detected
- %SYSTEM-S-NORMAL, normal successful completion
- Monitor created Database Recovery process RDM_RB73_15FA5 (001BDC2F)
- user termination suspended until recovery ready

3-AUG-2020 20:56:39.48 - Received recovery attach from 001BDC2F:1
- process name RDM_RB73_15FA5, user SYSTEM
- database name "$1$DGA3620:[dir]<.RDB file>
- 20 previously allocated global buffers freed
- 20 global buffers allocated; 16078 free out of 20000
- sending normal recovery attach reply to 001BDC2F:1

3-AUG-2020 20:56:39.49 - Received recovery ready from 001BDC2F:1
- process name RDM_RB73_15FA5, user SYSTEM
- database name "$1$DGA3620:[dir]<.RDB file>
- sending normal recovery ready reply to 001BDC2F:1

3-AUG-2020 20:56:39.50 - Received recovery status from 001BDC2F:1
- process name RDM_RB73_15FA5, user SYSTEM
- database name "$1$DGA3620:[dir]<.RDB file>
- dead process transaction 0 was not active

3-AUG-2020 20:56:39.51 - Received recovery image termination from 001BDC2F:1
- database name "$1$DGA3620:[dir]<.RDB file>
- 20 global buffers freed; 16098 free out of 20000
- recovery was successful

3-AUG-2020 20:56:39.51 - Received Recovery process termination from 001BDC2F:1
- database name "$1$DGA3620:[dir]<.RDB file>
Will it work properly without the Rdb logicals being defined?
No, you need at least some of the logicals setup by the startup file.

$ sh log rdb*$*

(LNM$SYSTEM_TABLE)

"RDB$DISPATCH_IDENT" = "V7.3-300"
"RDB$DISPATCH_VERSION_VARIANT" = "73"
"RDBVMS$IDENT" = "V7.3-300"
"RDBVMS$IVP_DIR" = "SYS$COMMON:[SYSTEST.RDB73]"
"RDBVMS$LIB" = "SYS$COMMON:[SYSLIB]RDBVMSLIB73.OLB"
"RDBVMS$OPTION" = "SYS$COMMON:[SYSLIB]RDBVMS73.OPT"
"RDBVMS$VARIANT" = "73"
"RDBVMS$VERSION" = "7.3"
Arne
Phillip Helbig (undress to reply)
2020-12-11 17:46:52 UTC
Permalink
Post by Jan-Erik Söderholm
Post by Craig A. Berry
Post by David Jones
Post by Stephen Hoffman
SQLite can be useful for storing the data, particularly for those
without Oracle licenses and that for whatever reason.
Using Oracle in your startup sequencer means you have to start it before
you can run
the thing that starts it -- somewhat problematic. A cloud-based database
still needs the
network running. I link the startup driver application statically with
the sqlite3 library
and run it with the data file flagged as immutable (readonly, no locking).
I believe in the case of Rdb there is no start-up because there is no
service -- you just link your code against the libraries and run it in
the context of whatever process.  In other words, the same way you are
running SQLite.
Rdb use file access (unless using the using one of the remote network
protocols that rely SQL service or a thin JDBC driver server).
SQL/Services is for the SQL/Services API (as used by ODBC drivers),
JDBC from Java based clients or OCI from clients that thinks they are
accessing an Oracle Classic database.
Then you can run "RDBSERVER" using it's own protocoll. That s mainly for
VMS-2-VMS access such as when you want the single-node optimazitions
within a cluster and having the other cluster nodes access Rdb remotely
instead of direct shared access (involves more cross-cluster locking).
Doesn't have to be a cluster, of course.
Or to save license costs?
Post by Jan-Erik Söderholm
If you are running DECnet, you can specify a remote file (just as when
using FAL) in the attach and the Rdb code will detect that and connect
the the remote RDBSERVER object without the application having to care.
(I thinkthat also works for TCPIP, do not use this).
I think that it can be made to work with TCPIP.
Post by Jan-Erik Söderholm
$ tcpip sh service rdbserver
Service Port Proto Process Address State
RDBSERVER 611 TCP RDB 0.0.0.0 Enabled
$ ncp show object rdbserver
Object Number File/PID User Id Password
RDBSERVER 35 RDBSERVER.COM RDB$REMOTE <pw>
$
Anyway, RDBSERVER has nothing to do with SQL/Services that uses
a different TCPIP port and another DECnet obbject.
Right.
Post by Jan-Erik Söderholm
But yes, you can call RDBREMOTE for "file access", just as you can
with FAL, since the application doesn't care.
Right.
Post by Jan-Erik Söderholm
But will Rdb work properly without the RDMS_MONITOR and RMU
processes running?
No, your application will not do anything with the data without having
an "OK" from the monitor. You send an "attach request" (well, the code
in the sharable images does) to the monitor and the monitor says "OK".
Right.
Post by Jan-Erik Söderholm
Will it work properly without the Rdb logicals being defined?
No, you need at least some of the logicals setup by the startup file.
Right.
Arne Vajhøj
2020-12-11 18:35:22 UTC
Permalink
Post by Jan-Erik Söderholm
Post by Craig A. Berry
I believe in the case of Rdb there is no start-up because there is no
service -- you just link your code against the libraries and run it in
the context of whatever process.  In other words, the same way you are
running SQLite.
Rdb use file access (unless using the using one of the remote network
protocols that rely SQL service or a thin JDBC driver server).
SQL/Services is for the SQL/Services API (as used by ODBC drivers),
JDBC from Java based clients or OCI from clients that thinks they are
accessing an Oracle Classic database.
JDBC native driver (type 2) does not require the server.

JDBC thin driver (which Oracle calls type 4 even though it to me seems
more like a type 3) does require the server.
Post by Jan-Erik Söderholm
Then you can run "RDBSERVER" using it's own protocoll. That s mainly for
VMS-2-VMS access such as when you want the single-node optimazitions
within a cluster and having the other cluster nodes access Rdb remotely
instead of direct shared access (involves more cross-cluster locking).
Doesn't have to be a cluster, of course.
That type of pro/con is beyond my current Rdb knowledge
(I have not worked with it professionally since 1998).
Post by Jan-Erik Söderholm
But will Rdb work properly without the RDMS_MONITOR and RMU processes running?
No, your application will not do anything with the data without having
an "OK" from the monitor. You send an "attach request" (well, the code
in the sharable images does) to the monitor and the monitor says "OK".
Will it work properly without the Rdb logicals being defined?
No, you need at least some of the logicals setup by the startup file.
That is what I thought.

Rdb is not really like SQLite.

Arne
Jan-Erik Söderholm
2020-12-11 23:55:42 UTC
Permalink
Post by Arne Vajhøj
That type of pro/con is beyond my current Rdb knowledge
(I have not worked with it professionally since 1998).
I have worked professional with Rdb since V3.2 (early 90's)
without a real break.
Post by Arne Vajhøj
Post by Jan-Erik Söderholm
But will Rdb work properly without the RDMS_MONITOR and RMU processes running?
No, your application will not do anything with the data without having
an "OK" from the monitor. You send an "attach request" (well, the code
in the sharable images does) to the monitor and the monitor says "OK".
Will it work properly without the Rdb logicals being defined?
No, you need at least some of the logicals setup by the startup file.
That is what I thought.
Rdb is not really like SQLite.
No. And I do not remember, but have anyone really said so?

I havn't looked too closely, but SQLite seems to me more like
dBase-III with an SQL front-end.
Post by Arne Vajhøj
Arne
Arne Vajhøj
2020-12-12 01:17:20 UTC
Permalink
Post by Jan-Erik Söderholm
Post by Arne Vajhøj
Post by Jan-Erik Söderholm
But will Rdb work properly without the RDMS_MONITOR and RMU
processes running?
No, your application will not do anything with the data without having
an "OK" from the monitor. You send an "attach request" (well, the code
in the sharable images does) to the monitor and the monitor says "OK".
Will it work properly without the Rdb logicals being defined?
No, you need at least some of the logicals setup by the startup file.
That is what I thought.
Rdb is not really like SQLite.
No. And I do not remember, but have anyone really said so?
This sub-topic was kicked off by this:

# I believe in the case of Rdb there is no start-up because there is no
# service -- you just link your code against the libraries and run it in
# the context of whatever process. In other words, the same way you are
# running SQLite.
Post by Jan-Erik Söderholm
I havn't looked too closely, but SQLite seems to me more like
dBase-III with an SQL front-end.
SQLite is a typical embedded database.

MS Access, FireBird, a bunch of Java databases (H2, HSQLDB, Derby)
are all accessed similar.

DBase III and IV was somewhat similar a number of decades ago.

Arne
Craig A. Berry
2020-12-12 00:03:38 UTC
Permalink
Post by Arne Vajhøj
Post by Jan-Erik Söderholm
But will Rdb work properly without the RDMS_MONITOR and RMU processes running?
No, your application will not do anything with the data without having
an "OK" from the monitor. You send an "attach request" (well, the code
in the sharable images does) to the monitor and the monitor says "OK".
Will it work properly without the Rdb logicals being defined?
No, you need at least some of the logicals setup by the startup file.
That is what I thought.
Rdb is not really like SQLite.
Well in general, obviously. But in the specific matter of not running
as a service, I stand corrected in that it does not run as a service but
requires a monitor service to be running, which is an odd design, but
there it is. So it has prerequisites for start-up, which possibly could
have been overcome in the DEC days when things were more integrated.
Jan-Erik Söderholm
2020-12-12 00:42:00 UTC
Permalink
Post by Arne Vajhøj
Post by Jan-Erik Söderholm
But will Rdb work properly without the RDMS_MONITOR and RMU processes running?
No, your application will not do anything with the data without having
an "OK" from the monitor. You send an "attach request" (well, the code
in the sharable images does) to the monitor and the monitor says "OK".
Will it work properly without the Rdb logicals being defined?
No, you need at least some of the logicals setup by the startup file.
That is what I thought.
Rdb is not really like SQLite.
Well in general, obviously.  But in the specific matter of not running
as a service, I stand corrected in that it does not run as a service but
requires a monitor service to be running, which is an odd design, but
there it is.
It is a highly efficient design. Each user process has direct contact
with the files where the data is stored. No nead to send either the
SQL statements over some process-2-process or network link. And in
the same way, no need to send the resulting data back over the same
P2P or network link. It's all in the hands of your own process.

If the user process is doing some "offline" batch updates, it can start
an "exclusive" transaction and have the tables involved for its own
use with most of the usual locking activity closed down.

And you can configure such a process with very differnt buffers and
such parameters then the default for the actual database. For a design
(the most common one) with a central "DB server" process doing all the
"work", buffers, caches and such are shared in the common process.

All database processing code is run within the user process space.
Well, apart from special cases like a recovery process started when
a user process crashes.
So it has prerequisites for start-up, which possibly could
have been overcome in the DEC days when things were more integrated.
It's not really clear what you are calling "start-up" here. There is
no start-up (as when you start up the DB-server processes for an Oracle
"classic" DB). Just start the "monitor" and define some logical names.

The monitor know nothing about any "databases" and at that time, the
files where the database is stored hasn't been touched yet.

The actual "database" (as defined by the files where the data is stored)
does not come into play until some user process opens them and starts
processing the data.
Arne Vajhøj
2020-12-12 01:22:26 UTC
Permalink
Post by Jan-Erik Söderholm
Post by Arne Vajhøj
That is what I thought.
Rdb is not really like SQLite.
Well in general, obviously.  But in the specific matter of not running
as a service, I stand corrected in that it does not run as a service but
requires a monitor service to be running, which is an odd design, but
there it is.
It is a highly efficient design. Each user process has direct contact
with the files where the data is stored. No nead to send either the
SQL statements over some process-2-process or network link. And in
the same way, no need to send the resulting data back over the same
P2P or network link. It's all in the hands of your own process.
Whether it is an odd design probably depends on the eyes looking
at it.

It is an unusual design among today's databases.

The common designs today are:
* a database server, a running process that clients talk to
over the network
* an embedded database, nothing is running and started,
the database is simply a library called by application code
* support for both models (yes - several databases
can be used both as a database server and embedded)

Arne
David Jones
2020-12-12 14:27:54 UTC
Permalink
Post by Arne Vajhøj
Post by Arne Vajhøj
That is what I thought.
Rdb is not really like SQLite.
...
It is an unusual design among today's databases.
SQLite is a tool for applications to access their persistent data using
relational database semantics, the data being saved in a prescribed
file format designed for that purpose. It will let multiple users access
the file, but that's not really its strong suit. Rdb's design choices seem
geared around making the database a system resource that can
efficiently support multiple concurrent access.
Stephen Hoffman
2020-12-12 20:14:32 UTC
Permalink
It (Rdb) is a highly efficient design.
No doubt.

Oracle Rdb is also overkill for a system startup.

If a site happens to have Oracle Rdb licenses, and/or if Rdb is
familiar or is otherwise deemed necessary, and/or if the site needs a
platform-specific mid-range database and has the budget for it, by all
means use Rdb.

SQLite does all that is needed for a startup procedure.

And yes, like Rdb, SQLite also directly accesses the data, and not a
client-server connection to a database server.

Though often thought of as small, SQLite scales to larger database
files than permissible with OpenVMS with the 2 TiB ODS-2 and ODS-5
addressing limitations, too.

If client-server is required, MariaDB is an option. A port of MariaDB
is available for OpenVMS, though that port was—for various
previously-discussed reasons—not the most current.
--
Pure Personal Opinion | HoffmanLabs LLC
Stephen Hoffman
2020-12-11 15:18:16 UTC
Permalink
Post by David Jones
Post by Stephen Hoffman
SQLite can be useful for storing the data, particularly for those
without Oracle licenses and that for whatever reason.
Using Oracle in your startup sequencer means you have to start it
before you can run the thing that starts it -- somewhat problematic. A
cloud-based database still needs the network running. I link the
startup driver application statically with the sqlite3 library and run
it with the data file flagged as immutable (readonly, no locking).
Or the user or the system runs the database shenanigans as needed,
generating the startup file once, and caching that startup. That
generation happening either on demand, or on shutdown. Again, this ties
back to installation and kit integration, and of automating these and
other system management tasks.

Caching is how other systems boot quickly. Not by running database
queries. Not by doing device configurations, either. Caching is how
folks run searches quickly, too. Optimizing for low storage or for low
memory makes little sense in this era, given the un-VAX-like
availability of disk space—albeit in 2 TiB hunks—and given the
availability of memory.

Should the startup cache somehow become corrupted, boot
conversationally and SET /STARTUP and bypass and regenerate it.

In my Computing Happy Place and beyond automatic startup processing,
the OpenVMS system detects the corrupted startup or other corrupt boot
caches, and can also detect bad system parameters, and the system then
regenerates the associated caches automatically, and makes limited
additional attempt to continue with the bootstrap normally. These
limited retry attempts to avoid triggering a boot loop. But that
Computing Happy Place is not today.

Can't see much reason to use Oracle Rdb or Oracle Classic here, given
the costs and the availability of that package. But other folks have
licenses and will consider using Oracle. Hence my comment. And yes,
SQLite queries could run at boot. But... why? We already run a cached
startup. It's just one that's manually edited and manually maintained.
--
Pure Personal Opinion | HoffmanLabs LLC
David Jones
2020-12-11 17:24:57 UTC
Permalink
Post by Stephen Hoffman
And yes,
SQLite queries could run at boot. But... why? We already run a cached
startup. It's just one that's manually edited and manually maintained.
--
The primary feature I was after was parallelism during boot, the database is
just to facilitate demarking synchronization points.
Loading...