Updates to VMSTAR and Wget for VMS

Post by Steven Schweda
http://antinode.info/dec/sw/vmstar.html (1.20.3a)
http://antinode.info/dec/sw/wget.html (V4.3)
As usual, near-minimal testing, but VMSTAR can now extract
almost all of an OpenSSL kit on a VAX (which is an
improvement in data integrity, whether or not it's very
useful). Check the Wget docs for info on recently added
features (because I know nothing).

As always many thanks for such efforts. I've used both of the above and
also ZIP and UNZIP, to which you have massively contributed.

The last time I checked, Hunter's software collection was still online.
Since most community-contributed third-party products for VMS are there,
it would be nice to have those above there as well (if they aren't
already).

I believe I read here recently that the VMS distribution from VSI is to
contain ZIP and UNZIP. I don't know what they do, if they just include
the latest executable, or have an installation with the proper HELP
files, or whatever.

Steven Schweda

2020-12-29 14:04:14 UTC

[...] I've used both of the above [...]

Same here, of course. The highlight of the new Wget might
be that it seems to work without errors when downloading VSI
PAKs, using a URL with a query string, like:
"https://vmssoftware.com/community-license.php?hash=<hash>"
(without using "-O <nice_name>"). You still get the hideous
file name, but without the "utime" complaints ("file
specification syntax error", from my utime).

Hunter Goatley

2021-01-04 14:00:03 UTC

Post by Phillip Helbig (undress to reply)

Post by Steven Schweda
http://antinode.info/dec/sw/vmstar.html (1.20.3a)
http://antinode.info/dec/sw/wget.html (V4.3)

[...]

Post by Phillip Helbig (undress to reply)
The last time I checked, Hunter's software collection was still online.
Since most community-contributed third-party products for VMS are there,
it would be nice to have those above there as well (if they aren't
already).

An old version of VMSTAR has been in my archive for a long time, and
I've long intended to update it.

Steven's VMSTAR and WGET can now both be found in my archive, along with
the links to Steven's web pages for both.

http://www.process.com/resources/openvms/index.html

--
Hunter
------
Hunter Goatley, Process Software, http://www.process.com/
***@goatley.com http://hunter.goatley.com/

Phillip Helbig (undress to reply)

2021-01-04 14:24:10 UTC

Post by Phillip Helbig (undress to reply)

Post by Hunter Goatley

Post by Steven Schweda
http://antinode.info/dec/sw/vmstar.html (1.20.3a)
http://antinode.info/dec/sw/wget.html (V4.3)

The last time I checked, Hunter's software collection was still online.
Since most community-contributed third-party products for VMS are there,
it would be nice to have those above there as well (if they aren't
already).

An old version of VMSTAR has been in my archive for a long time,

That's the one I have.

Post by Hunter Goatley
and
I've long intended to update it.
Steven's VMSTAR and WGET can now both be found in my archive, along with
the links to Steven's web pages for both.
http://www.process.com/resources/openvms/index.html

Thanks!

Sometime this year, I'll update software such as this and perhaps get
some more stuff from your site.

I don't know if it's worth the trouble, but it would be nice if
third-party software like this had some sort of standard setup, i.e. a
ZIP file would create a directory with the product name and version
number and unpack the files in it, then @[.DIR]BUILD would build it
(probably calling @COMPILE and @LINK) and @[.DIR]SETUP (possibly also
called from BUILD) would set it up initially (say buy adding the .HLB to
a library specified by a logical name or, if empty, prompt for it (NOT
add it to VMS HELP by default)) and then @[.DIR]INSTALL would be done
at boot for installing images, setting up DCL verbs, or whatever, and
@[.DIR]DEFINE would be invoked before using it, probably usually
indirectly via SYS$SYLOGIN or LOGIN.COM, in order to define a symbol or
whatever. One could then build a list of directories on the disk
(perhaps a concealed device) where such packages are on the fly at boot
to call all the INSTALL procedures and at login to call all the DEFINE
procedures.

Hunter Goatley

2021-01-04 18:32:01 UTC

Post by Phillip Helbig (undress to reply)
I don't know if it's worth the trouble, but it would be nice if
third-party software like this had some sort of standard setup, i.e. a
ZIP file would create a directory with the product name and version
called from BUILD) would set it up initially (say buy adding the .HLB to
a library specified by a logical name or, if empty, prompt for it (NOT
at boot for installing images, setting up DCL verbs, or whatever, and
@[.DIR]DEFINE would be invoked before using it, probably usually
indirectly via SYS$SYLOGIN or LOGIN.COM, in order to define a symbol or
whatever. One could then build a list of directories on the disk
(perhaps a concealed device) where such packages are on the fly at boot
to call all the INSTALL procedures and at login to call all the DEFINE
procedures.

Some of the .zip files in my archive (all of the ones I worked on or
repackaged) have a LINK.COM to create .EXE files from the supplied
binaries. I never had the time to go through all of packages and try to
do what you're describing.

I had once thought of trying to create VMSINSTAL kits for everything,
but that was way too much work and would take way too much time for the
free time I've ever had.

--
Hunter
------
Hunter Goatley, Process Software, http://www.process.com/
***@goatley.com http://hunter.goatley.com/

Arne Vajhøj

2021-01-04 18:41:51 UTC

Post by Hunter Goatley

That is actually very interesting idea.

The standard for VMS open source packaging.

Post by Hunter Goatley
Some of the .zip files in my archive (all of the ones I worked on or
repackaged) have a LINK.COM to create .EXE files from the supplied
binaries. I never had the time to go through all of packages and try to
do what you're describing.

It should probably fall on the maintainers of the open source not
the distribution host.

Post by Hunter Goatley
I had once thought of trying to create VMSINSTAL kits for everything,
but that was way too much work and would take way too much time for the
free time I've ever had.

I don't think VMSINSTAL is the right solution.

I believe it should work for no priv users and
be strictly isolated to the dir where it is unpacked.

Arne

Phillip Helbig (undress to reply)

2021-01-04 19:46:10 UTC

Post by Arne Vajhøj
That is actually very interesting idea.
The standard for VMS open source packaging.
It should probably fall on the maintainers of the open source not
the distribution host.

Right.

Post by Hunter Goatley
I had once thought of trying to create VMSINSTAL kits for everything,
but that was way too much work and would take way too much time for the
free time I've ever had.

I don't think VMSINSTAL is the right solution.
I believe it should work for no priv users and
be strictly isolated to the dir where it is unpacked.

Same here. I have DISK$SOFT for things like this. Each VERSION of each
product has its own directory. Everything is there. From within an
application, one can access the HELP files directly, but might also want
them visible in the main HELP, but in a @ for an extra third-party
branch.

Hunter Goatley

2021-01-08 22:23:31 UTC

It should probably fall on the maintainers of the open source not
the distribution host.

Yes, it should have, but most people didn't, and I modified packages
when I could....

Post by Hunter Goatley
I had once thought of trying to create VMSINSTAL kits for everything,
but that was way too much work and would take way too much time for
the free time I've ever had.

I don't think VMSINSTAL is the right solution.
I believe it should work for no priv users and
be strictly isolated to the dir where it is unpacked.

Yes, those were other reasons I didn't do it.

--
Hunter
------
Hunter Goatley, Process Software, http://www.process.com/
***@goatley.com http://hunter.goatley.com/

Phillip Helbig (undress to reply)

2021-01-04 19:43:37 UTC

Post by Hunter Goatley
Some of the .zip files in my archive (all of the ones I worked on or
repackaged) have a LINK.COM to create .EXE files from the supplied
binaries.

Useful for those without the corresponding compiler. Some might want to
build from sources. Others might prefer pre-built executables. One
could offer all.

Post by Hunter Goatley
I never had the time to go through all of packages and try to
do what you're describing.

There is a great xkcd cartoon which is a table which tells you whether a
job is worth doing; the axes are how much time you can shave off of a
task and how often you do it. Together, those give you the time you can
afford to invest in optimizing the test before you spend more time than
you gain.

Post by Hunter Goatley
I had once thought of trying to create VMSINSTAL kits for everything,
but that was way too much work and would take way too much time for the
free time I've ever had.

I prefer VMSINSTAL and PRODUCT kits for DEC/Compaq/HP/VSI stuff but it
is often overkill for other things. I also like to keep third-party
stuff off the system disk. On the other hand, I like VMS-style CLIs and
HELP files and so on for third-party stuff. (UN)ZIP and VMSTAR are
exemplary here.

Steven Schweda

2021-01-04 20:42:10 UTC

Post by Phillip Helbig (undress to reply)
I don't know if it's worth the trouble, but it would be nice if
third-party software like this had some sort of standard setup, [...]

I'm sure that it would be too much work (for me), but I'm not sure
that it would be nice. If all users had identical (or even similar)
environments, and/or requirements, then there might be some hope. As
soon as build options appear, everything gets worse.

My approach is to provide documentation, and hope for the best.

Post by Phillip Helbig (undress to reply)
Some of the .zip files in my archive (all of the ones I worked on or
repackaged) have a LINK.COM to create .EXE files from the supplied
binaries. [...]

And, as I recall, that caused problems by being too simple for recent
(cough) versions of Info-ZIP programs. Whose official builders should
have built-in LINK-only options.

Post by Phillip Helbig (undress to reply)
It should probably fall on the maintainers of the open source not
the distribution host.

Good luck with that. For example, the Wget folks are very good about
making changes which I request, but have zero interest is dealing with
VMS themselves.

Post by Phillip Helbig (undress to reply)
I don't think VMSINSTAL is the right solution.

I can beat that. I don't think there is a right solution.

And the latest OpenSSL stuff (1.1.1) wants to pollute my SYS$COMMON.
What a world, ...

Arne Vajhøj

2021-01-05 01:03:00 UTC

Post by Arne Vajhøj
It should probably fall on the maintainers of the open source not
the distribution host.

Good luck with that. For example, the Wget folks are very good about
making changes which I request, but have zero interest is dealing with
VMS themselves.

But you could give them the files to put in the dist.

It is no so much whether it is the generic open source
product maintainers or the VMS port maintainers that
does it.

My point is that it is not the web site hosting the
kits that does it.

Arne

Arne Vajhøj

2021-01-05 01:04:50 UTC

Post by Arne Vajhøj
It should probably fall on the maintainers of the open source not
the distribution host.

Good luck with that. For example, the Wget folks are very good about
making changes which I request, but have zero interest is dealing with
VMS themselves.

But you could give them the files to put in the dist.
It is no so much whether it is the generic open source
product maintainers or the VMS port maintainers that
does it.
My point is that it is not the web site hosting the
kits that does it.

Side note: VMS port maintainer*s* is a bit
over-optmistic for most open source projects.

Arne

Steven Schweda

2021-01-05 04:19:21 UTC

Post by Arne Vajhøj
But you could give them the files to put in the dist.

I could if they wanted them. "zero interest" means exactly that.
Which makes sense. They can't test changes.

Dave Froble

2021-01-05 03:41:28 UTC

Post by Phillip Helbig (undress to reply)
I don't know if it's worth the trouble, but it would be nice if
third-party software like this had some sort of standard setup, [...]

I'm sure that it would be too much work (for me), but I'm not sure
that it would be nice. If all users had identical (or even similar)
environments, and/or requirements, then there might be some hope. As
soon as build options appear, everything gets worse.
My approach is to provide documentation, and hope for the best.

I totally agree! I don't need others deciding how my system(s) are set
up. Perhaps provide an example, and if anyone is fine with that, they
can use it.

--
David Froble Tel: 724-529-0450
Dave Froble Enterprises, Inc. E-Mail: ***@tsoft-inc.com
DFE Ultralights, Inc.
170 Grimplin Road
Vanderbilt, PA 15486

Phillip Helbig (undress to reply)

2021-01-05 13:16:30 UTC

Post by Dave Froble

Post by Phillip Helbig (undress to reply)
I don't know if it's worth the trouble, but it would be nice if
third-party software like this had some sort of standard setup, [...]

I'm sure that it would be too much work (for me), but I'm not sure
that it would be nice. If all users had identical (or even similar)
environments, and/or requirements, then there might be some hope. As
soon as build options appear, everything gets worse.
My approach is to provide documentation, and hope for the best.

I totally agree! I don't need others deciding how my system(s) are set
up. Perhaps provide an example, and if anyone is fine with that, they
can use it.

One can have both: harmless defaults for quick-and-easy setup, but of
course modifiable according to the documentation.

Hunter Goatley

2021-01-08 22:25:01 UTC

Post by Phillip Helbig (undress to reply)

Post by Hunter Goatley
Some of the .zip files in my archive (all of the ones I worked on or
repackaged) have a LINK.COM to create .EXE files from the supplied
binaries.

Useful for those without the corresponding compiler. Some might want to
build from sources. Others might prefer pre-built executables. One
could offer all.

The ones I mentioned above provided everything: sources, .OBJs for those
who didn't have compilers but wanted to link the .EXE, and an .EXE for
those who just wanted to go.

--
Hunter
------
Hunter Goatley, Process Software, http://www.process.com/
***@goatley.com http://hunter.goatley.com/

2020-12-29 15:52:15 UTC

Post by Steven Schweda
As always, complaints are welcome.

Not a complaint, so I hope it's welcome, but I wanted to say
that you have done and continue to do very meaningful work.

One of the first things that many people usually do after
they installed VMS is to obtain file compression/archiving
software like the ones you kindly work on and provide.

- MG

Joukj

2021-01-05 09:30:13 UTC

I tried them both:

-VMSTAR : compiled out of the box both on my 8.4 Itaniums and
8.4-2L1 Alpha's
Thanks, as always, for the updated version

-wget : I had to tweak it a little bit (only tried on Itanium yet)
The build I tried was against OpenSSL 1.1.1h
problems I found + solutions/work-arounds
- One of the "Actions didn't update" made MMS to stop
do I added /ign=warning on the command line
- The compilation gave a warning of of an external in
the ssl-headers to be more than 31 characters. Since
OpenSSL is compiled by default with
/name=(as_is,short), I added the same to the CFLAGS
in the *.mms files.
By doing this you need two more things
1) upcase lib$initialize in the c-sources
2) in the *.mms files replace
/include = (main)
by
/include = ("main")
-The linking gives the following warning:

LINK /notraceback /executable =
[-.SRC.IA64L]WGET.EXE [-.SRC.IA64L]LI
BSRC.OLB /library /include = ("main"),
[-.LIB.IA64L]LIBLIB.OLB /library
, [-.VMS]WGET_SSL_OD_SHR.OPT /OPTIONS
%ILINK-W-MULDEF, symbol DECC$__UTC_UTIME multiply
defined
file: SYS$COMMON:[SYSLIB]DECC$SHR.EXE;1
%ILINK-W-MULDEF, symbol DECC$TXSNPRINTF multiply
defined
file: SYS$COMMON:[SYSLIB]DECC$SHR.EXE;1
%ILINK-W-MULDEF, symbol DECC$STRTOK_R multiply
defined
file: SYS$COMMON:[SYSLIB]DECC$SHR.EXE;1

I think this is harmless

regards
Jouk

Steven Schweda

2021-01-05 20:48:37 UTC

Interesting (but vague) report.

Post by Joukj
-wget : I had to tweak it a little bit (only tried on Itanium yet)
The build I tried was against OpenSSL 1.1.1h
problems I found + solutions/work-arounds

You might need to supply an actual problem report with actual
details. I did most of the work on my main rx2600 using "HP C V7.3-020
on OpenVMS IA64 V8.4" and "%MMS-I-IDENT, MMS V3.8-2 [...]", with no
problems. I just checked it there with OpenSSL 1.1.1i, also with no
problems. I also tried it on my rx2660, leading to that parenthetical
"(OpenVMS IA64 V8.4-2L1, VSI C V7.4-001, [...])" in [.vms]vms_notes.txt,
also with no problems. MMS there is V4.0, OpenSSL 1.1.1h.

ITS $ mms /mac = (OSSLD=1) DASHV # rx2600

5-JAN-2021 13:56:54
Destination: [.IA64L]

OPENSSL: OSSL$INCLUDE:[openssl]

OS_TYPE string: "VMS IA64 V8.4"

[...]
mcr [-.SRC.IA64L]wget "-V"
GNU Wget 1.20.3a built on VMS IA64 V8.4.

-cares +digest -gpgme +https +ipv6 -iri +large-file -metalink -nls
+ntlm +opie -psl +ssl/openssl

VMS V8.4, OpenSSL 1.1.1i 8 Dec 2020
[...]

ITX $ mms /mac = (OSSLD=1) DASHV # rx2660

5-JAN-2021 14:25:11
Destination: [.IA64L]

OPENSSL: OSSL$INCLUDE:[openssl]

OS_TYPE string: "VMS IA64 V8.4-2L1"

[...]
mcr [-.SRC.IA64L]wget "-V"
GNU Wget 1.20.3a built on VMS IA64 V8.4-2L1.

-cares +digest -gpgme +https +ipv6 -iri +large-file -metalink -nls
+ntlm +opie -psl +ssl/openssl

VMS V8.4-2L1, OpenSSL 1.1.1h 22 Sep 2020
[...]

Post by Joukj
- One of the "Actions didn't update" made MMS to stop [...]

I see none of those, but I thought that they were "-I-", and stopped
nothing.

Post by Joukj
- The compilation gave a warning of of an external in
the ssl-headers to be more than 31 characters. [...]

Not here.

Post by Joukj
1) upcase lib$initialize in the c-sources

I've added "#define lib$initialize LIB$INITIALIZE" in other places,
where CC /NAMES=AS_IS made sense. Never needed it for this.

Post by Joukj
%ILINK-W-MULDEF, symbol DECC$__UTC_UTIME multiply defined

I'd guess that that's a consequence of CC /NAMES=AS_IS.

Post by Joukj
I think this is harmless

I wouldn't bet on it. Wget should be using my utime(), not the CRTL
utime().

Similar for SNPRINTF and STRTOK_R. Case mismatch in the
/prefix_library_entries stuff? I never tried (or needed) to use
CC /NAMES=AS_IS, so trouble with that would be easy to believe.

Steven Schweda

2021-01-06 04:13:14 UTC

After some distraction and subconscious brain activity (almost the
only kind these days), my guess (without seeing the actual MMS
command(s), or running the experiment) would be that the wrong OpenSSL
MMS macro was specified for the Wget build, causing a mix of
old/new-version stuff, rather than a consistent set of header files and
objects/images.

In the comments in [.vms]descrip.mms or the "Instructions" section in
[.vms]vms_notes.txt, look for:
HPSSL=1
HPSSL1=1
OSSL=1
OSSLD=1
(OSSLOLB=1)

It might be possible to automate the OpenSSL selection more, but,
with all the variations now possible, this (explicit) scheme seemed
safer.

Joukj

2021-01-06 07:45:52 UTC

Post by Steven Schweda
Interesting (but vague) report.

Post by Joukj
-wget : I had to tweak it a little bit (only tried on Itanium yet)
The build I tried was against OpenSSL 1.1.1h
problems I found + solutions/work-arounds

The only difference is that my MMS version is 3.9-01
Command used:
MMS /MACRO = (OSSLD=1)/ign=warning

OpenSSL was build from source.

I will try to build Openssl 1.1.1i first to see, if that makes the
difference.

Post by Joukj
- One of the "Actions didn't update" made MMS to stop [...]

I see none of those, but I thought that they were "-I-", and stopped
nothing.

On my system they are with -W-:
%MMS-W-GWKACTNOUPD, Actions didn't update

Is this a difference between MMS 3.8 and 3.9?

Post by Joukj
- The compilation gave a warning of of an external in
the ssl-headers to be more than 31 characters. [...]

Not here.

I'll retry when I have compiled Openssl1.1.1i and if the error persists
I will send you a full error message.

Post by Joukj
1) upcase lib$initialize in the c-sources

I've added "#define lib$initialize LIB$INITIALIZE" in other places,
where CC /NAMES=AS_IS made sense. Never needed it for this.

I have some software (i.e. gtk) where the only difference between
externals is upper/lower case. So I use the /name=(as_is) as a
"standard" option.

Post by Joukj
%ILINK-W-MULDEF, symbol DECC$__UTC_UTIME multiply defined

I'd guess that that's a consequence of CC /NAMES=AS_IS.

Post by Joukj
I think this is harmless

I wouldn't bet on it. Wget should be using my utime(), not the CRTL
utime().
Similar for SNPRINTF and STRTOK_R. Case mismatch in the
/prefix_library_entries stuff? I never tried (or needed) to use
CC /NAMES=AS_IS, so trouble with that would be easy to believe.

What I see in other open-source projects: When you need to replace a
library-procedure with your own one, you use a preprocessor define to
prefix the routine (normally done in the config.h header).

Jouk

Steven Schweda

2021-01-06 13:28:53 UTC

Post by Joukj
The only difference is that my MMS version is 3.9-01

That might be whole explanation. It's been a long time, but I have a
dim recollection of trying some V3.9 or other (back when a hobbyist
could get current software), having a bunch of case-related problems
with it, and going back to V3.8. Could be false-memory syndrome, but
perhaps not.

I haven't been testing much lately with MMK, but it might have fewer
problems. I might try that.

Post by Joukj
I will try to build Openssl 1.1.1i first to see, if that makes the
difference.

Note that my rx2660 (V8.4-2L1) test used 1.1.1h.

Post by Joukj
%MMS-W-GWKACTNOUPD, Actions didn't update
Is this a difference between MMS 3.8 and 3.9?

Perhaps. I haven't seen a GWKACTNOUPD lately.

Post by Joukj
I have some software (i.e. gtk) where the only difference between
externals is upper/lower case. So I use the /name=(as_is) as a
"standard" option.

That's fine, but it affects many things, and, with no reason ro use
it, I've never tested building Wget that way. OpenSSL certainly doesn't
force its use. (I claim.)

Post by Joukj
What I see in other open-source projects: When you need to replace a
library-procedure with your own one, you use a preprocessor define to
prefix the routine (normally done in the config.h header).

That's one way, but Wget drags in a pile of GNU stuff, some of which
seems to work very hard to defeat that method. (Look for "#undef".) In
some cases, it's easier to use the GNU stuff (in [.lib]) always than it
would be to deal with the VMS CRTL version differences for recently
added functions.

As usual, many things are possible, and I don't claim that my way is
the only way, but it does seem to work for me. At least not using that
MMS version.

Steven Schweda

2021-01-06 19:30:14 UTC