VMS process communication

Post by Arne VajhÃ¸j
https://www.vajhoej.dk/arne/articles/vmsipc.html
Introduction
Shared Memory
Concept
Demo
System Service (C, Fortran, Pascal)
Memory Mapped File
Concept
Demo
System Service (C, Fortran, Pascal)
Writeable Shareable Image
Concept
Demo
Linker/installer (Fortran, Pascal, C, Basic)
TCP/IP Socket
Concept
Demo
Java API (Java)
C API (C)
Python API (Python, Jython)
Wrapper C API (Fortran, Pascal, Cobol, Basic)
Message Queue
Concept
Demo
JMS API (Java, Jython)
STOMP C library (C)
STOMP Python library (Python)
Wrapper C STOMP library (Fortran, Pascal, Cobol, Basic)
Index Sequential File
Concept
Demo
Language builtin (Pascal, Cobol, Basic, Fortran)
RMS API (C)
VMS Python IndexedFile (Python)
JVM ISAM library (Java, Jython)
SQLite Database
Concept
Demo
C API (C)
Python API (Python, Jython)
JDBC API (Java)
JPA API (Java, Jython)
Wrapper C API (Pascal)
Arne

Thanks so much. That's really terrific. And ditto for your other
articles https://www.vajhoej.dk/arne/articles.html which I hadn't
even been aware of. Maybe you might arrange/aggregate them into one
or more books along the lines of Stallman's just released C tutorial
and reference https://git.savannah.gnu.org/git/c-intro-and-ref.git

--
John Forkosh ( mailto: ***@f.com where j=john and f=forkosh )

Arne Vajhøj

2022-09-21 23:12:46 UTC

Post by John Forkosh

Post by Arne VajhÃ¸j
https://www.vajhoej.dk/arne/articles/vmsipc.html

Post by John Forkosh
Thanks so much. That's really terrific. And ditto for your other
articles https://www.vajhoej.dk/arne/articles.html which I hadn't
even been aware of.

Thank you.

Post by John Forkosh
Maybe you might arrange/aggregate them into one
or more books along the lines of Stallman's just released C tutorial
and reference https://git.savannah.gnu.org/git/c-intro-and-ref.git

I have actually at occasions played with the idea.

But it would require some effort.

Maybe.

Arne

Dave Froble

2022-09-13 03:23:28 UTC

Why no mention of mailboxs ?

--
David Froble Tel: 724-529-0450
Dave Froble Enterprises, Inc. E-Mail: ***@tsoft-inc.com
DFE Ultralights, Inc.
170 Grimplin Road
Vanderbilt, PA 15486

Single Stage to Orbit

2022-09-13 07:39:02 UTC

Post by Dave Froble
Why no mention of mailboxs ?

I did wonder about that, too.

--
Tactical Nuclear Kittens

Craig A. Berry

2022-09-13 12:52:32 UTC

Post by Dave Froble
Why no mention of mailboxs ?

I did wonder about that, too.

My first thought was why no ICC. And then I actually looked at the
page, which says:

Some ways of process communication are not covered (yet) including:

mailboxes
sys$icc_* system services

Arne Vajhøj

2022-09-14 00:34:27 UTC

Post by Dave Froble
Why no mention of mailboxs ?

I did wonder about that, too.

My first thought was why no ICC. And then I actually looked at the
mailboxes
sys$icc_* system services

They will hopefully both show up. But they did not make
it for version 1.0.

I do not have any experience wth sys$icc_* so that will
need a little research.

I have actually started on mailboxes, I encountered some
problems with what I want to do, so it got postponed.

Arne

Dave Froble

2022-09-14 01:03:07 UTC

Post by Dave Froble
Why no mention of mailboxs ?

I did wonder about that, too.

My first thought was why no ICC. And then I actually looked at the
mailboxes
sys$icc_* system services

They will hopefully both show up. But they did not make
it for version 1.0.
I do not have any experience wth sys$icc_* so that will
need a little research.
I have actually started on mailboxes, I encountered some
problems with what I want to do, so it got postponed.
Arne

Ask your questions.

Do you want some examples?

--
David Froble Tel: 724-529-0450
Dave Froble Enterprises, Inc. E-Mail: ***@tsoft-inc.com
DFE Ultralights, Inc.
170 Grimplin Road
Vanderbilt, PA 15486

Arne Vajhøj

2022-09-18 23:58:45 UTC

Post by Dave Froble
Why no mention of mailboxs ?

I did wonder about that, too.

My first thought was why no ICC. And then I actually looked at the
mailboxes
sys$icc_* system services

They will hopefully both show up. But they did not make
it for version 1.0.
I do not have any experience wth sys$icc_* so that will
need a little research.
I have actually started on mailboxes, I encountered some
problems with what I want to do, so it got postponed.

Updated with mailboxes and DECnet task-to-task.

Mailboxes

Concept
Demo
Standard IO (C, Fortran, Pascal, Cobol, Basic, Java, Python)

DECnet Task-to-Task

Concept
Demo
RMS (DCL, C, Fortran, Pascal, Cobol, Basic, Python)

SYS$ICC_* and Redis on the TODO list.

Arne

Jan-Erik Söderholm

2022-09-19 09:28:27 UTC

Post by Dave Froble
Why no mention of mailboxs ?

I did wonder about that, too.

My first thought was why no ICC. And then I actually looked at the
mailboxes
sys$icc_* system services

They will hopefully both show up. But they did not make
it for version 1.0.
I do not have any experience wth sys$icc_* so that will
need a little research.
I have actually started on mailboxes, I encountered some
problems with what I want to do, so it got postponed.

Updated with mailboxes and DECnet task-to-task.
Mailboxes
    Concept
    Demo
    Standard IO (C, Fortran, Pascal, Cobol, Basic, Java, Python)
DECnet Task-to-Task
    Concept
    Demo
    RMS (DCL, C, Fortran, Pascal, Cobol, Basic, Python)
SYS$ICC_* and Redis on the TODO list.
Arne

Interesting! Now, what was that link now... :-)

Single Stage to Orbit

2022-09-19 18:21:38 UTC

Post by Jan-Erik SÃ¶derholm
Interesting! Now, what was that link now... :-)

I bookmarked it some time ago:
https://www.vajhoej.dk/arne/articles/vmsipc.html

--
Tactical Nuclear Kittens

Arne Vajhøj

2022-09-19 20:41:40 UTC

Post by Jan-Erik SÃ¶derholm
Interesting! Now, what was that link now... :-)

https://www.vajhoej.dk/arne/articles/vmsipc.html

Yep. Sorry for not include the link.

Arne

Jan-Erik Söderholm

2022-09-19 22:06:21 UTC

Post by Jan-Erik SÃ¶derholm
Interesting! Now, what was that link now... :-)

https://www.vajhoej.dk/arne/articles/vmsipc.html

Yep. Sorry for not include the link.
Arne

No problem... :-)

In the part about mailboxes, I miss one language, DCL.

$ create/mailbox/perm abc

$ open/write abc
$ write abc "Hello".
$ close abc

$ open/read abc
$ read abc inline
$ write sys$output 'inline'

Very short example, and it might not work...

Arne Vajhøj

2022-09-19 22:51:27 UTC

Post by Jan-Erik SÃ¶derholm
Interesting! Now, what was that link now... :-)

https://www.vajhoej.dk/arne/articles/vmsipc.html

Yep. Sorry for not include the link.

No problem... :-)
In the part about mailboxes, I miss one language, DCL.
$ create/mailbox/perm abc
$ open/write abc
$ write abc "Hello".
$ close abc
$ open/read abc
$ read abc inline
$ write sys$output 'inline'
Very short example, and it might not work...

It should be possible to make it work.

Thinking about it - index-sequential files are
also accessible from DCL.

More to go on the TODO list.

Arne

Arne Vajhøj

2022-09-21 00:02:47 UTC

Post by Jan-Erik SÃ¶derholm
Interesting! Now, what was that link now... :-)

https://www.vajhoej.dk/arne/articles/vmsipc.html

Yep. Sorry for not include the link.

No problem... :-)
In the part about mailboxes, I miss one language, DCL.
$ create/mailbox/perm abc
$ open/write abc
$ write abc "Hello".
$ close abc
$ open/read abc
$ read abc inline
$ write sys$output 'inline'
Very short example, and it might not work...

It should be possible to make it work.
Thinking about it - index-sequential files are
also accessible from DCL.
More to go on the TODO list.

Added:
* mailboxes / DCL
* socket / PHP
* index-sequential file / DCL
* SQLite / PHP

Arne

Stephen Hoffman

2022-09-20 22:34:22 UTC

Post by Jan-Erik SÃ¶derholm
In the part about mailboxes, I miss one language, DCL.
...

Here's DCL DECnet task-to-task:

https://www.digiater.nl/openvms/freeware/v80/hoffman_examples/xqtype.com

Wouldn't suggest using any of this for anything particularly relevent
or sensitive, and it's dicy even on a trusted network.

Unfortunately, DCL lacks TCP, TLS, and ssh connectivity support, and
lacks async / full duplex support.

--
Pure Personal Opinion | HoffmanLabs LLC

Arne Vajhøj

2022-10-04 23:11:24 UTC

Post by Arne VajhÃ¸j
I do not have any experience wth sys$icc_* so that will
need a little research.

Done.

https://www.vajhoej.dk/arne/articles/vmsipc.html#icc

ICC
Concept
Demo
System Service (C, Fortran, Pascal)

Arne

Ian Miller

2022-10-05 09:28:14 UTC

Post by Arne VajhÃ¸j
I do not have any experience wth sys$icc_* so that will
need a little research.

Done.
https://www.vajhoej.dk/arne/articles/vmsipc.html#icc
ICC
Concept
Demo
System Service (C, Fortran, Pascal)
Arne

the ICC routines are intriguing. Good to see an example. Did you report the issue with Pascal to VSI?

John Reagan

2022-10-05 19:17:40 UTC

Post by Arne VajhÃ¸j
I do not have any experience wth sys$icc_* so that will
need a little research.

Done.
https://www.vajhoej.dk/arne/articles/vmsipc.html#icc
ICC
Concept
Demo
System Service (C, Fortran, Pascal)
Arne

the ICC routines are intriguing. Good to see an example. Did you report the issue with Pascal to VSI?

I have not heard about any issues with the ICC definitions being Pascal "unfriendly".

If you ever want to override/bypass the type checking between the actual parameter and the formal parameter, you can use an explicit %IMMED, %REF, or %STDESCR as appropriate on the actual parameter. In that case, the compiler will skip the check and just pass what you specified. You can even provide more arguments than in the format definition by just using explicit argument mechanisms on the "extra" arguments.

Arne Vajhøj

2022-10-05 23:06:02 UTC

Post by John Reagan

Post by Arne VajhÃ¸j
I do not have any experience wth sys$icc_* so that will
need a little research.

Done.
https://www.vajhoej.dk/arne/articles/vmsipc.html#icc
ICC
Concept
Demo
System Service (C, Fortran, Pascal)

the ICC routines are intriguing. Good to see an example. Did you report the issue with Pascal to VSI?

I have not heard about any issues with the ICC definitions being Pascal "unfriendly".

I think only a small fraction of VMS sites use the ICC stuff and only a
small fraction of those use Pascal as language (the ICC stuff is new
enough to C being most common language).

But the fact that I could not get it working does not prove there
is a problem in SYS$LIBRARY:STARLET.PAS - maybe the problem is 40 cm
(16 inches) in front of the screen.

:-)

But some things puzzles me.

The client programs all use:
sys$icc_connectw
sys$icc_transmitw
sys$icc_receivew
sys$icc_disconnectw

All of them have an IOSB argument.

Per the documentation those are:
8 byte IOSB
8 byte IOSB
16 byte IOSB
8 byte IOSB

C has a:
struct _ios_icc

And defines the arguments for the 4 system services as as:
struct _ios_icc *ios_icc
struct _ios_icc *ios_icc
struct _ios_icc *ios_icc
struct _iosb *iosb

Which works fine. It is not obvious to me why the the last one is
different, but no problem.

Pascal has a record IOS_ICC$TYPE and it works fine for me, so that is
all good.

But the argument to the functions are defined as:
VAR IOS_ICC : [VOLATILE] $DEFPTR;
VAR IOS_ICC : [VOLATILE] $UQUAD;
VAR IOS_ICC : [VOLATILE] $DEFPTR;
VAR IOSB : [VOLATILE] $UQUAD;

With:
$DEFPTR = [UNSAFE] ^$DEFTYP;
$DEFTYP = [UNSAFE] INTEGER;
$UQUAD = [QUAD,UNSAFE] RECORD L0,L1:UNSIGNED; END;

Which seems inconsistent and hard/impossible to use for me
($DEFPTR really gave me a hard time).

So I simply used other declarations with arguments:
VAR IOS_ICC : [VOLATILE] IOS_ICC$TYPE;
VAR IOS_ICC : [VOLATILE] IOS_ICC$TYPE;
VAR IOS_ICC : [VOLATILE] IOS_ICC$TYPE;
VAR IOSB : [VOLATILE] IOS_ICC$TYPE;

It works fine and seems a lot more logical to me.

I also had problems with the buffers. $ICC_TRANSMITW
has a SEND_BUF and $ICC_RECEIVEW has a RECV_BUF but they
are declared as $DEFPTR and UNSIGNED. That gave my problems with
transmit so I changed SEND_BUF to also be UNSIGNED.

Arne

Arne Vajhøj

2022-10-05 23:26:29 UTC

Post by Arne VajhÃ¸j
I also had problems with the buffers. $ICC_TRANSMITW
has a SEND_BUF and $ICC_RECEIVEW has a RECV_BUF but they
are declared as $DEFPTR and UNSIGNED. That gave my problems with
transmit so I changed SEND_BUF to also be UNSIGNED.

Let me elaborate on this.

The sys$libary:starlet.pas definition is:
SEND_BUF : $DEFPTR;

And I cannot get that to work.

I am probably missing something very basic, but ...

Demo:

$ type zb.c
#include <stdio.h>

void f(unsigned char *s)
{
printf("%02X %02X %02X %02X %02X %02X %02X %02X\n", s[0], s[1],
s[2], s[3], s[4], s[5], s[6], s[7]);
}
$ cc zb
$ type za.pas
program za(input,output);

type
$DEFTYP = [UNSAFE] INTEGER;
$DEFPTR = [UNSAFE] ^$DEFTYP;

[external('F')] procedure f1(%IMMED s : unsigned); external;
[external('F')] procedure f2(s : $DEFPTR); external;
[external('F')] procedure f3(%IMMED s : $DEFPTR); external;

var
s : varying [255] of char;

begin
s := 'ABCDEF' + chr(0);
f1(iaddress(s.body));
f2(s.body);
f2(iaddress(s.body));
f3(s.body);
f3(iaddress(s.body));
end.
$ pas za
$ link za + zb
$ run za
41 42 43 44 45 46 00 00
41 42 43 44 00 00 00 00
CA 79 E2 7A 00 00 00 00
00 00 00 00 00 00 00 00
41 42 43 44 45 46 00 00

Especially:
f2(s.body);
output puzzles me.

Arne

Marc Van Dyck

2022-09-13 08:02:16 UTC

Why no mention of mailboxs ?

Was going to ask the same. Simple and efficient, covers many of the
needs. Enough for me to motivate the development of a wrapper allowing
me to use mailboxes from DCL.

--
Marc Van Dyck

Single Stage to Orbit

2022-09-13 09:12:48 UTC

Post by Marc Van Dyck

Post by Dave Froble
Why no mention of mailboxs ?

Was going to ask the same. Simple and efficient, covers many of the
needs. Enough for me to motivate the development of a wrapper
allowing me to use mailboxes from DCL.

Back in the early 90s at Portsmouth Uni, we put together a MUD game
using only mailboxes as the chaps running the VAX systems limited us to
only the TMPMBX privileges and a ridiculously small 200 block quota

--
Tactical Nuclear Kittens

Lee Gleason

2022-09-13 16:08:30 UTC

Post by Arne VajhÃ¸j
https://www.vajhoej.dk/arne/articles/vmsipc.html
    Introduction
    Shared Memory
        Concept
        Demo
        System Service (C, Fortran, Pascal)
    Memory Mapped File
        Concept
        Demo
        System Service (C, Fortran, Pascal)
    Writeable Shareable Image
        Concept
        Demo
        Linker/installer (Fortran, Pascal, C, Basic)
    TCP/IP Socket
        Concept
        Demo
        Java API (Java)
        C API (C)
        Python API (Python, Jython)
        Wrapper C API (Fortran, Pascal, Cobol, Basic)
    Message Queue
        Concept
        Demo
        JMS API (Java, Jython)
        STOMP C library (C)
        STOMP Python library (Python)
        Wrapper C STOMP library (Fortran, Pascal, Cobol, Basic)
    Index Sequential File
        Concept
        Demo
        Language builtin (Pascal, Cobol, Basic, Fortran)
        RMS API (C)
        VMS Python IndexedFile (Python)
        JVM ISAM library (Java, Jython)
    SQLite Database
        Concept
        Demo
        C API (C)
        Python API (Python, Jython)
        JDBC API (Java)
        JPA API (Java, Jython)
        Wrapper C API (Pascal)
Arne

Why no mention of mailboxs ?

And event flags and locks?

--
Lee K. Gleason N5ZMR
Control-G Consultants
***@comcast.net

Dave Froble

2022-09-13 20:52:39 UTC

Why no mention of mailboxs ?

And event flags and locks?
--
Lee K. Gleason N5ZMR
Control-G Consultants

While some things can be used for simple communications, I'd expect much more
from "communication". But yeah, a simple True/False can be considered
communication.

--
David Froble Tel: 724-529-0450
Dave Froble Enterprises, Inc. E-Mail: ***@tsoft-inc.com
DFE Ultralights, Inc.
170 Grimplin Road
Vanderbilt, PA 15486

Robert A. Brooks

2022-09-13 23:23:14 UTC

Post by Lee Gleason
And event flags and locks?

While some things can be used for simple communications, I'd expect
much more from "communication". But yeah, a simple True/False can
be considered communication.

Lock value blocks can pass up to 16 bytes of data, or 64 bytes if using extended lock value blocks.

Shadowing driver is a heavy consumer of locks and lock value blocks.

-- Rob

Dave Froble

2022-09-14 00:54:45 UTC

Post by Robert A. Brooks

Post by Lee Gleason
And event flags and locks?

While some things can be used for simple communications, I'd expect much more
from "communication". But yeah, a simple True/False can
be considered communication.

Lock value blocks can pass up to 16 bytes of data, or 64 bytes if using extended
lock value blocks.
Shadowing driver is a heavy consumer of locks and lock value blocks.

There are all types of communication requirements. Some simple, some not so
simple. For example:

When I need to tell a detached posting program to process an incoming order,
There are multiple things required in the message.

My utility for such communications passes messages almost 512 bytes in length.
Don't always need that much, but when I do, it's there.

--
David Froble Tel: 724-529-0450
Dave Froble Enterprises, Inc. E-Mail: ***@tsoft-inc.com
DFE Ultralights, Inc.
170 Grimplin Road
Vanderbilt, PA 15486

Richard Maher

2022-09-14 03:05:01 UTC

Post by Robert A. Brooks

Post by Lee Gleason
And event flags and locks?

While some things can be used for simple communications, I'd expect
much more from "communication". But yeah, a simple True/False
can be considered communication.

Lock value blocks can pass up to 16 bytes of data, or 64 bytes if
using extended lock value blocks.
Shadowing driver is a heavy consumer of locks and lock value blocks.
-- Rob

And I believe with Oracle's DLM the LVB can incorporate an entire
database page which forms the infrastructure needed for Cache Fusion.

The rest of us just have to BLAST the holder and ping it to disk unless
you're a single node with global buffers.

Anyway, I'd just like to say well done @Arne an not bitch about "but you
didn't do this". If Arne's put it in GitHub then knock yourselves out.

Arne Vajhøj

2022-09-14 00:38:11 UTC

Post by Arne VajhÃ¸j
https://www.vajhoej.dk/arne/articles/vmsipc.html

Why no mention of mailboxs ?

And event flags and locks?

To me those are synchronization techniques not communication
techniques - or at least not general communication techniques.

Arne

Dave Froble

2022-09-14 00:56:36 UTC

Post by Arne VajhÃ¸j
https://www.vajhoej.dk/arne/articles/vmsipc.html

Why no mention of mailboxs ?

And event flags and locks?

To me those are synchronization techniques not communication
techniques - or at least not general communication techniques.

As Rob pointed out, one can do a lot with the DLM.

Isn't synchronization itself a form of communication?

--
David Froble Tel: 724-529-0450
Dave Froble Enterprises, Inc. E-Mail: ***@tsoft-inc.com
DFE Ultralights, Inc.
170 Grimplin Road
Vanderbilt, PA 15486

Arne Vajhøj

2022-09-14 01:14:32 UTC

Post by Arne VajhÃ¸j
https://www.vajhoej.dk/arne/articles/vmsipc.html

Why no mention of mailboxs ?

And event flags and locks?

To me those are synchronization techniques not communication
techniques - or at least not general communication techniques.

As Rob pointed out, one can do a lot with the DLM.
Isn't synchronization itself a form of communication?

Yes.

But not what I would call general communication.

It is not intended to send arbitrary messages - it is
intended to synchronize flows.

Arne

Stephen Hoffman

2022-09-14 01:18:42 UTC

Post by Dave Froble
As Rob pointed out, one can do a lot with the DLM.
Isn't synchronization itself a form of communication?

Yes.
But not what I would call general communication.
It is not intended to send arbitrary messages - it is intended to
synchronize flows.

Doorbell locks are common—albeit when last I checked, the documentation
on the technical was, well, missing—and locks and lock value blocks are
how an app passes around state even in the event of cluster host
failures.

If you want a primary-secondary design or anything similar with a
counter or similar app-local context or related failovers, locks are
how that's done on OpenVMS.

--
Pure Personal Opinion | HoffmanLabs LLC

Richard Maher

2022-09-14 03:14:12 UTC

Post by Arne VajhÃ¸j
https://www.vajhoej.dk/arne/articles/vmsipc.html

Why no mention of mailboxs ?

And event flags and locks?

To me those are synchronization techniques not communication
techniques - or at least not general communication techniques.

As Rob pointed out, one can do a lot with the DLM.
Isn't synchronization itself a form of communication?

Yes.
But not what I would call general communication.
It is not intended to send arbitrary messages - it is intended to
synchronize flows.
Arne

Example: Trade Matching and Clearing system

Half-trades would come in as they happen.
Supplier process simply cannot be write to an ISAM file quick enough to
keep up.
If Consumer process reads past eof of matching sequential queue file it
will have to close the SEQ file, open it again and re-read the many
thousands of already processed trades to get back up to dat.

Solution: Lock Value Block

The ID of the last txn written is stored in the LVBLK.
When the Consumer obtains the lock and discovers the last txn available
is the same as the last it just processed it goes to sleep and waits for
a BLAST from the producer.

Summary: I call that interprocess communication

Dave Froble

2022-09-14 03:18:02 UTC

Post by Arne VajhÃ¸j
https://www.vajhoej.dk/arne/articles/vmsipc.html

Why no mention of mailboxs ?

And event flags and locks?

To me those are synchronization techniques not communication
techniques - or at least not general communication techniques.

As Rob pointed out, one can do a lot with the DLM.
Isn't synchronization itself a form of communication?

Yes.
But not what I would call general communication.
It is not intended to send arbitrary messages - it is
intended to synchronize flows.

From that perspective then more on your list aren't what I'd consider easy
communications. Global sections for example are some serious work. Nor would I
stoop to using files or databases for inter-process communications. Sockets and
mailboxes might be what I'd consider "low hanging fruit".

--
David Froble Tel: 724-529-0450
Dave Froble Enterprises, Inc. E-Mail: ***@tsoft-inc.com
DFE Ultralights, Inc.
170 Grimplin Road
Vanderbilt, PA 15486

Bob Gezelter

2022-09-13 09:29:19 UTC

Arne,

My compliments. I know all too well how much effort it takes to produce such a compendium.

As has been observed, mailboxes are not included.

DECnet Task-to-Task would also make a useful addition. While DECnet does not include encryption, an omission that I have been quietly outraged by for decades, when tunneled over an encrypted IP tunnel OR used locally, it is quite useful. DECnet logical links are message-based, unlike stream-based TCP sockets, which does simplify programming to an extent.

Both network-based techniques have a significant synchronization advantage over shared memory techniques. When either partner of a TCP socket or a DECnet logical link exits, the corresponding partner is automatically notified, allowing the appropriate steps to be taken. Detecting a failed correspondent is far more complex with shared memory techniques. Network links are also atomic. A message is sent or not, there are no partial messages. Shared memory approaches have the inherent danger that an update may be incomplete, corrupting the shared data structure.

My normal recommendation has been to implement using message-passing techniques, unless one can conclusively demonstrate that the performance increment achieved by shared memory can be justified.

- Bob Gezelter, http://www.rlgsc.com

Simon Clubley

2022-09-13 17:52:33 UTC

Post by Bob Gezelter
My compliments. I know all too well how much effort it takes to produce such a compendium.
As has been observed, mailboxes are not included.
DECnet Task-to-Task would also make a useful addition. While DECnet does
not include encryption, an omission that I have been quietly outraged by
for decades, when tunneled over an encrypted IP tunnel OR used locally, it
is quite useful. DECnet logical links are message-based, unlike
stream-based TCP sockets, which does simplify programming to an extent.

However, encryption only protects the data while it is in transit. It does
nothing to stop attacks against the protocol stack in the target server
itself if the client has access to the target server.

Post by Bob Gezelter
Both network-based techniques have a significant synchronization
advantage over shared memory techniques. When either partner of a TCP
socket or a DECnet logical link exits, the corresponding partner is
automatically notified, allowing the appropriate steps to be taken.
Detecting a failed correspondent is far more complex with shared memory
techniques. Network links are also atomic. A message is sent or not, there
are no partial messages. Shared memory approaches have the inherent danger
that an update may be incomplete, corrupting the shared data structure.

The no-partial-message guarantee is only true for stream protocols if you
implement something on top of the stream protocol that guarantees you have
received a full message.

In a stream protocol, just because the transmitter sent 4000 bytes, there's
no guarantee that all 4000 bytes were delivered to the receiver before
some connection interruption occurred.

Simon.

--
Simon Clubley, ***@remove_me.eisner.decus.org-Earth.UFP
Walking destinations on a map are further away than they appear.

Bob Gezelter

2022-09-13 20:23:54 UTC

The no-partial-message guarantee is only true for stream protocols if you
implement something on top of the stream protocol that guarantees you have
received a full message.
In a stream protocol, just because the transmitter sent 4000 bytes, there's
no guarantee that all 4000 bytes were delivered to the receiver before
some connection interruption occurred.
Simon.
--
Walking destinations on a map are further away than they appear.

Simon,

I mentioned message framing, although may phrasing may have been very soft. Of course, when using a stream connection, there is no mapping between QIO on the sender and the receive at the recipient. The only guarantee is that the bytes will be delivered in order. If the connection is severed, the results are "undefined." [If memory serves, Gene Amdahl was credited with that phrasing, which appeared in the IBM Systems/360 Principles of Operation manual.]

- Bob Gezelter, http://www.rlgsc.com

Dave Froble

2022-09-13 20:59:40 UTC

Post by Bob Gezelter

The no-partial-message guarantee is only true for stream protocols if you
implement something on top of the stream protocol that guarantees you have
received a full message.
In a stream protocol, just because the transmitter sent 4000 bytes, there's
no guarantee that all 4000 bytes were delivered to the receiver before
some connection interruption occurred.
Simon.
--
Walking destinations on a map are further away than they appear.

Simon,
I mentioned message framing, although may phrasing may have been very soft. Of course, when using a stream connection, there is no mapping between QIO on the sender and the receive at the recipient. The only guarantee is that the bytes will be delivered in order. If the connection is severed, the results are "undefined." [If memory serves, Gene Amdahl was credited with that phrasing, which appeared in the IBM Systems/360 Principles of Operation manual.]

That's what "handshakes" are for.

Hey, I want to send some data, 4000 bytes ...
Ok, ready to receive data ...
Send the data, wait for a response ...
Ok, received 4000 bytes ...

Anything else would be considered a failure.

In addition, checksums and such could be included to insure the 4000 bytes sent
is the same 4000 bytes received.

--
David Froble Tel: 724-529-0450
Dave Froble Enterprises, Inc. E-Mail: ***@tsoft-inc.com
DFE Ultralights, Inc.
170 Grimplin Road
Vanderbilt, PA 15486

Stephen Hoffman

2022-09-13 22:34:18 UTC

Post by Dave Froble
That's what "handshakes" are for.
Hey, I want to send some data, 4000 bytes ...
Ok, ready to receive data ...
Send the data, wait for a response ...
Ok, received 4000 bytes ...
Anything else would be considered a failure.
In addition, checksums and such could be included to insure the 4000
bytes sent is the same 4000 bytes received.

4K is ~one (modern) disk block, but yeah.

More common to send 4K with a checksum, or 4K encrypted with a
checksum, and let the receiver figure it out, though. An
application-level TCP-style handshake is closer to what is described is
certainly workable, but the chatter greatly slows things. I'd probably
look at the QUIC handshake, or at a message-passing or transaction
framework, if I needed the extra chatter. DECdtm, or Paxos protocol,
or MQTT, and tracking the transaction state, etc. But more generally
with the use of TCP or QUIC, you will get notification of either a
successful send, partial send, or of a failure to send.

It's up to the app or the transaction manager to determine where the
failure occurred, just as it's up to app-local error processing the
trap shadow on an Alpha to determine where the error occurred, or other
sorts of errors.

Put somewhat differently, it's usually best to design the connection
for speed and simplicity and for the common cause, and to (potentially)
make the error path and error recovery path somewhat more ugly.

PS: The TCP send operation has four potential outcomes: all bytes
transferred, a partial transfer with a need to loop and send more, a
local failure with no transfer, or a connection failure from the remote
side with no data transferred. Handling this usually involves a state
machine.

PPS: one of the more common errors here with TCP and such is assuming a
1-to-1 mapping of writes to reads. TCP and QUIC don't (necessarily)
work that way. One big write can become many smaller reads, for
instance.

PPPS: Depending on the platform, QUIC can have a specific API:
https://developer.apple.com/documentation/network/quic_options — with
that, QUIC can be switched to datagram, for instance. And AFAIK,
OpenVMS TCP/IP Services has UDP, TCP, and SCTP, but lacks QUIC.

PPPPS: having a framework that deals with this makes this whole area
far easier to deal with, and with fewer mistakes, and with fewer
problems with encryption or authentication.

--
Pure Personal Opinion | HoffmanLabs LLC

Dave Froble

2022-09-14 00:59:57 UTC

4K is ~one (modern) disk block, but yeah.
More common to send 4K with a checksum, or 4K encrypted with a checksum, and let
the receiver figure it out, though. An application-level TCP-style handshake is
closer to what is described is certainly workable, but the chatter greatly slows
things. I'd probably look at the QUIC handshake, or at a message-passing or
transaction framework, if I needed the extra chatter. DECdtm, or Paxos
protocol, or MQTT, and tracking the transaction state, etc. But more generally
with the use of TCP or QUIC, you will get notification of either a successful
send, partial send, or of a failure to send.
It's up to the app or the transaction manager to determine where the failure
occurred, just as it's up to app-local error processing the trap shadow on an
Alpha to determine where the error occurred, or other sorts of errors.
Put somewhat differently, it's usually best to design the connection for speed
and simplicity and for the common cause, and to (potentially) make the error
path and error recovery path somewhat more ugly.
PS: The TCP send operation has four potential outcomes: all bytes transferred, a
partial transfer with a need to loop and send more, a local failure with no
transfer, or a connection failure from the remote side with no data transferred.
Handling this usually involves a state machine.
PPS: one of the more common errors here with TCP and such is assuming a 1-to-1
mapping of writes to reads. TCP and QUIC don't (necessarily) work that way. One
big write can become many smaller reads, for instance.
https://developer.apple.com/documentation/network/quic_options — with that, QUIC
can be switched to datagram, for instance. And AFAIK, OpenVMS TCP/IP Services
has UDP, TCP, and SCTP, but lacks QUIC.
PPPPS: having a framework that deals with this makes this whole area far easier
to deal with, and with fewer mistakes, and with fewer problems with encryption
or authentication.

Well, yeah, but I learned communications over a serial line ...

--
David Froble Tel: 724-529-0450
Dave Froble Enterprises, Inc. E-Mail: ***@tsoft-inc.com
DFE Ultralights, Inc.
170 Grimplin Road
Vanderbilt, PA 15486

Stephen Hoffman

2022-09-14 01:14:38 UTC

Post by Dave Froble
Well, yeah, but I learned communications over a serial line ...

I did, too.

With home-grown protocols.

With X/Y/ZMODEM.

With a whole lot of serial wiring hand-soldered together, or later with
pin-insertion, and checked with a breakout box.

With loading code from audio tape cassettes, and I'm not referring to DDS here.

And variously with busted UARTs, or with PIO activity that saturated
the processor, or other such.

With a side-trip through SCADA and PLCs, too many of the now-older PLCs
and controllers having completely demented serial protocols.

Times change.

The newer Ethernet pass-through connectors and crimping tools are
better and faster than non-passthrough crimping, and all vastly faster
and more reliable than drilling 10BASE5 taps.

We're all slinging a whole lot more data than we did back then, too.

And with vastly better tooling and apps and networking protocols available.

--
Pure Personal Opinion | HoffmanLabs LLC

Dave Froble

2022-09-14 03:22:37 UTC

Post by Dave Froble
Well, yeah, but I learned communications over a serial line ...

I did, too.
With home-grown protocols.
With X/Y/ZMODEM.
With a whole lot of serial wiring hand-soldered together, or later with
pin-insertion, and checked with a breakout box.
With loading code from audio tape cassettes, and I'm not referring to DDS here.
And variously with busted UARTs, or with PIO activity that saturated the
processor, or other such.
With a side-trip through SCADA and PLCs, too many of the now-older PLCs and
controllers having completely demented serial protocols.
Times change.
The newer Ethernet pass-through connectors and crimping tools are better and
faster than non-passthrough crimping, and all vastly faster and more reliable
than drilling 10BASE5 taps.
We're all slinging a whole lot more data than we did back then, too.
And with vastly better tooling and apps and networking protocols available.

Not arguing any of that. All I was implying was that both sides of a
communication must insure a successful communication, or, it never happened.

--
David Froble Tel: 724-529-0450
Dave Froble Enterprises, Inc. E-Mail: ***@tsoft-inc.com
DFE Ultralights, Inc.
170 Grimplin Road
Vanderbilt, PA 15486

Simon Clubley

2022-09-14 17:34:38 UTC

Post by Dave Froble
Not arguing any of that. All I was implying was that both sides of a
communication must insure a successful communication, or, it never happened.

Not that simple (for performance reasons).

Protocols such as TCP, ZMODEM, and Kermit (with sliding windows enabled)
are all asynchronous protocols where the delivery of data to the end user
program is decoupled from the sending of ACKs back to the transmitter.

The receiver end user program can easily have received a good amount of
data from the transmitter before the transmitter has received the ACKs
from the receiver to confirm receipt of that data.

You need something that guarantees the receiver end user program and
the sender end user program are in sync at all times ? You can do that
with a layer on top of the underlying protocol, but you will take a
serious performace hit for doing so.

Simon.

--
Simon Clubley, ***@remove_me.eisner.decus.org-Earth.UFP
Walking destinations on a map are further away than they appear.

Dave Froble

2022-09-14 17:40:29 UTC

Post by Dave Froble
Not arguing any of that. All I was implying was that both sides of a
communication must insure a successful communication, or, it never happened.

Not that simple (for performance reasons).
Protocols such as TCP, ZMODEM, and Kermit (with sliding windows enabled)
are all asynchronous protocols where the delivery of data to the end user
program is decoupled from the sending of ACKs back to the transmitter.
The receiver end user program can easily have received a good amount of
data from the transmitter before the transmitter has received the ACKs
from the receiver to confirm receipt of that data.
You need something that guarantees the receiver end user program and
the sender end user program are in sync at all times ? You can do that
with a layer on top of the underlying protocol, but you will take a
serious performace hit for doing so.
Simon.

What type of hit might one experience if the customer billing is say $1000 and
the last 2 zeros are chopped off the message?

Not that that is anything real. You cannot convince me that if the
communication was not verified, it is anything but "it never happened".

Accuracy is a great deal more important than performance. Actually, accuracy
isn't everything, it's the ONLY thing.

--
David Froble Tel: 724-529-0450
Dave Froble Enterprises, Inc. E-Mail: ***@tsoft-inc.com
DFE Ultralights, Inc.
170 Grimplin Road
Vanderbilt, PA 15486

Fred. Zwarts

2022-09-15 07:58:48 UTC

Not arguing any of that. All I was implying was that both sides of a
communication must insure a successful communication, or, it never happened.

Not that simple (for performance reasons).
Protocols such as TCP, ZMODEM, and Kermit (with sliding windows enabled)
are all asynchronous protocols where the delivery of data to the end user
program is decoupled from the sending of ACKs back to the transmitter.
The receiver end user program can easily have received a good amount of
data from the transmitter before the transmitter has received the ACKs
from the receiver to confirm receipt of that data.
You need something that guarantees the receiver end user program and
the sender end user program are in sync at all times ? You can do that
with a layer on top of the underlying protocol, but you will take a
serious performace hit for doing so.
Simon.

What type of hit might one experience if the customer billing is say
$1000 and the last 2 zeros are chopped off the message?
Not that that is anything real. You cannot convince me that if the
communication was not verified, it is anything but "it never happened".
Accuracy is a great deal more important than performance. Actually,
accuracy isn't everything, it's the ONLY thing.

If accuracy is the only thing, it will be impossible to do a
transaction. In theory it will need an endless series of acknowledging
the acknowledgements. In practice we usually have enough trust in the
underlying communication channels that a one or two level
acknowledgement is sufficient.
There is a trade-of between the needed accuracy and the performance. Not
all communications need the same accuracy. A damaged frame when viewing
a video is a smaller problem than a missing 0 in a bank transfer. So,
the latter will use a more accurate communication protocol. But even
then we accept an error once in a million years, because otherwise
transactions are not possible at all.

Stephen Hoffman

2022-09-13 20:41:55 UTC

Post by Bob Gezelter
As has been observed, mailboxes are not included.
DECnet Task-to-Task would also make a useful addition. While DECnet
does not include encryption, an omission that I have been quietly
outraged by for decades, when tunneled over an encrypted IP tunnel OR
used locally, it is quite useful. DECnet logical links are
message-based, unlike stream-based TCP sockets, which does simplify
programming to an extent.
Both network-based techniques have a significant synchronization
advantage over shared memory techniques. When either partner of a TCP
socket or a DECnet logical link exits, the corresponding partner is
automatically notified, allowing the appropriate steps to be taken.
Detecting a failed correspondent is far more complex with shared memory
techniques. Network links are also atomic. A message is sent or not,
there are no partial messages. Shared memory approaches have the
inherent danger that an update may be incomplete, corrupting the shared
data structure.
My normal recommendation has been to implement using message-passing
techniques, unless one can conclusively demonstrate that the
performance increment achieved by shared memory can be justified.

Network message-passing and shared-file communcations code is usually a
win, as it's common code for either local operations or for local and
remote operations, and it avoids all the "fun" with shared memory cache
coherency. If the performance of shared memory is necessary, it's best
treated as an add-on or an addition to network transfers.

OpenVMS lacks any sort of consistent communications API, with a whole
box of easy-to-make-mistakes some-assembly-required parts including
mailboxes, DECnet and IP, shared memory, DCE RPC and (IIRC) SunRPC, the
ever-popular TLS, intracluster communications services, IRPs and ACPs,
shared files, a couple of different message-passing add-on frameworks
including MQTT, etc.

Missing features here include IP and TLS integration with DCL and with
OpenVMS more generally, and preferably with frameworks intended to deal
with communications including IPv4 and IPv6, DNS, and with UDP, TCP,
QUIC, etc., support for REST, and the rest. Writing this code for
OpenVMS really reminds me of writing assembler. It's absolutely
possible to write this stuff for OpenVMS (including writing that code
in now-compiled VAX assembler), but other platforms abstract all this
for the developer. Which makes this stuff easier and faster to deploy,
and with fewer mistakes.

And preferably with connection security, data encryption, logging, and
authentication—that might not be necessary entirely locally, but I've
seen more than a few of these local YOLO pre-Y2K designs re-used for
remote connections, and where sadness ensued.

I've seen more than a few mistakes made with mailbox and DECnet and IP
communications, too. Mailboxes and DECnet and IP all have a
backpressure design which ~everybody gets wrong when they first
encounter it in production, too. Voice Of Experience: do NOT leave a
session in the debugger and head out to lunch.

And eightcubed has a wealth of source code examples.

--
Pure Personal Opinion | HoffmanLabs LLC

Arne Vajhøj

2022-09-14 00:36:22 UTC

Post by Bob Gezelter

Post by Arne VajhÃ¸j
https://www.vajhoej.dk/arne/articles/vmsipc.html

DECnet Task-to-Task would also make a useful addition.

Agree.

I had forgotten about that one.

It will go on the TODO list.

Post by Bob Gezelter
My normal recommendation has been to implement using message-passing
techniques, unless one can conclusively demonstrate that the
performance increment achieved by shared memory can be justified.

I have always liked message queues.

:-)

Arne

Marc Van Dyck

2022-09-14 09:13:30 UTC

Should layered/third party products like RTR, DEC MessageQ or even
MQ Series (and its Opensource equivalents) be mentioned in such a work
too ? Honest open question, not saying that they are missing...

--
Marc Van Dyck

Arne Vajhøj

2022-09-14 21:24:27 UTC

Post by Marc Van Dyck
Should layered/third party products like RTR, DEC MessageQ or even
MQ Series (and its Opensource equivalents) be mentioned in such a work
too ?

ActiveMQ is already there. If I were to add another message queue then
I would probably pick RabbitMQ as I think ActiveMQ/ArtemisMQ and
RabbitMQ are the two big ones today. But RabbitMQ code would be the
same as ActiveMQ code, because STOMP protocol and JMS API would
still be used. On the other side MQSeries/WebSphere MQ/IBM MQ
has a separate API besides the standard protocols/API so it
could be relevant. I just don't have access to the software.

I don't know much about RTR, but I got the impression that
it is way than a message queue - and I would probably
consider it out of scope.

Arne

Craig A. Berry

2022-09-14 22:27:47 UTC

Post by Marc Van Dyck
Should layered/third party products like RTR

I don't know much about RTR, but I got the impression that
it is way than a message queue - and I would probably
consider it out of scope.

I don't know much either but I think you are right. It does distributed
transactions like TransactionScope in .NET will do when more than one
server is involved. It may use some IPC for some of what it does, but
it's not really IPC.

Dave Froble

2022-09-14 22:30:08 UTC

Post by Marc Van Dyck
Should layered/third party products like RTR, DEC MessageQ or even
MQ Series (and its Opensource equivalents) be mentioned in such a work
too ?

ActiveMQ is already there.

Am I missing something? I thought all those were third party products?

--
David Froble Tel: 724-529-0450
Dave Froble Enterprises, Inc. E-Mail: ***@tsoft-inc.com
DFE Ultralights, Inc.
170 Grimplin Road
Vanderbilt, PA 15486

Arne Vajhøj

2022-09-14 22:45:58 UTC

Post by Marc Van Dyck
Should layered/third party products like RTR, DEC MessageQ or even
MQ Series (and its Opensource equivalents) be mentioned in such a work
too ?

ActiveMQ is already there.

Am I missing something? I thought all those were third party products?

I believe RTR is VSI, but DEC MessageQ must be Oracle today, MQSeries
is IBM and ActiveMQ is open source (but VSI does make it available
https://vmssoftware.com/products/activemq/ for Itanium).

But if it is usable from VMS applications then I may
consider it in scope.

Arne

Jan-Erik Söderholm

2022-09-15 06:44:29 UTC

Post by Marc Van Dyck
Should layered/third party products like RTR, DEC MessageQ or even
MQ Series (and its Opensource equivalents) be mentioned in such a work
too ?

ActiveMQ is already there.

Am I missing something? I thought all those were third party products?

So what? If they enable IPC on VMS they are of interest, of course.

Richard Maher

2022-09-15 06:58:17 UTC

Post by Marc Van Dyck
Should layered/third party products like RTR, DEC MessageQ or
even MQ Series (and its Opensource equivalents) be mentioned in
such a work too ?

ActiveMQ is already there.

Am I missing something? I thought all those were third party
products?

So what? If they enable IPC on VMS they are of interest, of course.

Not necessarily. Show me VMS/Redis Cache, VMS GitHub, something to load
my VMS Tier3 ASCII files to GitHub

RTR is dead
DMQ is dead
MSMQ is dead
TIBCO is dead
Rendezvous is dead
ESB is dead

*** GraphQL is still born ***

To be brutally honest
VMS is dead

If someone can show me an easy way of getting ASCII fils CR/LF into a
GitHub repository I am happy to do it

Dave Froble

2022-09-15 16:03:02 UTC

Post by Marc Van Dyck
Should layered/third party products like RTR, DEC MessageQ or
even MQ Series (and its Opensource equivalents) be mentioned in
such a work too ?

ActiveMQ is already there.

Am I missing something? I thought all those were third party products?

So what? If they enable IPC on VMS they are of interest, of course.

Not necessarily. Show me VMS/Redis Cache, VMS GitHub, something to load my VMS
Tier3 ASCII files to GitHub
RTR is dead
DMQ is dead
MSMQ is dead
TIBCO is dead
Rendezvous is dead
ESB is dead
*** GraphQL is still born ***
To be brutally honest
VMS is dead
If someone can show me an easy way of getting ASCII fils CR/LF into a GitHub
repository I am happy to do it

So Richard, if you don't use it, then it''s dead. Have I got that right?

What about the rest of us?

--
David Froble Tel: 724-529-0450
Dave Froble Enterprises, Inc. E-Mail: ***@tsoft-inc.com
DFE Ultralights, Inc.
170 Grimplin Road
Vanderbilt, PA 15486

Richard Maher

2022-09-16 02:26:22 UTC

Post by Richard Maher

Post by Marc Van Dyck
Should layered/third party products like RTR, DEC MessageQ
or even MQ Series (and its Opensource equivalents) be
mentioned in such a work too ?

ActiveMQ is already there.

Am I missing something? I thought all those were third party products?

So what? If they enable IPC on VMS they are of interest, of
course.

Not necessarily. Show me VMS/Redis Cache, VMS GitHub, something to
load my VMS Tier3 ASCII files to GitHub
RTR is dead DMQ is dead MSMQ is dead TIBCO is dead Rendezvous is
dead ESB is dead
*** GraphQL is still born ***
To be brutally honest VMS is dead
If someone can show me an easy way of getting ASCII fils CR/LF into
a GitHub repository I am happy to do it

So Richard, if you don't use it, then it''s dead. Have I got that right?
What about the rest of us?

David you should really (re)watch the Sixth Sense. You've been (brain)
dead for years :-)

Arne Vajhøj

2022-09-15 20:32:40 UTC

Post by Richard Maher

Post by Marc Van Dyck
Should layered/third party products like RTR, DEC MessageQ or
even MQ Series (and its Opensource equivalents) be mentioned in
such a work too ?

ActiveMQ is already there.

Am I missing something? I thought all those were third party products?

So what? If they enable IPC on VMS they are of interest, of course.

Not necessarily. Show me VMS/Redis Cache, VMS GitHub, something to load
my VMS Tier3 ASCII files to GitHub

Redis should be available:

https://vmssoftware.com/products/redis/

BTW, Redis would fit quite nicely into this article, so I will put that
on the TODO list.

Post by Richard Maher
RTR is dead
DMQ is dead
MSMQ is dead
TIBCO is dead
Rendezvous is dead
ESB is dead
To be brutally honest
VMS is dead

VMS is still alive. And hopefully will be so for many many
years to come.

RTR is also still alive. Not sure how much usage there is.

DEC MessageQ was bought by BEA I believe and I suspect the
product was effectively EOL'd when Oracle bought BEA.

MSMQ is still alive. Although a lot of Windows shops supposedly
use RabbitMQ today.

TIBCO rendezvous and other ESB products also still exist
even though their importance has declined.

Arne

Arne Vajhøj

2022-09-29 00:02:46 UTC

Post by Richard Maher
Not necessarily. Show me VMS/Redis Cache, VMS GitHub, something to
load my VMS Tier3 ASCII files to GitHub

https://vmssoftware.com/products/redis/
BTW, Redis would fit quite nicely into this article, so I will put that
on the TODO list.

Done.

https://www.vajhoej.dk/arne/articles/vmsipc.html#redis

Redis Cache
Concept
Demo
Hiredis (C)
Official VMS wrapper hiredis (Fortran, Pascal, Cobol, Basic)
Custom VMS wrapper hiredis (Fortran, Pascal, Cobol, Basic)
Jedis (Java, Jython)
Python module (Python)

And just to be clear: this is Redis server running on VMS and client
applications running on VMS.

Arne

Simon Clubley

2022-09-29 12:57:13 UTC

Post by Richard Maher
Not necessarily. Show me VMS/Redis Cache, VMS GitHub, something to
load my VMS Tier3 ASCII files to GitHub

https://vmssoftware.com/products/redis/
BTW, Redis would fit quite nicely into this article, so I will put that
on the TODO list.

Done.
https://www.vajhoej.dk/arne/articles/vmsipc.html#redis

$ set response/mode=good_natured

This is just a guess Arne, but have you by any chance currently got
a lot of spare time on your hands ?

:-)

Simon.

--
Simon Clubley, ***@remove_me.eisner.decus.org-Earth.UFP
Walking destinations on a map are further away than they appear.

Arne Vajhøj

2022-09-30 00:14:42 UTC

Post by Richard Maher
Not necessarily. Show me VMS/Redis Cache, VMS GitHub, something to
load my VMS Tier3 ASCII files to GitHub

https://vmssoftware.com/products/redis/
BTW, Redis would fit quite nicely into this article, so I will put that
on the TODO list.

Done.
https://www.vajhoej.dk/arne/articles/vmsipc.html#redis

$ set response/mode=good_natured
This is just a guess Arne, but have you by any chance currently got
a lot of spare time on your hands ?
:-)

Not really.

I started writing stuff in 2004.

Recently I have just written a lot about VMS.

And even though is is obviously some work, then I reuse
stuff - for Redis I had most of the API's covered from
when I wrote about cache servers in general back in 2020 - all
the tests back then was done on Windows but the C, Java and
Python API's are the same on on VMS, so I did not have to
start from scratch.

Arne

Stephen Hoffman

2022-09-15 20:35:39 UTC

Post by Richard Maher
If someone can show me an easy way of getting ASCII fils CR/LF into a
GitHub repository I am happy to do it

I prefer to convert source files and text files to Stream LF files as
part of kitting a release, and that particular issue goes alway.

Mostly.

Most OpenVMS apps will read Stream LF, but some apps do prefer not to
create that file format by default.

Which means reworking the OpenVMS environment to work around that.

With some of the available source code control tools, it's possible to
convert the sequential file format during check-in, as well as running
a pretty-printer on source files.

It'd be nice if the OpenVMS vGit git client flagged VFC and other
~incompatible file formats or reported and converted those files before
upload, but—not having verified this—I'd tend to assume it does not.

--
Pure Personal Opinion | HoffmanLabs LLC

Richard Maher

2022-09-16 02:29:28 UTC

Post by Richard Maher
If someone can show me an easy way of getting ASCII fils CR/LF into a
GitHub repository I am happy to do it

I prefer to convert source files and text files to Stream LF files as
part of kitting a release, and that particular issue goes alway.
Mostly.
Most OpenVMS apps will read Stream LF, but some apps do prefer not to
create that file format by default.
Which means reworking the OpenVMS environment to work around that.
With some of the available source code control tools, it's possible to
convert the sequential file format during check-in, as well as running a
pretty-printer on source files.
It'd be nice if the OpenVMS vGit git client flagged VFC and other
~incompatible file formats or reported and converted those files before
upload, but—not having verified this—I'd tend to assume it does not.

Ok. But what tool (that comes standard on the VMS side) do you use? FTP?

And on the Windows client? Filezilla?

Stephen Hoffman

2022-09-16 21:07:27 UTC

Post by Richard Maher

Post by Richard Maher
If someone can show me an easy way of getting ASCII fils CR/LF into a
GitHub repository I am happy to do it

I prefer to convert source files and text files to Stream LF files as
part of kitting a release, and that particular issue goes alway.
Mostly.
Most OpenVMS apps will read Stream LF, but some apps do prefer not to
create that file format by default.
Which means reworking the OpenVMS environment to work around that.
With some of the available source code control tools, it's possible to
convert the sequential file format during check-in, as well as running
a pretty-printer on source files.
It'd be nice if the OpenVMS vGit git client flagged VFC and other
~incompatible file formats or reported and converted those files before
upload, but—not having verified this—I'd tend to assume it does not.

Ok. But what tool (that comes standard on the VMS side) do you use?

To convert a VFC or other RMS sequential format into RMS sequential
format Stream LF:

$ CONVERT /FDL="RECORD; FORMAT STREAM_LF" from.txt to.txt

I've used a DCL loop with an f$file to detect and convert sequential
files not Stream LF, or similar code in the build or release or kitting
or check-in procedures to convert the file format.

Once converted, most OpenVMS tools will continue to work with the
converted file, and will preserve the Stream LF.

There's also ANALYZE /FDL and CREATE /FDL, depending on local
requirements, and fdl$create() for creating files in OpenVMS apps and
languages that don't do Stream LF well, or the FAB/RAB/XAB slog.

I haven't looked at whether any check-in hooks or format-detection
features are available in vGit, as mentioned earlier.

Post by Richard Maher
FTP?

FTP? FTP gets used grudgingly around here. And is not a tool I'd choose
to use for RMS file format conversions.

Post by Richard Maher
And on the Windows client? Filezilla?

Windows? Best ask somebody else about that.

--
Pure Personal Opinion | HoffmanLabs LLC

Arne Vajhøj

2022-10-30 22:59:05 UTC

Post by Arne VajhÃ¸j
https://www.vajhoej.dk/arne/articles/vmsipc.html
    TCP/IP Socket
        Concept
        Demo
        Java API (Java)
        C API (C)
        Python API (Python, Jython)
        Wrapper C API (Fortran, Pascal, Cobol, Basic)

Just added:

HTTP
Concept
Demo
Embedded server (Java, Python)
Java API (Java)
urllib3 (Python)
Curl (C, PHP)
Direct socket (C)
Wrapper direct socket (Fortran, Pascal, Cobol, Basic)

Arne

Simon Clubley

2022-10-31 13:18:47 UTC

Post by Arne VajhÃ¸j
https://www.vajhoej.dk/arne/articles/vmsipc.html

A suggestion: you should add a link back to the main entry point on your
website to the top of each of these pages. There's no immediate obvious
way to get to all of your website when you go in at a specific page such
as this one.

Post by Arne VajhÃ¸j
    TCP/IP Socket
        Concept
        Demo
        Java API (Java)
        C API (C)
        Python API (Python, Jython)
        Wrapper C API (Fortran, Pascal, Cobol, Basic)

HTTP
Concept
Demo
Embedded server (Java, Python)
Java API (Java)
urllib3 (Python)
Curl (C, PHP)
Direct socket (C)
Wrapper direct socket (Fortran, Pascal, Cobol, Basic)

Have you considered adding articles about 64-bit memory access programming
on VMS and its limits ? Or is everyone already (painfully :-)) aware of
the issues involved here ?

Simon.

--
Simon Clubley, ***@remove_me.eisner.decus.org-Earth.UFP
Walking destinations on a map are further away than they appear.

Arne Vajhøj

2022-11-01 01:06:02 UTC

Post by Arne VajhÃ¸j
https://www.vajhoej.dk/arne/articles/vmsipc.html

There is a link at the bottom.

HTTP
Concept
Demo
Embedded server (Java, Python)
Java API (Java)
urllib3 (Python)
Curl (C, PHP)
Direct socket (C)
Wrapper direct socket (Fortran, Pascal, Cobol, Basic)

Have you considered adding articles about 64-bit memory access programming
on VMS and its limits ? Or is everyone already (painfully :-)) aware of
the issues involved here ?

I have not considered that.

Long opinionated writing about what DEC decided 30 years ago,
why, consequences of that and, discussions about whether DEC
made the right decision and discussions about what VSI should do
now are out of scope for the series.

And from the more practical perspective I don't think there
is enough to write about. One page with background,
one page with all the stuff one can do in C and one
page about the fun of Java calling C via JNI. That
must be about it.

Arne

Arne Vajhøj

2023-03-10 21:01:35 UTC

Post by Simon Clubley
Have you considered adding articles about 64-bit memory access programming
on VMS and its limits ? Or is everyone already (painfully :-)) aware of
the issues involved here ?

I have not considered that.
Long opinionated writing about what DEC decided 30 years ago,
why, consequences of that and, discussions about whether DEC
made the right decision and discussions about what VSI should do
now are out of scope for the series.
And from the more practical perspective I don't think there
is enough to write about. One page with background,
one page with all the stuff one can do in C and one
page about the fun of Java calling C via JNI. That
must be about it.

I decided to do it anyway.

And it turned out to be more than 3 pages to write.

Pre-release for comments:

https://www.vajhoej.dk/arne/articles/vms64.html

Arne

Craig A. Berry

2023-03-10 23:09:44 UTC

Post by Simon Clubley
Have you considered adding articles about 64-bit memory access programming
on VMS and its limits ? Or is everyone already (painfully :-)) aware

I decided to do it anyway.
And it turned out to be more than 3 pages to write.
https://www.vajhoej.dk/arne/articles/vms64.html

Looks good. Thanks for doing that. You may be interested in the
following tidbits that showed up in the release notes for the new
clang++ compiler:

1.3 Differences Between C++ on OpenVMS Itanium and OpenVMS x86

The datatypes 'long', 'size_t', 'nullptr_t', 'ptrdiff_t' are 64-bits
wide on OpenVMS x86 but only 32-bits wide on OpenVMS Itanium.

The default size for pointers is 64-bits on OpenVMS x86 but only 32-bits
on OpenVMS Itanium.

Arne Vajhøj

2023-03-11 00:00:08 UTC

Post by Simon Clubley
Have you considered adding articles about 64-bit memory access programming
on VMS and its limits ? Or is everyone already (painfully :-)) aware

I decided to do it anyway.
And it turned out to be more than 3 pages to write.
https://www.vajhoej.dk/arne/articles/vms64.html

Looks good. Thanks for doing that. You may be interested in the
following tidbits that showed up in the release notes for the new
1.3 Differences Between C++ on OpenVMS Itanium and OpenVMS x86
The datatypes 'long', 'size_t', 'nullptr_t', 'ptrdiff_t' are 64-bits
wide on OpenVMS x86 but only 32-bits wide on OpenVMS Itanium.
The default size for pointers is 64-bits on OpenVMS x86 but only 32-bits
on OpenVMS Itanium.

So /POINTER=64 is default. I should make a note about that.

Curious about long being 64 bit.

So:
Alpha & Itanium x86-64
short 16 16
int 32 32
long 32 64
long long 64 ?

?

I guess it is way overdue to always use intxx_t!

:-)

Arne

Simon Clubley

2023-03-13 13:32:43 UTC

Post by Arne VajhÃ¸j
So /POINTER=64 is default. I should make a note about that.
Curious about long being 64 bit.
Alpha & Itanium x86-64
short 16 16
int 32 32
long 32 64
long long 64 ?
?
I guess it is way overdue to always use intxx_t!
:-)

Even better is to use uintXX_t unless you really want a signed integer. :-)

A big mistake languages made (IMHO) was to make their integers signed
by default, instead of you needing to ask for a signed integer if you
really needed one.

Simon.

--
Simon Clubley, ***@remove_me.eisner.decus.org-Earth.UFP
Walking destinations on a map are further away than they appear.

Arne Vajhøj

2023-03-13 13:42:54 UTC

Even better is to use uintXX_t unless you really want a signed integer. :-)
A big mistake languages made (IMHO) was to make their integers signed
by default, instead of you needing to ask for a signed integer if you
really needed one.

That is not really C specific.

Fortran INTEGER is signed. Pascal INTEGER is signed (even though Pascal
got CARDINAL for unsigned).

Java does not have unsigned integers.

Arne

Simon Clubley

2023-03-13 14:01:13 UTC

Post by Simon Clubley
Even better is to use uintXX_t unless you really want a signed integer. :-)
A big mistake languages made (IMHO) was to make their integers signed
by default, instead of you needing to ask for a signed integer if you
really needed one.

That is not really C specific.

Notice I didn't say C, but "languages". :-)

I consider it a general design flaw in languages, probably based on the
need to mirror underlying early architecture and then it just became
established as the default.

Post by Arne VajhÃ¸j
Fortran INTEGER is signed. Pascal INTEGER is signed (even though Pascal
got CARDINAL for unsigned).
Java does not have unsigned integers.

Simon.

--
Simon Clubley, ***@remove_me.eisner.decus.org-Earth.UFP
Walking destinations on a map are further away than they appear.

Arne Vajhøj

2023-03-13 14:19:49 UTC

That is not really C specific.

Notice I didn't say C, but "languages". :-)
I consider it a general design flaw in languages, probably based on the
need to mirror underlying early architecture and then it just became
established as the default.

I consider it rather natural to have integers signed by default.

Except byte that I like unsigned.

I really like C# where short, int and long are signed while
byte is unsigned - and one can pick ushort, uint, ulong and
sbyte when one want the other way.

Arne

Dave Froble

2023-03-13 19:17:41 UTC

That is not really C specific.

Notice I didn't say C, but "languages". :-)
I consider it a general design flaw in languages, probably based on the
need to mirror underlying early architecture and then it just became
established as the default.

I consider it rather natural to have integers signed by default.

The way most/all people consider math, numbers can be either positive or
negative. Yes, "natural". Computer languages were developed using this
concept. While unsigned numbers are useful, they aren't something people
normally consider. Not sure where Simon is coming from with his assertions.

Post by Arne VajhÃ¸j
Except byte that I like unsigned.
I really like C# where short, int and long are signed while
byte is unsigned - and one can pick ushort, uint, ulong and
sbyte when one want the other way.

The problem with all of those is they are vague, and are defined differently on
different HW and software. Precision is to be desired.

--
David Froble Tel: 724-529-0450
Dave Froble Enterprises, Inc. E-Mail: ***@tsoft-inc.com
DFE Ultralights, Inc.
170 Grimplin Road
Vanderbilt, PA 15486

Arne Vajhøj

2023-03-13 19:46:15 UTC

Post by Arne VajhÃ¸j
I really like C# where short, int and long are signed while
byte is unsigned - and one can pick ushort, uint, ulong and
sbyte when one want the other way.

The problem with all of those is they are vague, and are defined
differently on different HW and software. Precision is to be desired.

In C the types short, int and long are vaguely defined.

Not in C# and Java - there they are 16, 32 and 64 bit two's complement.
Only aspect left optional is big vs little endian.

Arne

Simon Clubley

2023-03-14 13:08:56 UTC

Post by Dave Froble
The way most/all people consider math, numbers can be either positive or
negative. Yes, "natural". Computer languages were developed using this
concept. While unsigned numbers are useful, they aren't something people
normally consider. Not sure where Simon is coming from with his assertions.

Much of the time you don't need signed integers and the problem you are
trying to solve is a better match with unsigned integers.

Let me put it this way: How many filesystems and file sizes and other
limits through computing history are half the maximum size they should
be because everyone used signed integers instead of unsigned integers ?

Simon.

--
Simon Clubley, ***@remove_me.eisner.decus.org-Earth.UFP
Walking destinations on a map are further away than they appear.

Dave Froble

2023-03-14 17:37:26 UTC

Much of the time you don't need signed integers and the problem you are
trying to solve is a better match with unsigned integers.
Let me put it this way: How many filesystems and file sizes and other
limits through computing history are half the maximum size they should
be because everyone used signed integers instead of unsigned integers ?
Simon.

Understand, I think unsigned numbers would be very useful. I wish Basic support
such. But my point is, they are not so "natural" to humans, and after all,
humans are the users of computers. So development followed what people do.

--
David Froble Tel: 724-529-0450
Dave Froble Enterprises, Inc. E-Mail: ***@tsoft-inc.com
DFE Ultralights, Inc.
170 Grimplin Road
Vanderbilt, PA 15486

Arne Vajhøj

2023-03-14 18:05:17 UTC

Much of the time you don't need signed integers and the problem you are
trying to solve is a better match with unsigned integers.
Let me put it this way: How many filesystems and file sizes and other
limits through computing history are half the maximum size they should
be because everyone used signed integers instead of unsigned integers ?

When it involves data size and the demand grow x2 every 2 years or so,
then that extra bit will not save they day for long.

Understand, I think unsigned numbers would be very useful. I wish Basic
support such.

I miss them in Java as well.

But my point is, they are not so "natural" to humans, and
after all, humans are the users of computers. So development followed
what people do.

If you stop 100 random people on the street and ask about integers, then
99 will include negative numbers, that is the common definition in
math/english/danish/whatever.

Those that remember their math will remember the term natural
numbers to exclude negative numbers.

Arne

Arne Vajhøj

2023-03-14 18:08:28 UTC

Much of the time you don't need signed integers and the problem you are
trying to solve is a better match with unsigned integers.
Let me put it this way: How many filesystems and file sizes and other
limits through computing history are half the maximum size they should
be because everyone used signed integers instead of unsigned integers ?

When it involves data size and the demand grow x2 every 2 years or so,
then that extra bit will not save they day for long.

Understand, I think unsigned numbers would be very useful. I wish
Basic support such.

I miss them in Java as well.

But my point is, they are not so "natural" to humans,
and after all, humans are the users of computers. So development
followed what people do.

If you stop 100 random people on the street and ask about integers, then
99 will include negative numbers, that is the common definition in
math/english/danish/whatever.
Those that remember their math will remember the term natural
numbers to exclude negative numbers.

Note that it is really highly unlikely that a given
problem domain actually need types:

-2147483648..2147483647
0..4294967295

It is way more likely to have a need for types like:

1..100
0..99
-10..10
etc.

But that is not in fashion today (Pascal, Modula-2,
Oberon, Ada etc. are rare today).

Arne

Simon Clubley

2023-03-14 18:31:46 UTC

Post by Arne VajhÃ¸j
1..100
0..99
-10..10
etc.
But that is not in fashion today (Pascal, Modula-2,
Oberon, Ada etc. are rare today).

Unfortunately. :-(

Are there any languages other than the Wirth-inspired ones which do
have them as part of the core language ?

BTW, given what the Rust people claim about Rust's target markets,
I consider that to be a surprising omission from Rust, given that they
had all these languages to look at when they were designing Rust.

Such types are used to help model the underlying problem more accurately,
which is exactly the kind of thing the Rust people (and any other such
language designers) should have latched onto.

Simon.

--
Simon Clubley, ***@remove_me.eisner.decus.org-Earth.UFP
Walking destinations on a map are further away than they appear.

Arne Vajhøj

2023-03-14 18:44:11 UTC

Post by Arne VajhÃ¸j
1..100
0..99
-10..10
etc.
But that is not in fashion today (Pascal, Modula-2,
Oberon, Ada etc. are rare today).

Unfortunately. :-(
Are there any languages other than the Wirth-inspired ones which do
have them as part of the core language ?

Not that I am aware of.

Post by Simon Clubley
BTW, given what the Rust people claim about Rust's target markets,
I consider that to be a surprising omission from Rust, given that they
had all these languages to look at when they were designing Rust.

A large chunk of Rust target market is the low level
near to the hardware market and there it may actually
be the requirement that there is N bits to fill out.

But in the much bigger business related market then there is
rarely a bit requirements - most likely those defining the
requirements do not know what a bit is.

Arne

bill

2023-03-14 18:53:01 UTC

Post by Arne VajhÃ¸j
If you stop 100 random people on the street and ask about integers, then
99 will include negative numbers,

If you stop 100 random people on the street and ask about integers, then
99 won't have a clue what you are talking about.

bill

Arne Vajhøj

2023-03-14 19:00:18 UTC

Post by Arne VajhÃ¸j
If you stop 100 random people on the street and ask about integers, then
99 will include negative numbers,

If you stop 100 random people on the street and ask about integers, then
99 won't have a clue what you are talking about.

Those that remember their math should think of Z.

The rest should vaguely remember that "numbers without decimals" are
called integers.

Arne

Johnny Billquist

2023-03-14 13:24:06 UTC

That is not really C specific.

Notice I didn't say C, but "languages". :-)
I consider it a general design flaw in languages, probably based on the
need to mirror underlying early architecture and then it just became
established as the default.

I don't think there is anything in the underlying architecture that
leads one way or another on this topic (no matter which architecture).

I think it's much more likely that the chose to have them signed as most
people when they work with integers expect them to be able to represent
both positive and negative numbers.

Johnny

John Dallman

2023-03-13 13:40:00 UTC

Post by Craig A. Berry
Looks good. Thanks for doing that. You may be interested in the
following tidbits that showed up in the release notes for the new
1.3 Differences Between C++ on OpenVMS Itanium and OpenVMS x86
The datatypes 'long', 'size_t', 'nullptr_t', 'ptrdiff_t' are 64-bits
wide on OpenVMS x86 but only 32-bits wide on OpenVMS Itanium.

Have you got a link for that?

John

Craig A. Berry

2023-03-13 18:14:52 UTC

Have you got a link for that?

As far as I know it's only available in the service portal for people
with a support contract. I assume it will become part of the standard,
publicly-available docs once that compiler is out of field test.

John Dallman

2023-03-13 18:53:00 UTC

Post by John Dallman
Have you got a link for that?

As far as I know it's only available in the service portal for
people with a support contract. I assume it will become part of the
standard, publicly-available docs once that compiler is out of field
test.

Makes sense, thanks. It makes porting C/C++ applications to VMS a good
deal simpler.

John

Arne Vajhøj

2023-03-13 19:08:32 UTC

Post by John Dallman
Have you got a link for that?

As far as I know it's only available in the service portal for
people with a support contract. I assume it will become part of the
standard, publicly-available docs once that compiler is out of field
test.

Makes sense, thanks. It makes porting C/C++ applications to VMS a good
deal simpler.

I believe it always have been the message that:
* Fortran, Pascal, Cobol, Basic and C will get all the DCL command
qualifiers, VMS specific language extensions and VMS specific CRTL
functions needed to compile old existing code
* C++ would be clang ported to VMS close to "as is"

When such a high level idea meet reality, then things can change
a little.

But it makes sense. There are probably not that many old
VMS specific C++ applications out there. For the simple
reason that when C++ for VMS showed up and became
reasonable stable, then VMS was no longer a common
platform choice for new applications.

But there are a lot of stuff developed on *nix and Windows
that need a C++ compiler with support for newer C++ standards
to be able to build on VMS. Including latest versions of
MySQL/MariaDB and OpenJDK.

Arne

John Reagan

2023-03-13 22:14:09 UTC

Post by John Dallman
Have you got a link for that?

As far as I know it's only available in the service portal for
people with a support contract. I assume it will become part of the
standard, publicly-available docs once that compiler is out of field
test.

Makes sense, thanks. It makes porting C/C++ applications to VMS a good
deal simpler.

* Fortran, Pascal, Cobol, Basic and C will get all the DCL command
qualifiers, VMS specific language extensions and VMS specific CRTL
functions needed to compile old existing code
* C++ would be clang ported to VMS close to "as is"

There has been a constant drumbeat from customers (including many here
on this forum) about our odd choice of long=32-bit; ptrdiff_t=32-bit;
size_t=32-bit all while you can say /POINTER=64. For C++/clang, we just
get what clang gives you for a 64-bit compiler.

We have yet to provide a similar setting for the C frontend as so far it hasn't been
necessary.

We have modified/enhanced some headers to replace "long" with "int" to make sure
that certain fields that are 32-bits are still 32-bits with both x86 C and x86 C++.

My desire is to limit the # of clang OpenVMS-isms to those actually needed and to
skip the ones that have been just dragged forward from VAX for no real reason other
than just the fear of pissing off a customer. I'm in the "if you need it, ask and we'll
talk about it" camp.

John Dallman

2023-03-14 10:22:00 UTC

For C++/clang, we just get what clang gives you for a 64-bit compiler.
We have yet to provide a similar setting for the C frontend as so
far it hasn't been necessary.

Makes sense. If I ever get to port my employers' product to x86-64
OpenVMS, I'll need the Clang C setup to match the Clang C++ setup.
However, there hasn't been any interest in VMS from customers since the
late 1990s, so I fear the odds are against this happening.

John

John Reagan

2023-03-13 22:08:28 UTC

Post by John Dallman
Have you got a link for that?

As far as I know it's only available in the service portal for
people with a support contract. I assume it will become part of the
standard, publicly-available docs once that compiler is out of field
test.

Makes sense, thanks. It makes porting C/C++ applications to VMS a good
deal simpler.
John

That /USAGE qualifier is a Pascal qualifier. It asks the Pascal compiler to violate
the Calling Standard and not sign-extend a 32-bit pointer to duplicate what you can
get (by accident) on Itanium.

It wouldn't be a C or C++ qualifier since neither of those compilers do anything with
descriptors behind your back.

Arne Vajhøj

2023-03-14 01:04:51 UTC

Post by John Reagan
That /USAGE qualifier is a Pascal qualifier. It asks the Pascal compiler to violate
the Calling Standard and not sign-extend a 32-bit pointer to duplicate what you can
get (by accident) on Itanium.

Is it x86-64 only?

The value is not accepted with Pascal 6.2 on neither Alpha nor Itanium.

Arne

John Reagan

2023-03-14 02:06:04 UTC

That /USAGE qualifier is a Pascal qualifier. It asks the Pascal compiler to violate
the Calling Standard and not sign-extend a 32-bit pointer to duplicate what you can
get (by accident) on Itanium.

Is it x86-64 only?
The value is not accepted with Pascal 6.2 on neither Alpha nor Itanium.
Arne

There hasn't been a Alpha or Itanium release with it. Since neither Alpha or Itanium
put code into P2 space that first chunk of memory is where a 64-bit NEW would
get memory from. With code in P2 space on x86, that shoves the 64-bit heap
up more and you need to get help from the compiler. It has never been a good thing
to pass around P2 addresses in the 32-bit field of descriptors.

Andreas Gruhl

2023-03-13 09:56:27 UTC

Post by Simon Clubley
Have you considered adding articles about 64-bit memory access programming
on VMS and its limits ? Or is everyone already (painfully :-)) aware of
the issues involved here ?

I have not considered that.
Long opinionated writing about what DEC decided 30 years ago,
why, consequences of that and, discussions about whether DEC
made the right decision and discussions about what VSI should do
now are out of scope for the series.
And from the more practical perspective I don't think there
is enough to write about. One page with background,
one page with all the stuff one can do in C and one
page about the fun of Java calling C via JNI. That
must be about it.

I decided to do it anyway.
And it turned out to be more than 3 pages to write.
https://www.vajhoej.dk/arne/articles/vms64.html
Arne

Thanks for your effort.

I like to add, that you can successfully access the lower 2 GB of P2
(%X00000000 80000000..%X00000000 FFFFFFFF) with 32 bit
pointers and descriptors.
You only have to treat the pointers as unsigned instead of signed
avoiding them to be interpreted as S0/S1 addresses.
We successfully use this in Pascal, which is missing 64-Bit descriptors.
The caller of a routine can then simply pass a structure in lower P2
without any special adaptions (except having to use the compiler
qualifier /USAGE=64BIT_TO_DESCR).
The routine itself has to do some additional work. It must extract the
structure's address from the descriptor and copy it to a 64-Bit Pointer
WITHOUT sign extension.
This makes the structure directly accessible although it resides in P2.

I wished treating descriptor pointers as unsigned would be the default
behaviour. The reason is simple: You will find extremely few application
programmers who ever needed to access an S0 or S1 structure via
descriptor - but many who would like to access P2 data.

Treating 32-bit pointers as unsigned is probably possible in any language
available under VMS.

Andreas

Arne Vajhøj

2023-03-13 17:58:48 UTC

Post by Andreas Gruhl

Post by Arne VajhÃ¸j
https://www.vajhoej.dk/arne/articles/vms64.html

I like to add, that you can successfully access the lower 2 GB of P2
(%X00000000 80000000..%X00000000 FFFFFFFF) with 32 bit
pointers and descriptors.
You only have to treat the pointers as unsigned instead of signed
avoiding them to be interpreted as S0/S1 addresses.
We successfully use this in Pascal, which is missing 64-Bit descriptors.
The caller of a routine can then simply pass a structure in lower P2
without any special adaptions (except having to use the compiler
qualifier /USAGE=64BIT_TO_DESCR).
The routine itself has to do some additional work. It must extract the
structure's address from the descriptor and copy it to a 64-Bit Pointer
WITHOUT sign extension.
This makes the structure directly accessible although it resides in P2.

Is /USAGE=64BIT_TO_DESCR new for x86-64?

Arne

JOUKJ

2023-03-13 10:22:29 UTC

Post by Simon Clubley
Have you considered adding articles about 64-bit memory access programming
on VMS and its limits ? Or is everyone already (painfully :-)) aware of
the issues involved here ?

I have not considered that.
Long opinionated writing about what DEC decided 30 years ago,
why, consequences of that and, discussions about whether DEC
made the right decision and discussions about what VSI should do
now are out of scope for the series.
And from the more practical perspective I don't think there
is enough to write about. One page with background,
one page with all the stuff one can do in C and one
page about the fun of Java calling C via JNI. That
must be about it.

I decided to do it anyway.
And it turned out to be more than 3 pages to write.
https://www.vajhoej.dk/arne/articles/vms64.html
Arne

I maintain a web-page on 64-bit pointers when using VSI-Fortran. The
site contains many problems you can encounter and what the work-arounds are.

Jouk

JOUKJ

2023-03-13 10:24:15 UTC

Post by JOUKJ
I maintain a web-page on 64-bit pointers when using VSI-Fortran. The
site contains many problems you can encounter and what the work-arounds are.

I forgot to copy the link:

http://nchrem.tnw.tudelft.nl/openvms/fortran64.html

Jan-Erik Söderholm

2023-03-13 12:39:17 UTC

Post by Arne VajhÃ¸j
https://www.vajhoej.dk/arne/articles/vms64.html
Arne

A bit pre-dated also... :-)

Article history:
Version Date Description
0.9 March 10th 20223 Pre-release

Dave Froble

2023-03-13 19:22:02 UTC

Post by Arne VajhÃ¸j
https://www.vajhoej.dk/arne/articles/vms64.html
Arne

A bit pre-dated also... :-)
Version Date Description
0.9 March 10th 20223 Pre-release

Ah, the extreme power and capability of a keystroke, even time travel.

:-)

--
David Froble Tel: 724-529-0450
Dave Froble Enterprises, Inc. E-Mail: ***@tsoft-inc.com
DFE Ultralights, Inc.
170 Grimplin Road
Vanderbilt, PA 15486

Simon Clubley

2023-03-13 13:28:12 UTC