Discussion:
MATH$RANDOM for OpenVMS x86
Add Reply
John Reagan
2017-03-31 15:31:26 UTC
Reply
Permalink
Raw Message
It is Friday and I'm in a good mood. I figured I'll toss something out for a friendly discussion.

For Alpha and Itanium, the MATH$RANDOMxxx functions use exactly the same algorithm as VAX. Is is a rather simple piece of code that is pseudo-random. If you have them the same seed value, you'll get the same sequence over and over. Of course some of you compute your initial seed with some randomness (time of day, bits out of some system cell, a hash of the tweets by the President, etc.) that at least makes you get different sequences on different runs.

For x86-64, recent chips have real random number generators,

7.3.17.1 RDRAND

The RDRAND instruction returns a random number. All Intel processors that
support the RDRAND instruction indicate the availability of the RDRAND
instruction via reporting CPUID.01H:ECX.RDRAND[bit 30] = 1.

RDRAND returns random numbers that are supplied by a cryptographically secure,
deterministic random bit generator DRBG. The DRBG is designed to meet the NIST
SP 800-90A standard. The DRBG is re-seeded frequently from an on-chip
non-deterministic entropy source to guarantee data returned by RDRAND is statistically uniform, nonperiodic and non-deterministic.

7.3.17.2 RDSEED

The RDSEED instruction returns a random number. All Intel processors that
support the RDSEED instruction indicate the availability of the RDSEED
instruction via reporting CPUID.(EAX=07H, ECX=0H):EBX.RDSEED[bit 18] = 1.

RDSEED returns random numbers that are supplied by a cryptographically secure,
enhanced non-deterministic random bit generator (Enhanced NRBG). The NRBG is
designed to meet the NIST SP 800-90B and NIST SP800-90C
standards.


So should we switch MTH$RANDOM/MATH$RANDOM to use the on-chip, much-better, random number generator RDSEED? The downside is that even if start with the same seed value (we'd essentially ignore the seed value), you'll get different sequences every single time. We'll still have to extract bits to form the floating result of course.

For reference, here's the current algorithm:

28 ** Abstract:
29 **
30 ** This file is used to generate object modules for vms_rand.
31 ** The interface is: F_TYPE y = vms_rand(unsigned int *seed).
32 **
33 ** The routine updates seed with
34 ** seed = (69069 * seed + 1) mod (2 ^ 32)
35 ** and then uses the high order bits to form a floating
36 ** point value. The goal of this routine is compatibility
37 ** with the VAX/VMS mth$random routine.
Steven Schweda
2017-03-31 15:57:21 UTC
Reply
Permalink
Raw Message
So should we switch [...]
The principle of least surprise might lead to a DECC$
option to allow a choice. Or add a whole different function,
and make the victim cause his own problems. It would
certainly make sense to provide the capability. It might
make less sense to mandate it.
Robert A. Brooks
2017-03-31 16:04:11 UTC
Reply
Permalink
Raw Message
Post by Steven Schweda
So should we switch [...]
The principle of least surprise might lead to a DECC$
option to allow a choice.
[...]
The sound you hear is John's head exploding at the thought of a feature
logical to control this behaviour.
--
-- Rob
V***@SendSpamHere.ORG
2017-03-31 17:38:14 UTC
Reply
Permalink
Raw Message
Post by Robert A. Brooks
Post by Steven Schweda
So should we switch [...]
The principle of least surprise might lead to a DECC$
option to allow a choice.
[...]
The sound you hear is John's head exploding at the thought of a feature
logical to control this behaviour.
LOL!

If this internal random doesn't require a seed, perhaps it would be sage
to keep the M(A)TH$RANDOM behavior as it is and only opt for the internal
if a seed value is omitted???

Relying on M(A)TH$RANDOM to produce a repeatable sequence is, obvioulsy,
the definition of randomness... NOT!
--
VAXman- A Bored Certified VMS Kernel Mode Hacker VAXman(at)TMESIS(dot)ORG

I speak to machines with the voice of humanity.
David Froble
2017-03-31 20:27:48 UTC
Reply
Permalink
Raw Message
Post by Robert A. Brooks
Post by Steven Schweda
So should we switch [...]
The principle of least surprise might lead to a DECC$
option to allow a choice.
[...]
The sound you hear is John's head exploding at the thought of a feature
logical to control this behaviour.
Really? I thought it was Steve when I heard the explosion ...

:-)

Thinking about it a bit more, Basic has such:

Option Size = ( Integer Word , Real Double )

Now, the question I'd ask, why does C use logicals, rather than options within
the actual code?
Steven Schweda
2017-03-31 21:26:38 UTC
Reply
Permalink
Raw Message
Post by David Froble
Really? I thought it was Steve when I heard the explosion ...
That was my expectation. But clearly, "DECC$" was wrong.
For this, we clearly need a set of M[A]TH$ logical jive.

Or a new function with the new functionality.
Post by David Froble
Option Size = ( Integer Word , Real Double )
Now, the question I'd ask, why does C use logicals, rather
than options within the actual code?
The question I'd ask is what that BASIC stuff does, and
what you think that the DECC$whatever jive does. As I read
the HELP, they're not very similar.

One reason for the DECC$whatever logical jive is to aid in
porting foreign code to VMS, where standard CRTL functions
may behave in ways different from those on other systems.
(Or in ways different from the ways on old VMS systems.)

Those of us who get out very little may see little use for
such mechanisms, but some of us do bump into file names which
require ODS5 features, and many C programs do not imagine
that ";" has any special significance in a file spec. Among
many other things.
Stephen Hoffman
2017-03-31 22:31:38 UTC
Reply
Permalink
Raw Message
Post by David Froble
Now, the question I'd ask, why does C use logicals, rather than options
within the actual code?
There is an API, but it's not useful for all of the settings. This
because some of the settings have to be accessed before the main code
in the executable gets rolling, which can mean entirely out-of-band
settings ahead of the image activation, or code in the initialize
psect, for instance. There's no preferences mechanism in OpenVMS,
so... everybody does their own. Fallout from these and related
decisions — the lack of a standard preferences or configuration file,
or warring requirements for settings even within a single image, or the
inconsistencies within various implementations, for instance — will
haunt the platform for the foreseeable future.

Similar sorts of inevitable fallout from not moving to a CPRNG, whether
for reasons of performance or compatibility or otherwise.
--
Pure Personal Opinion | HoffmanLabs LLC
John Reagan
2017-04-01 02:39:08 UTC
Reply
Permalink
Raw Message
Post by David Froble
Post by Robert A. Brooks
Post by Steven Schweda
So should we switch [...]
The principle of least surprise might lead to a DECC$
option to allow a choice.
[...]
The sound you hear is John's head exploding at the thought of a feature
logical to control this behaviour.
Really? I thought it was Steve when I heard the explosion ...
:-)
Option Size = ( Integer Word , Real Double )
Now, the question I'd ask, why does C use logicals, rather than options within
the actual code?
Those options control the compiler behavior for generating code. The DECC$ logicals control the RTL's behavior. They could have been encoded in the source program, but each routine would then have to call some RTL interface to initialize its desired behavior. Such behavior would also have to be saved/restored around routine calls. Sounds expensive.
John Reagan
2017-03-31 16:51:36 UTC
Reply
Permalink
Raw Message
Post by Steven Schweda
So should we switch [...]
The principle of least surprise might lead to a DECC$
option to allow a choice. Or add a whole different function,
and make the victim cause his own problems. It would
certainly make sense to provide the capability. It might
make less sense to mandate it.
I said a FRIENDLY discussion! :) :) :)

It wouldn't be a DECC$ logical since it impacts any caller. Some languages besides C have builtins as well.

Linux's /dev/random and /dev/urandom work from an entropy pool in the OS. There is no seed, there is no repeatability. Other folks use openssl to generate random numbers as those hash functions are almost certain to have more entropy than the language library versions (which aren't suitable for serious crypto)

I can just leave them alone and let people who care find their own solutions.
Richard Maher
2017-03-31 23:39:52 UTC
Reply
Permalink
Raw Message
Post by John Reagan
Post by Steven Schweda
So should we switch [...]
The principle of least surprise might lead to a DECC$
option to allow a choice. Or add a whole different function,
and make the victim cause his own problems. It would
certainly make sense to provide the capability. It might
make less sense to mandate it.
I said a FRIENDLY discussion! :) :) :)
It wouldn't be a DECC$ logical since it impacts any caller. Some languages besides C have builtins as well.
Linux's /dev/random and /dev/urandom work from an entropy pool in the OS. There is no seed, there is no repeatability. Other folks use openssl to generate random numbers as those hash functions are almost certain to have more entropy than the language library versions (which aren't suitable for serious crypto)
I can just leave them alone and let people who care find their own solutions.
Precedent been set with lib$wait behaviour change (sys$hiber user)
Richard Maher
2017-04-01 05:22:22 UTC
Reply
Permalink
Raw Message
The answer
http://h41379.www4.hpe.com/doc/82final/5932/5932pro_054.html

Flags? Float-type?

Hello darkness my old friend
I come to talk with you again. . .
Richard Maher
2017-04-02 12:10:12 UTC
Reply
Permalink
Raw Message
Post by Richard Maher
The answer
http://h41379.www4.hpe.com/doc/82final/5932/5932pro_054.html
Flags? Float-type?
Hello darkness my old friend
I come to talk with you again. . .
https://au.pinterest.com/pin/351421577150292980/

Can someone look up the lib$wait nowake change authors so I can right to
them and let them know that all you guys and especially John Regan think
their a bunch of fucking numpties who did it so wrong that you have
spent man weeks trying to imagine a correction to their solution?
Bob Koehler
2017-03-31 18:11:32 UTC
Reply
Permalink
Raw Message
Post by Steven Schweda
So should we switch [...]
The principle of least surprise might lead to a DECC$
option to allow a choice. Or add a whole different function,
and make the victim cause his own problems. It would
certainly make sense to provide the capability. It might
make less sense to mandate it.
It doesn't look like a C specific function, so DECC$ is the wrong
prefix. But either a new to get the random number, ot a new
API to control the behaviour.

This one doesn't scream out as needed so bad that there should be
breaking of compatability.
David Froble
2017-03-31 16:04:50 UTC
Reply
Permalink
Raw Message
Post by John Reagan
It is Friday and I'm in a good mood. I figured I'll toss something out for a friendly discussion.
For Alpha and Itanium, the MATH$RANDOMxxx functions use exactly the same algorithm as VAX. Is is a rather simple piece of code that is pseudo-random. If you have them the same seed value, you'll get the same sequence over and over. Of course some of you compute your initial seed with some randomness (time of day, bits out of some system cell, a hash of the tweets by the President, etc.) that at least makes you get different sequences on different runs.
For x86-64, recent chips have real random number generators,
Sounds like a no brainer to me. The only time there might be a problem is if
someone used a random number generator, but expected to know the result. For
that fictional (hopefully) person, they more than most deserve this change.

:-)

Really, if you got a better way to do something, then just do it ...
abrsvc
2017-03-31 16:13:19 UTC
Reply
Permalink
Raw Message
One thing to keep in mind is that all "random" number generators are not truly random. That being said, anyone that does performance testing, usually prefers to keep the "random" sequence the same given the same initial seed. This makes comparison studies easier as the sequence remains the same.

How the behavior is controlled makes no difference to me, but I believe that having the same sequence generated with the same seed should be an available option.

If you examine the different types of generators available (Knuth's book comes to mind), all of them create the same sequence when given the same seed. This behavior should remain.

Dan
Warren Kahle
2017-03-31 16:26:40 UTC
Reply
Permalink
Raw Message
It sounds like a better way to me and I always like the better way.

*Warren Kahle* CISSP
CSA CSE Security+
PointSecure Technologies Inc.
Phone: 713-868-1222
Cell: 713-906-5600
***@pointsecure.com

On Fri, Mar 31, 2017 at 11:04 AM, David Froble via Info-vax <
Post by David Froble
Post by John Reagan
It is Friday and I'm in a good mood. I figured I'll toss something out
for a friendly discussion.
For Alpha and Itanium, the MATH$RANDOMxxx functions use exactly the same
algorithm as VAX. Is is a rather simple piece of code that is
pseudo-random. If you have them the same seed value, you'll get the same
sequence over and over. Of course some of you compute your initial seed
with some randomness (time of day, bits out of some system cell, a hash of
the tweets by the President, etc.) that at least makes you get different
sequences on different runs.
For x86-64, recent chips have real random number generators,
Sounds like a no brainer to me. The only time there might be a problem is
if someone used a random number generator, but expected to know the
result. For that fictional (hopefully) person, they more than most deserve
this change.
:-)
Really, if you got a better way to do something, then just do it ...
_______________________________________________
Info-vax mailing list
http://rbnsn.com/mailman/listinfo/info-vax_rbnsn.com
Bob Gezelter
2017-03-31 16:47:11 UTC
Reply
Permalink
Raw Message
Post by John Reagan
It is Friday and I'm in a good mood. I figured I'll toss something out for a friendly discussion.
For Alpha and Itanium, the MATH$RANDOMxxx functions use exactly the same algorithm as VAX. Is is a rather simple piece of code that is pseudo-random. If you have them the same seed value, you'll get the same sequence over and over. Of course some of you compute your initial seed with some randomness (time of day, bits out of some system cell, a hash of the tweets by the President, etc.) that at least makes you get different sequences on different runs.
For x86-64, recent chips have real random number generators,
7.3.17.1 RDRAND
The RDRAND instruction returns a random number. All Intel processors that
support the RDRAND instruction indicate the availability of the RDRAND
instruction via reporting CPUID.01H:ECX.RDRAND[bit 30] = 1.
RDRAND returns random numbers that are supplied by a cryptographically secure,
deterministic random bit generator DRBG. The DRBG is designed to meet the NIST
SP 800-90A standard. The DRBG is re-seeded frequently from an on-chip
non-deterministic entropy source to guarantee data returned by RDRAND is statistically uniform, nonperiodic and non-deterministic.
7.3.17.2 RDSEED
The RDSEED instruction returns a random number. All Intel processors that
support the RDSEED instruction indicate the availability of the RDSEED
instruction via reporting CPUID.(EAX=07H, ECX=0H):EBX.RDSEED[bit 18] = 1.
RDSEED returns random numbers that are supplied by a cryptographically secure,
enhanced non-deterministic random bit generator (Enhanced NRBG). The NRBG is
designed to meet the NIST SP 800-90B and NIST SP800-90C
standards.
So should we switch MTH$RANDOM/MATH$RANDOM to use the on-chip, much-better, random number generator RDSEED? The downside is that even if start with the same seed value (we'd essentially ignore the seed value), you'll get different sequences every single time. We'll still have to extract bits to form the floating result of course.
29 **
30 ** This file is used to generate object modules for vms_rand.
31 ** The interface is: F_TYPE y = vms_rand(unsigned int *seed).
32 **
33 ** The routine updates seed with
34 ** seed = (69069 * seed + 1) mod (2 ^ 32)
35 ** and then uses the high order bits to form a floating
36 ** point value. The goal of this routine is compatibility
37 ** with the VAX/VMS mth$random routine.
John,

For compatibility reasons, I would suggest keeping MTH$RANDOM the same way. I know of many people who have used Linear Congruential Random Number Generators (for a description, see Knuth's Art of Computer Programming, Volume 2), together with inputting the seed, to get predictable sequences of pseudo random numbers. The resulting numbers are admittedly pseudorandom, but reproducible run-to-run with the same seed, which is useful when debugging.

That said, having an alternative function, say MTH$HRDRANDOM, which accesses the hardware Pseudo Random Number Generator.

If someone wants to switch the MTH$RANDOM calls to the hardware provided generator, a simple macro definition or LINKER directive will accomplish the task. The documented behavior of MTH$RANDOM would remain the default (and compatible with existing code bases).

- Bob Gezelter, http://www.rlgsc.com
John Reagan
2017-03-31 16:54:02 UTC
Reply
Permalink
Raw Message
Post by Bob Gezelter
Post by John Reagan
It is Friday and I'm in a good mood. I figured I'll toss something out for a friendly discussion.
For Alpha and Itanium, the MATH$RANDOMxxx functions use exactly the same algorithm as VAX. Is is a rather simple piece of code that is pseudo-random. If you have them the same seed value, you'll get the same sequence over and over. Of course some of you compute your initial seed with some randomness (time of day, bits out of some system cell, a hash of the tweets by the President, etc.) that at least makes you get different sequences on different runs.
For x86-64, recent chips have real random number generators,
7.3.17.1 RDRAND
The RDRAND instruction returns a random number. All Intel processors that
support the RDRAND instruction indicate the availability of the RDRAND
instruction via reporting CPUID.01H:ECX.RDRAND[bit 30] = 1.
RDRAND returns random numbers that are supplied by a cryptographically secure,
deterministic random bit generator DRBG. The DRBG is designed to meet the NIST
SP 800-90A standard. The DRBG is re-seeded frequently from an on-chip
non-deterministic entropy source to guarantee data returned by RDRAND is statistically uniform, nonperiodic and non-deterministic.
7.3.17.2 RDSEED
The RDSEED instruction returns a random number. All Intel processors that
support the RDSEED instruction indicate the availability of the RDSEED
instruction via reporting CPUID.(EAX=07H, ECX=0H):EBX.RDSEED[bit 18] = 1.
RDSEED returns random numbers that are supplied by a cryptographically secure,
enhanced non-deterministic random bit generator (Enhanced NRBG). The NRBG is
designed to meet the NIST SP 800-90B and NIST SP800-90C
standards.
So should we switch MTH$RANDOM/MATH$RANDOM to use the on-chip, much-better, random number generator RDSEED? The downside is that even if start with the same seed value (we'd essentially ignore the seed value), you'll get different sequences every single time. We'll still have to extract bits to form the floating result of course.
29 **
30 ** This file is used to generate object modules for vms_rand.
31 ** The interface is: F_TYPE y = vms_rand(unsigned int *seed).
32 **
33 ** The routine updates seed with
34 ** seed = (69069 * seed + 1) mod (2 ^ 32)
35 ** and then uses the high order bits to form a floating
36 ** point value. The goal of this routine is compatibility
37 ** with the VAX/VMS mth$random routine.
John,
For compatibility reasons, I would suggest keeping MTH$RANDOM the same way. I know of many people who have used Linear Congruential Random Number Generators (for a description, see Knuth's Art of Computer Programming, Volume 2), together with inputting the seed, to get predictable sequences of pseudo random numbers. The resulting numbers are admittedly pseudorandom, but reproducible run-to-run with the same seed, which is useful when debugging.
That said, having an alternative function, say MTH$HRDRANDOM, which accesses the hardware Pseudo Random Number Generator.
If someone wants to switch the MTH$RANDOM calls to the hardware provided generator, a simple macro definition or LINKER directive will accomplish the task. The documented behavior of MTH$RANDOM would remain the default (and compatible with existing code bases).
- Bob Gezelter, http://www.rlgsc.com
Seems like a reasonable compromise.

This was just something that popped into my head when I read a link on Facebook about random numbers (plus listening to Security Now podcast)
George Cornelius
2017-04-02 06:31:08 UTC
Reply
Permalink
Raw Message
Post by John Reagan
It is Friday and I'm in a good mood. I figured I'll toss something out f
or a friendly discussion.
For Alpha and Itanium, the MATH$RANDOMxxx functions use exactly the same
algorithm as VAX. Is is a rather simple piece of code that is pseudo-rando
m. If you have them the same seed value, you'll get the same sequence over
and over. Of course some of you compute your initial seed with some rando
mness (time of day, bits out of some system cell, a hash of the tweets by t
he President, etc.) that at least makes you get different sequences on diff
erent runs.
[...]
For compatibility reasons, I would suggest keeping MTH$RANDOM the same way.=
I know of many people who have used Linear Congruential Random Number Gene=
rators (for a description, see Knuth's Art of Computer Programming, Volume =
2), together with inputting the seed, to get predictable sequences of pseud=
o random numbers. The resulting numbers are admittedly pseudorandom, but re=
producible run-to-run with the same seed, which is useful when debugging.=
Yes, and Knuth mentions the particular multiplier 69069 that mth$random
was eventually implemented to use, and, partially because it was so
easy to remember, said that you might consider it the best of all
possible multipliers for 32 bit pseudo-randmon numbers. [Now, coming
from the 36-bit world, he actually considered 32 bit random numbers
to be somewhat flawed to begin with].
That said, having an alternative function, say MTH$HRDRANDOM, which accesse=
s the hardware Pseudo Random Number Generator.
If someone wants to switch the MTH$RANDOM calls to the hardware provided ge=
nerator, a simple macro definition or LINKER directive will accomplish the =
task. The documented behavior of MTH$RANDOM would remain the default (and c=
ompatible with existing code bases).
I tend to agree with this.

Those who claim we should not have repeatable random number
generators forget tht during development, especially, repeatability
is extremely important. Also for reporting problems with software.

If a problem only happens on odd Thursdays after a full moon on
in the first week in March following an Easter Sunday, it's
kind of hard to reproduce if you have to deal with an injection
of true randomness as well.

And as a general principle, I disagree with Intel's approach.
Randomly injecting randomness into an otherwise pseudorandom
generator sounds like trying to correct a bad design by a
periodic hack. Is the hack due to the fact that "purely
random" generators actually are not anything of the kind?

I guess I can still remember when Rand Corporation published
a book of random numbers (or random digits, perhaps), and
then had to publish a means for correcting for the systematic
errors that were eventually found. Turns out their source
of true randomness actually wasn't.

George
- Bob Gezelter, http://www.rlgsc.com
Steven Schweda
2017-04-02 13:32:15 UTC
Reply
Permalink
Raw Message
Post by George Cornelius
I guess I can still remember when Rand Corporation published
a book of random numbers (or random digits, perhaps), [...]
I was too young in 1955 to remember the publication of "A
Million Random Digits with 100,000 Normal Deviates", but I
did pick it up at my college library. It's been a while,
but, as I recall, the introduction was quite interesting
(much more than the body of the text).
George Cornelius
2017-04-03 02:44:13 UTC
Reply
Permalink
Raw Message
Post by Steven Schweda
Post by George Cornelius
I guess I can still remember when Rand Corporation published
a book of random numbers (or random digits, perhaps), [...]
I was too young in 1955 to remember the publication of "A
Million Random Digits with 100,000 Normal Deviates", but I
did pick it up at my college library. It's been a while,
but, as I recall, the introduction was quite interesting
(much more than the body of the text).
Yeah, same here. I just must have thought it was new when I
first ran across it. And it was new to _me_ in the mid 60's.

I suppose now you're going to tell me I wasn't actually around
when they invented fire, either!

George

P.S. I guess I cannot really claim that anyone's "pure random"
source actually is not - but I think there has been a history that
doesn't bode well for those making such claims, especially since all
tests for "true" randomness are ultimately doomed to failure.
Simon Clubley
2017-03-31 17:17:51 UTC
Reply
Permalink
Raw Message
Post by John Reagan
So should we switch MTH$RANDOM/MATH$RANDOM to use the on-chip, much-better,
random number generator RDSEED?
Provided the output is not intended to be used in anything that has
security implications, then that would be acceptable.

However, modifying code in VMS to use RDRAND/RDSEED as a source of
cryptographically secure numbers would be a very bad idea indeed.

Please read this for the background on why this would be such a bad idea:

https://en.wikipedia.org/wiki/RdRand#Reception

Ever since the Snowden revelations, I simply do not trust pure
hardware random number generators any more.

Simon.
--
Simon Clubley, ***@remove_me.eisner.decus.org-Earth.UFP
Microsoft: Bringing you 1980s technology to a 21st century world
John Reagan
2017-03-31 17:38:51 UTC
Reply
Permalink
Raw Message
Post by Simon Clubley
Post by John Reagan
So should we switch MTH$RANDOM/MATH$RANDOM to use the on-chip, much-better,
random number generator RDSEED?
Provided the output is not intended to be used in anything that has
security implications, then that would be acceptable.
However, modifying code in VMS to use RDRAND/RDSEED as a source of
cryptographically secure numbers would be a very bad idea indeed.
https://en.wikipedia.org/wiki/RdRand#Reception
Ever since the Snowden revelations, I simply do not trust pure
hardware random number generators any more.
Simon.
--
Microsoft: Bringing you 1980s technology to a 21st century world
Thanks for that. I wasn't intentionally trying to make MATH$RANDOM cryptographically secure, only a better generator. After following some links, reading about Yarrow, et al., I think that I might just leave the existing routines alone and make sure we provide builtin access to the RDRAND/RDSEED instructions so C programmers can access them and then do what they wish...

How about the AES instructions? Need C builtins for those?

Brian, don't worry, I'll make sure Macro32 can get to them also.
Bob Gezelter
2017-03-31 19:35:38 UTC
Reply
Permalink
Raw Message
Post by John Reagan
Post by Simon Clubley
Post by John Reagan
So should we switch MTH$RANDOM/MATH$RANDOM to use the on-chip, much-better,
random number generator RDSEED?
Provided the output is not intended to be used in anything that has
security implications, then that would be acceptable.
However, modifying code in VMS to use RDRAND/RDSEED as a source of
cryptographically secure numbers would be a very bad idea indeed.
https://en.wikipedia.org/wiki/RdRand#Reception
Ever since the Snowden revelations, I simply do not trust pure
hardware random number generators any more.
Simon.
--
Microsoft: Bringing you 1980s technology to a 21st century world
Thanks for that. I wasn't intentionally trying to make MATH$RANDOM cryptographically secure, only a better generator. After following some links, reading about Yarrow, et al., I think that I might just leave the existing routines alone and make sure we provide builtin access to the RDRAND/RDSEED instructions so C programmers can access them and then do what they wish...
How about the AES instructions? Need C builtins for those?
Brian, don't worry, I'll make sure Macro32 can get to them also.
John,

The AES functions are different from the RDRAND related question.

The AES functions are deterministic (for the same inputs, they produce the same outputs). I do not have the time to check, but I would expect that Intel has taken the time to have them certified to the applicable US government standards.

They should be far faster than the equivalent software routines. They should be available as intrinsics, certainly from C and MACRO32. Encryption is worth some focus, it is being required in more and more contexts. OpenVMS will need the performance benefits of specialized hardware implementations.

In addition to the instrinsic support, there should be corresponding LIB$ equivalents.

- Bob Gezelter, http://www.rlgsc.com
David Froble
2017-03-31 20:38:05 UTC
Reply
Permalink
Raw Message
Post by Bob Gezelter
Post by John Reagan
Post by Simon Clubley
Post by John Reagan
So should we switch MTH$RANDOM/MATH$RANDOM to use the on-chip, much-better,
random number generator RDSEED?
Provided the output is not intended to be used in anything that has
security implications, then that would be acceptable.
However, modifying code in VMS to use RDRAND/RDSEED as a source of
cryptographically secure numbers would be a very bad idea indeed.
https://en.wikipedia.org/wiki/RdRand#Reception
Ever since the Snowden revelations, I simply do not trust pure
hardware random number generators any more.
Simon.
--
Microsoft: Bringing you 1980s technology to a 21st century world
Thanks for that. I wasn't intentionally trying to make MATH$RANDOM cryptographically secure, only a better generator. After following some links, reading about Yarrow, et al., I think that I might just leave the existing routines alone and make sure we provide builtin access to the RDRAND/RDSEED instructions so C programmers can access them and then do what they wish...
I just have a problem with anyone complaining they don't get a random number
when that's what they asked for? Doesn't make any sense. Not talking about
security, just a random number.
Post by Bob Gezelter
Post by John Reagan
How about the AES instructions? Need C builtins for those?
Brian, don't worry, I'll make sure Macro32 can get to them also.
Sure, us Basic, Fortran, and such people are just 2nd, or 3rd, class, don't
worry about us ....
Post by Bob Gezelter
John,
The AES functions are different from the RDRAND related question.
The AES functions are deterministic (for the same inputs, they produce the same outputs). I do not have the time to check, but I would expect that Intel has taken the time to have them certified to the applicable US government standards.
They should be far faster than the equivalent software routines. They should be available as intrinsics, certainly from C and MACRO32. Encryption is worth some focus, it is being required in more and more contexts. OpenVMS will need the performance benefits of specialized hardware implementations.
In addition to the instrinsic support, there should be corresponding LIB$ equivalents.
Now, that is maybe the first good suggestion ....

And C programmers can call the LIB$ function, just like the rest of us.

:-)
Simon Clubley
2017-04-02 23:30:13 UTC
Reply
Permalink
Raw Message
Post by David Froble
I just have a problem with anyone complaining they don't get a random number
when that's what they asked for? Doesn't make any sense. Not talking about
security, just a random number.
It's the _quality_ of the random number that matters when it comes
to cryptographic applications and you also need to be able to have
confidence in the random number generator itself.

For the reasons I mentioned, I have now lost confidence in any
hardware based random number generators such as those embedded
in some CPUs.

Simon.
--
Simon Clubley, ***@remove_me.eisner.decus.org-Earth.UFP
Microsoft: Bringing you 1980s technology to a 21st century world
Arne Vajhøj
2017-04-03 00:06:53 UTC
Reply
Permalink
Raw Message
Post by Simon Clubley
Post by David Froble
I just have a problem with anyone complaining they don't get a random number
when that's what they asked for? Doesn't make any sense. Not talking about
security, just a random number.
It's the _quality_ of the random number that matters when it comes
to cryptographic applications and you also need to be able to have
confidence in the random number generator itself.
Cryptographic RNG's have different requirements than ordinary RNG's.

I am not sure that I would make it a quality matter. An RNG can
be great for simulation and poor for cryptography or the other
way around.

Arne
David Froble
2017-04-03 01:43:33 UTC
Reply
Permalink
Raw Message
Post by Arne Vajhøj
Post by Simon Clubley
Post by David Froble
I just have a problem with anyone complaining they don't get a random number
when that's what they asked for? Doesn't make any sense. Not talking about
security, just a random number.
It's the _quality_ of the random number that matters when it comes
to cryptographic applications and you also need to be able to have
confidence in the random number generator itself.
Cryptographic RNG's have different requirements than ordinary RNG's.
I am not sure that I would make it a quality matter. An RNG can
be great for simulation and poor for cryptography or the other
way around.
As I wrote, I wasn't addressing crypto stuff.

For example, if I wanted to shuffle a deck of cards, do I want it shuffled to
the same state every time? Just an example, Ok?
Arne Vajhøj
2017-04-03 01:54:01 UTC
Reply
Permalink
Raw Message
Post by David Froble
Post by Arne Vajhøj
Post by Simon Clubley
Post by David Froble
I just have a problem with anyone complaining they don't get a random number
when that's what they asked for? Doesn't make any sense. Not talking about
security, just a random number.
It's the _quality_ of the random number that matters when it comes
to cryptographic applications and you also need to be able to have
confidence in the random number generator itself.
Cryptographic RNG's have different requirements than ordinary RNG's.
I am not sure that I would make it a quality matter. An RNG can
be great for simulation and poor for cryptography or the other
way around.
As I wrote, I wasn't addressing crypto stuff.
Correct. Simon briougth that up in what I was replying to.
Post by David Froble
For example, if I wanted to shuffle a deck of cards, do I want it
shuffled to the same state every time? Just an example, Ok?
If you need to do a single "random" shuffle then an ordinary RNG
with a simple external seed would be fine.

If you need to do simulation of some card game including shuffling,
then an ordinary RNG is fine.

But if you need to shuffle cards for a game with real players, then
you need a cryptographic RNG because you need the same
characteristics as cryptography needs.

Arne
Richard Levitte
2017-04-03 06:41:24 UTC
Reply
Permalink
Raw Message
Post by Arne Vajhøj
Post by David Froble
Post by Arne Vajhøj
Post by Simon Clubley
Post by David Froble
I just have a problem with anyone complaining they don't get a random number
when that's what they asked for? Doesn't make any sense. Not talking about
security, just a random number.
It's the _quality_ of the random number that matters when it comes
to cryptographic applications and you also need to be able to have
confidence in the random number generator itself.
Cryptographic RNG's have different requirements than ordinary RNG's.
I am not sure that I would make it a quality matter. An RNG can
be great for simulation and poor for cryptography or the other
way around.
As I wrote, I wasn't addressing crypto stuff.
Correct. Simon briougth that up in what I was replying to.
Post by David Froble
For example, if I wanted to shuffle a deck of cards, do I want it
shuffled to the same state every time? Just an example, Ok?
If you need to do a single "random" shuffle then an ordinary RNG
with a simple external seed would be fine.
If you need to do simulation of some card game including shuffling,
then an ordinary RNG is fine.
But if you need to shuffle cards for a game with real players, then
you need a cryptographic RNG because you need the same
characteristics as cryptography needs.
So, uhmmm, this discussion is actually a bit perplexing... While I do understand that MTH$RANDOM has a documented algorithm and breaking that is breaking the "social contract", I do not understand the whole discussion on "ordinary RNG" vs "cryptographic RNG", 'cause really, if you're asking for "just a random number" without much thought on quality and end up getting a superb cryptographically secure random number, are you really going to complain? Or even care? Srsly?

Cheers,
Richard ( wouldn't consider complaining about getting a nice file mignon when having asked for a $2 hamburger )
Arne Vajhøj
2017-04-04 00:06:13 UTC
Reply
Permalink
Raw Message
Post by Richard Levitte
Post by Arne Vajhøj
Post by David Froble
For example, if I wanted to shuffle a deck of cards, do I want
it shuffled to the same state every time? Just an example, Ok?
If you need to do a single "random" shuffle then an ordinary RNG
with a simple external seed would be fine.
If you need to do simulation of some card game including
shuffling, then an ordinary RNG is fine.
But if you need to shuffle cards for a game with real players,
then you need a cryptographic RNG because you need the same
characteristics as cryptography needs.
So, uhmmm, this discussion is actually a bit perplexing... While I
do understand that MTH$RANDOM has a documented algorithm and breaking
that is breaking the "social contract", I do not understand the whole
discussion on "ordinary RNG" vs "cryptographic RNG", 'cause really,
if you're asking for "just a random number" without much thought on
quality and end up getting a superb cryptographically secure random
number, are you really going to complain? Or even care? Srsly?
( wouldn't consider complaining about getting a nice
file mignon when having asked for a $2 hamburger )
The price is likely not the same. Cryptographic secure
RNG's are usually significant slower than ordinary RNG's.

Besides that then I will be slightly concerned whether a given
cryptographic secure RNG has as good statistical characteristics
as an ordinary RNG. Cryptographic secure RNG's are designed
to be unpredictable and have a good distribution. Ordinary
RNG's are designed just for good distribution. Cryptographic
secure RNG's are typical used for only small data. Ordinary
RNG's are often used for large data (simulation). I see it as
a risk using something in a way it was not intended for, because
the risk of some problem not having been found yet is greater.

To make an analogy in plain encryption. AES256 and RSA2048 are
both considered secure for what they are used for. But what
would you say if someone suggested encrypting MB of data
with RSA2048 (let us assume IV, proper block chaining etc.)?
I guess you would be skeptical. That usage is not what RSA
was created for. You can not prove that it is weaker than
AES, but the concept is sort of suspect.

Arne
Richard Levitte
2017-04-04 08:13:31 UTC
Reply
Permalink
Raw Message
Post by Arne Vajhøj
Post by Richard Levitte
Post by Arne Vajhøj
Post by David Froble
For example, if I wanted to shuffle a deck of cards, do I want
it shuffled to the same state every time? Just an example, Ok?
If you need to do a single "random" shuffle then an ordinary RNG
with a simple external seed would be fine.
If you need to do simulation of some card game including
shuffling, then an ordinary RNG is fine.
But if you need to shuffle cards for a game with real players,
then you need a cryptographic RNG because you need the same
characteristics as cryptography needs.
So, uhmmm, this discussion is actually a bit perplexing... While I
do understand that MTH$RANDOM has a documented algorithm and breaking
that is breaking the "social contract", I do not understand the whole
discussion on "ordinary RNG" vs "cryptographic RNG", 'cause really,
if you're asking for "just a random number" without much thought on
quality and end up getting a superb cryptographically secure random
number, are you really going to complain? Or even care? Srsly?
( wouldn't consider complaining about getting a nice
file mignon when having asked for a $2 hamburger )
The price is likely not the same. Cryptographic secure
RNG's are usually significant slower than ordinary RNG's.
Uhmmm, wasn't this thread started with a discussion around the instruction RDRAND and RDSEED, which I presume are reasonably fast (?), taking the issue of speed off the table?
Post by Arne Vajhøj
Besides that then I will be slightly concerned whether a given
cryptographic secure RNG has as good statistical characteristics
as an ordinary RNG. Cryptographic secure RNG's are designed
to be unpredictable and have a good distribution. Ordinary
RNG's are designed just for good distribution. Cryptographic
secure RNG's are typical used for only small data. Ordinary
RNG's are often used for large data (simulation). I see it as
a risk using something in a way it was not intended for, because
the risk of some problem not having been found yet is greater.
I might be wrong here, but I've always imagined that large uses of random bits often use a small entropy source as seed, and then calculate randomness from those. Unless I'm mistaken, that's the relationship between /dev/random ("true" entropy source that will block if you pick too much from it at a time) and /dev/urandom (pseudorandom bits that you can pick huge amounts of)...

Of course, this should mean that we could benefit from two sources on VMS as well, one being more a "true" random source of such is available, the other being calculated random bits, both hopefully with better quality than what MTH$RANDOM delivers.

But coming back to instructions like RDRAND and RDSEED, I'm unsure that there is as much a need to differentiate between "true" random bits that you can only pull small quantities from and pseudorandom bits that you can pull large amounts from. Technology does move forward after all...
(point in this case, OpenSSL 1.1.0 and on has deprecated RAND_pseudo_bytes(), because no implementation really makes any difference between that and RAND_bytes(). In the internal implementation, the only difference between the two is an error message (!))
Post by Arne Vajhøj
To make an analogy in plain encryption. AES256 and RSA2048 are
both considered secure for what they are used for. But what
would you say if someone suggested encrypting MB of data
with RSA2048 (let us assume IV, proper block chaining etc.)?
I guess you would be skeptical.
I would be more than sceptical, considering you can only encrypt messages with RSA that are as large as the key itself.
Post by Arne Vajhøj
That usage is not what RSA
was created for. You can not prove that it is weaker than
AES, but the concept is sort of suspect.
Cheers,
Richard
Arne Vajhøj
2017-04-04 14:51:53 UTC
Reply
Permalink
Raw Message
Post by Richard Levitte
Post by Richard Levitte
Post by Arne Vajhøj
Post by David Froble
For example, if I wanted to shuffle a deck of cards, do I
want it shuffled to the same state every time? Just an
example, Ok?
If you need to do a single "random" shuffle then an ordinary
RNG with a simple external seed would be fine.
If you need to do simulation of some card game including
shuffling, then an ordinary RNG is fine.
But if you need to shuffle cards for a game with real players,
then you need a cryptographic RNG because you need the same
characteristics as cryptography needs.
So, uhmmm, this discussion is actually a bit perplexing... While
I do understand that MTH$RANDOM has a documented algorithm and
breaking that is breaking the "social contract", I do not
understand the whole discussion on "ordinary RNG" vs
"cryptographic RNG", 'cause really, if you're asking for "just a
random number" without much thought on quality and end up getting
a superb cryptographically secure random number, are you really
going to complain? Or even care? Srsly?
( wouldn't consider complaining about getting a nice file mignon
when having asked for a $2 hamburger )
The price is likely not the same. Cryptographic secure RNG's are
usually significant slower than ordinary RNG's.
Uhmmm, wasn't this thread started with a discussion around the
instruction RDRAND and RDSEED, which I presume are reasonably fast
(?), taking the issue of speed off the table?
But coming back to instructions like RDRAND and RDSEED, I'm unsure
that there is as much a need to differentiate between "true" random
bits that you can only pull small quantities from and pseudorandom
bits that you can pull large amounts from. Technology does move
forward after all... (point in this case, OpenSSL 1.1.0 and on has
deprecated RAND_pseudo_bytes(), because no implementation really
makes any difference between that and RAND_bytes(). In the internal
implementation, the only difference between the two is an error
message (!))
Maybe RDRAND will change things.

But it does not seem to be an established fact yet.
Post by Richard Levitte
To make an analogy in plain encryption. AES256 and RSA2048 are both
considered secure for what they are used for. But what would you
say if someone suggested encrypting MB of data with RSA2048 (let us
assume IV, proper block chaining etc.)? I guess you would be
skeptical.
I would be more than sceptical, considering you can only encrypt
messages with RSA that are as large as the key itself.
????

#let us assume IV, proper block chaining etc.

Arne
Richard Levitte
2017-04-05 08:10:56 UTC
Reply
Permalink
Raw Message
Post by Arne Vajhøj
Post by Richard Levitte
To make an analogy in plain encryption. AES256 and RSA2048 are both
considered secure for what they are used for. But what would you
say if someone suggested encrypting MB of data with RSA2048 (let us
assume IV, proper block chaining etc.)? I guess you would be
skeptical.
I would be more than sceptical, considering you can only encrypt
messages with RSA that are as large as the key itself.
????
#let us assume IV, proper block chaining etc.
That doesn't apply to RSA. It does apply to symmetric ciphers such as AES256.

So, quick explanation: when you "encrypt with RSA", what really happens is roughly this:

1. a random key for a symmetric cipher (say, AES256) is created.
2. that random key is encrypted with RSA using the recipients public key. Let's call the result K.
3. the MB of data is encrypted using the symmetric cipher (AES256 in this case), using K as the encryption key. Let's call the result C.
4. both K and C are delivered to the recipient.

Here's a quick tutorial showing exactly this using OpenSSL commands:

https://raymii.org/s/tutorials/Encrypt_and_decrypt_files_to_public_keys_via_the_OpenSSL_Command_Line.html

Cheers,
Richard

Stephen Hoffman
2017-04-04 16:18:54 UTC
Reply
Permalink
Raw Message
Post by Richard Levitte
I might be wrong here, but I've always imagined that large uses of
random bits often use a small entropy source as seed, and then
calculate randomness from those. Unless I'm mistaken, that's the
relationship between /dev/random ("true" entropy source that will block
if you pick too much from it at a time) and /dev/urandom (pseudorandom
bits that you can pick huge amounts of)...
http://www.2uo.de/myths-about-urandom/

Use /dev/urandom

Both are the same CPRNG, too. Which also means that Linux system and
application code all gets CPRNG values by default, too.

On OpenVMS, the most obvious APIs are PRNGs.

OpenVMS has no easy path to add entropy hardware, hopefully the pool is
seeded with RDRAND or otherwise as available, and that crypto hardware
API mentioned earlier in the thread or analogous would be a nice
distant-future-release addition. OpenVMS lacks arc4random or other
common C features in the C library, as well. Best option — for folks
that know to find and use it, and the glue code — is the OpenSSL CPRNG,
or the OpenSSL arc4random bits, etc OpenSSL also accesses RDRAND when
that's available, so the OpenVMS x86-64 port should get access to that.

Having arc4random (chacha20) in the C library and a similar CPRNG
behind MATH$RANDOM would make this a whole lot easier for application
developers, and mean that most OpenVMS application code had and used a
CPRNG. JR already has a long list of C functions to add. Hauling
over BSD C library functions piecemeal — as I've been doing for various
projects — is never fun. But I digress.

Some background on OpenSSL and random number generation:
https://wiki.openssl.org/index.php/Random_Numbers

For those that prefer video presentations, one on arc4random from a few
years ago.

--
Pure Personal Opinion | HoffmanLabs LLC
Stephen Hoffman
2017-04-04 14:42:54 UTC
Reply
Permalink
Raw Message
Cryptographic secure RNG's are usually significant slower than ordinary RNG's.
How about we optimize later? Once it's proven to be an issue for an
app, and if the server itself is sufficiently overloaded for this to
matter?

Default behaviors that are secure are better.

Documentation that is more approachable is better.

Handling folks examples that are correct and consistent and secure is better.

Working to make an operating system more approachable and effective and
easier to manage is better.

Assisting less-experienced users in more complex tasks – random number
generation, network security, integrated certificates, use of common
terminology, cookbooks — is better.

Interfaces with better defaults, less complexity and easier (and
secure) installations and configurations are better.
--
Pure Personal Opinion | HoffmanLabs LLC
Stephen Hoffman
2017-04-03 16:17:12 UTC
Reply
Permalink
Raw Message
Post by David Froble
Post by Arne Vajhøj
Post by David Froble
I just have a problem with anyone complaining they don't get a random
number when that's what they asked for? Doesn't make any sense. Not
talking about security, just a random number.
It's the _quality_ of the random number that matters when it comes to
cryptographic applications and you also need to be able to have
confidence in the random number generator itself.
Cryptographic RNG's have different requirements than ordinary RNG's.
I am not sure that I would make it a quality matter. An RNG can be
great for simulation and poor for cryptography or the other way around.
As I wrote, I wasn't addressing crypto stuff.
For example, if I wanted to shuffle a deck of cards, do I want it
shuffled to the same state every time? Just an example, Ok?
A CPRNG is an equivalent or better choice than a PRNG, absent some
specific performance requirements or other app-specific details. So
if the choice is returning a CPRNG or a PRNG, there'd better be a good
reason to select and use a PRNG. Does this matter if you're using
this to shuffle a deck of cards in a simple game? No. Can use of a
non-CPRNG burn a dev badly in other contexts? Oh, yes. Yes, a PRNG
can burn. Badly.

Apps and devs needing and using random numbers don't always use
appropriate generators, and sometimes the random number generators
contain bugs:
https://www.reddit.com/r/StLouis/comments/5sgf3c/russians_engineer_a_brilliant_slot_machine_cheat/

https://nakedsecurity.sophos.com/2008/05/16/debian-ubuntu-users-beware-of-the-openssl-vulnerability/


Errors and even the need for randomness can sometimes be subtle, too.
As the Wehrmacht and others have learned over the years, whether with
their serial numbers, or with user account numbers, bill or invoice
numbers, database identifiers, etc.
https://en.wikipedia.org/wiki/German_tank_problem Folks don't always
spot the ways that they're leaking potentially-sensitive data.

Defaulting to a CPRNG takes away one more potential mistake a dev can make.

If we're in the
real-and-always-expert-developers-willfully-writing-glue-code-using-only-responsibly-sourced-artisanal-glacial-random-bits,
well, we're not in that era anymore. If we ever were.

(C)PRNG: (Cryptographic) Pseudo Random Number Generator
--
Pure Personal Opinion | HoffmanLabs LLC
John Reagan
2017-04-01 02:40:40 UTC
Reply
Permalink
Raw Message
Post by Bob Gezelter
Post by John Reagan
Post by Simon Clubley
Post by John Reagan
So should we switch MTH$RANDOM/MATH$RANDOM to use the on-chip, much-better,
random number generator RDSEED?
Provided the output is not intended to be used in anything that has
security implications, then that would be acceptable.
However, modifying code in VMS to use RDRAND/RDSEED as a source of
cryptographically secure numbers would be a very bad idea indeed.
https://en.wikipedia.org/wiki/RdRand#Reception
Ever since the Snowden revelations, I simply do not trust pure
hardware random number generators any more.
Simon.
--
Microsoft: Bringing you 1980s technology to a 21st century world
Thanks for that. I wasn't intentionally trying to make MATH$RANDOM cryptographically secure, only a better generator. After following some links, reading about Yarrow, et al., I think that I might just leave the existing routines alone and make sure we provide builtin access to the RDRAND/RDSEED instructions so C programmers can access them and then do what they wish...
How about the AES instructions? Need C builtins for those?
Brian, don't worry, I'll make sure Macro32 can get to them also.
John,
The AES functions are different from the RDRAND related question.
I know. I was changing topics. When I was in the NonStop group porting to x86, we had several customers who wanted access to the AES instructions from C. I was just wondering if OpenVMS customers would be interested in that also.
t***@glaver.org
2017-04-01 02:55:50 UTC
Reply
Permalink
Raw Message
Post by John Reagan
I know. I was changing topics. When I was in the NonStop group porting to x86, we had several customers who wanted access to the AES instructions from C. I was just wondering if OpenVMS customers would be interested in that also.
Will the compiler(s) support both inline asm("foo") and standalone .S files? If so, users should be able to use those to get to any [unprivileged] instructions they need. That might be a reasonable low-workload (for VSI) solution for the initial release(s). If it turns out that VSI or users are porting code from Linux or elsewhere that expects these to be available as language or library features, then they can be added later on.
Stephen Hoffman
2017-04-01 19:10:51 UTC
Reply
Permalink
Raw Message
Post by John Reagan
I know. I was changing topics. When I was in the NonStop group
porting to x86, we had several customers who wanted access to the AES
instructions from C. I was just wondering if OpenVMS customers would
be interested in that also.
Um... Isn't VSI going to want the crypto performance provided by that
access? For use within VSI products, such as VSI IP or ENCRYPT or
OpenSSL, or for OpenVMS itself as security and networking and
encryption all (inevitably) gets rolled into the base product?

As for clang or gcc, see the x86intrin.h header. Might serve as a
starting point, if that clang has already doesn't work directly.
https://clang.llvm.org/doxygen/x86intrin_8h_source.html
http://stackoverflow.com/questions/11228855/header-files-for-x86-simd-intrinsics/22291538#22291538


Related discussions:
http://codereview.stackexchange.com/questions/147656/checking-if-cpu-supports-rdrand

https://github.com/BrianGladman/AES/blob/master/aes_ni.c

As for interfacing with any local crypto-related hardware that might be
available, an application crypto framework such as that available via
OpenBSD is increasingly typical. I certainly don't expect such to be
available on OpenVMS in the next five years, though.
https://www.openbsd.org/papers/ocf.pdf
--
Pure Personal Opinion | HoffmanLabs LLC
David Wade
2017-04-01 09:06:01 UTC
Reply
Permalink
Raw Message
Post by John Reagan
Post by Simon Clubley
Post by John Reagan
So should we switch MTH$RANDOM/MATH$RANDOM to use the on-chip, much-better,
random number generator RDSEED?
Provided the output is not intended to be used in anything that has
security implications, then that would be acceptable.
However, modifying code in VMS to use RDRAND/RDSEED as a source of
cryptographically secure numbers would be a very bad idea indeed.
https://en.wikipedia.org/wiki/RdRand#Reception
Ever since the Snowden revelations, I simply do not trust pure
hardware random number generators any more.
Simon.
--
Microsoft: Bringing you 1980s technology to a 21st century world
Thanks for that. I wasn't intentionally trying to make MATH$RANDOM cryptographically secure, only a better generator. After following some links, reading about Yarrow, et al., I think that I might just leave the existing routines alone and make sure we provide builtin access to the RDRAND/RDSEED instructions so C programmers can access them and then do what they wish...
How about the AES instructions? Need C builtins for those?
Brian, don't worry, I'll make sure Macro32 can get to them also.
Some existing code may rely on the fact its a bad generator.

Dave
Paul Sture
2017-04-01 09:38:23 UTC
Reply
Permalink
Raw Message
Post by David Wade
Some existing code may rely on the fact its a bad generator.
Back in the day I used the Fortran random number routine to generate a
repeatable set of numbers. This was useful for re-running the same
tests on different hardware configurations or with tuning tweaks.

From the "Programming VAX FORTRAN" manual (1984):

"RAN Returns the next number from a sequence of pseudo random numbers
of uniform distribution over the range 0 to 1."

which reminds me that the gcc compiler also mentions uniform distribution.

<https://gcc.gnu.org/onlinedocs/gfortran/RAND.html>

"RAND(FLAG) returns a pseudo-random number from a uniform distribution
between 0 and 1. If FLAG is 0, the next number in the current sequence
is returned; if FLAG is 1, the generator is restarted by CALL SRAND(0);
if FLAG has any other value, it is used as a new seed with SRAND."

Do the hardware random number generators also provide a uniform
distribution?
--
A supercomputer is a device for turning compute-bound problems into
I/O-bound problems. ---Ken Batcher
Paul Sture
2017-04-01 09:52:35 UTC
Reply
Permalink
Raw Message
Post by Paul Sture
Post by David Wade
Some existing code may rely on the fact its a bad generator.
Back in the day I used the Fortran random number routine to generate a
repeatable set of numbers. This was useful for re-running the same
tests on different hardware configurations or with tuning tweaks.
"RAN Returns the next number from a sequence of pseudo random numbers
of uniform distribution over the range 0 to 1."
which reminds me that the gcc compiler also mentions uniform distribution.
^^^
gfortran
Post by Paul Sture
<https://gcc.gnu.org/onlinedocs/gfortran/RAND.html>
"RAND(FLAG) returns a pseudo-random number from a uniform distribution
between 0 and 1. If FLAG is 0, the next number in the current sequence
is returned; if FLAG is 1, the generator is restarted by CALL SRAND(0);
if FLAG has any other value, it is used as a new seed with SRAND."
Do the hardware random number generators also provide a uniform
distribution?
--
A supercomputer is a device for turning compute-bound problems into
I/O-bound problems. ---Ken Batcher
Bob Gezelter
2017-04-01 10:34:26 UTC
Reply
Permalink
Raw Message
Post by Paul Sture
Post by David Wade
Some existing code may rely on the fact its a bad generator.
Back in the day I used the Fortran random number routine to generate a
repeatable set of numbers. This was useful for re-running the same
tests on different hardware configurations or with tuning tweaks.
"RAN Returns the next number from a sequence of pseudo random numbers
of uniform distribution over the range 0 to 1."
which reminds me that the gcc compiler also mentions uniform distribution.
<https://gcc.gnu.org/onlinedocs/gfortran/RAND.html>
"RAND(FLAG) returns a pseudo-random number from a uniform distribution
between 0 and 1. If FLAG is 0, the next number in the current sequence
is returned; if FLAG is 1, the generator is restarted by CALL SRAND(0);
if FLAG has any other value, it is used as a new seed with SRAND."
Do the hardware random number generators also provide a uniform
distribution?
--
A supercomputer is a device for turning compute-bound problems into
I/O-bound problems. ---Ken Batcher
Paul,

Any of the random (or for that matter, pseudo-random) generators produces a uniform distribution.

While it is a bit dated (I used it as an undergraduate, decades ago), Knuth Volume 2 has a fairly good description of the issues surrounding "random" number generators. Hint: The original IBM SSP RNG, used throughout the world for a long, long time, was spectacularly poor when examined from a mathematical perspective.

There have been substantial work since Knuth, but it is a good starting point.

- Bob Gezelter, http://www.rlgsc.com
abrsvc
2017-04-01 12:54:35 UTC
Reply
Permalink
Raw Message
Post by Bob Gezelter
Any of the random (or for that matter, pseudo-random) generators produces a uniform distribution.
- Bob Gezelter, http://www.rlgsc.com
Bob,

I disagree. There are random number generators that will produce different distributions. Uniform means that over the range, there is an equal chance for any value to appear at any time. There are generators that produce random numbers that follow a bell curve for example. These different types of generators are also described in Knuth's book and are used for different purposes. I would not use the statement you listed without qualifications.

Dan
Arne Vajhøj
2017-04-01 20:26:42 UTC
Reply
Permalink
Raw Message
Post by abrsvc
Post by Bob Gezelter
Any of the random (or for that matter, pseudo-random) generators
produces a uniform distribution.
I disagree. There are random number generators that will produce
different distributions. Uniform means that over the range, there is
an equal chance for any value to appear at any time. There are
generators that produce random numbers that follow a bell curve for
example. These different types of generators are also described in
Knuth's book and are used for different purposes. I would not use
the statement you listed without qualifications.
Often/usually the random numbers with different distribution
is derived from one or more random numbers with uniform
distribution.

Arne
Paul Sture
2017-04-01 14:47:35 UTC
Reply
Permalink
Raw Message
Post by Bob Gezelter
Paul,
Any of the random (or for that matter, pseudo-random) generators
produces a uniform distribution.
While it is a bit dated (I used it as an undergraduate, decades ago),
Knuth Volume 2 has a fairly good description of the issues surrounding
"random" number generators. Hint: The original IBM SSP RNG, used
throughout the world for a long, long time, was spectacularly poor
when examined from a mathematical perspective.
There have been substantial work since Knuth, but it is a good starting point.
Thanks for the pointer, Bob.
--
The First of April: The only day of the year that people critically
evaluate news stories before believing them.
Scott Dorsey
2017-04-01 15:35:04 UTC
Reply
Permalink
Raw Message
Post by Paul Sture
Post by Bob Gezelter
Paul,
Any of the random (or for that matter, pseudo-random) generators
produces a uniform distribution.
While it is a bit dated (I used it as an undergraduate, decades ago),
Knuth Volume 2 has a fairly good description of the issues surrounding
"random" number generators. Hint: The original IBM SSP RNG, used
throughout the world for a long, long time, was spectacularly poor
when examined from a mathematical perspective.
There have been substantial work since Knuth, but it is a good starting point.
Thanks for the pointer, Bob.
Well, the truth is that sometimes you don't _want_ a uniform distribution, and
you need to take the random number source and redistribute it so it falls the
way you want. And Knuth doesn't talk about that, but Numerical Methods does.

So, that's kind of incorrect. Most random number generators produce a uniform
distribution, and there's no reason for them not to produce a uniform
distribution unless they are either broken or because you want a non-uniform
distribution. But not all of them do, because sometimes you want an
exponential distribution for something.
--scott
--
"C'est un Nagra. C'est suisse, et tres, tres precis."
John E. Malmberg
2017-04-01 18:38:11 UTC
Reply
Permalink
Raw Message
https://xkcd.com/221/
Stephen Hoffman
2017-04-01 19:41:54 UTC
Reply
Permalink
Raw Message
Do the hardware random number generators also provide a uniform distribution?
Only if they're working right.
--
Pure Personal Opinion | HoffmanLabs LLC
j***@yahoo.co.uk
2017-03-31 17:37:07 UTC
Reply
Permalink
Raw Message
Post by John Reagan
It is Friday and I'm in a good mood. I figured I'll toss something out for a friendly discussion.
For Alpha and Itanium, the MATH$RANDOMxxx functions use exactly the same algorithm as VAX. Is is a rather simple piece of code that is pseudo-random. If you have them the same seed value, you'll get the same sequence over and over. Of course some of you compute your initial seed with some randomness (time of day, bits out of some system cell, a hash of the tweets by the President, etc.) that at least makes you get different sequences on different runs.
For x86-64, recent chips have real random number generators,
7.3.17.1 RDRAND
The RDRAND instruction returns a random number. All Intel processors that
support the RDRAND instruction indicate the availability of the RDRAND
instruction via reporting CPUID.01H:ECX.RDRAND[bit 30] = 1.
RDRAND returns random numbers that are supplied by a cryptographically secure,
deterministic random bit generator DRBG. The DRBG is designed to meet the NIST
SP 800-90A standard. The DRBG is re-seeded frequently from an on-chip
non-deterministic entropy source to guarantee data returned by RDRAND is statistically uniform, nonperiodic and non-deterministic.
7.3.17.2 RDSEED
The RDSEED instruction returns a random number. All Intel processors that
support the RDSEED instruction indicate the availability of the RDSEED
instruction via reporting CPUID.(EAX=07H, ECX=0H):EBX.RDSEED[bit 18] = 1.
RDSEED returns random numbers that are supplied by a cryptographically secure,
enhanced non-deterministic random bit generator (Enhanced NRBG). The NRBG is
designed to meet the NIST SP 800-90B and NIST SP800-90C
standards.
So should we switch MTH$RANDOM/MATH$RANDOM to use the on-chip, much-better, random number generator RDSEED? The downside is that even if start with the same seed value (we'd essentially ignore the seed value), you'll get different sequences every single time. We'll still have to extract bits to form the floating result of course.
29 **
30 ** This file is used to generate object modules for vms_rand.
31 ** The interface is: F_TYPE y = vms_rand(unsigned int *seed).
32 **
33 ** The routine updates seed with
34 ** seed = (69069 * seed + 1) mod (2 ^ 32)
35 ** and then uses the high order bits to form a floating
36 ** point value. The goal of this routine is compatibility
37 ** with the VAX/VMS mth$random routine.
Thanks for asking :)

1) What would Steve Lionel's outfit do (if it was Fortran)?

I believe Steve's no longer with his previous employer?

2) What would Steve Jobs outfit do (if it was Apple)?

I believe Steve's no longer with his previous employer?

3) What does the rest of the industry do (outside Intel)
with respect to the x86-specific RNGstuff? See e.g. (1) ?

Prime candidates for further reading might include AMD
(it's not immediately obvious that they have capability
directly matching the Intel x86 one) and (for entertainment)
ARM, both architectures and common implementations.

There seems to be a certain amount of existing discussion
on the subject in general (putting the VMS specifics
aside). This isn't as easy as it might at first appear,
either technically or otherwise.

I see Simon (Clubley) has already provided one starting
point for some further reading, some other further
reading is available on various stackexchange etc
discussions.

Are you sure you wanted to ask this :)
John Reagan
2017-03-31 17:42:05 UTC
Reply
Permalink
Raw Message
Post by j***@yahoo.co.uk
Post by John Reagan
It is Friday and I'm in a good mood. I figured I'll toss something out for a friendly discussion.
For Alpha and Itanium, the MATH$RANDOMxxx functions use exactly the same algorithm as VAX. Is is a rather simple piece of code that is pseudo-random. If you have them the same seed value, you'll get the same sequence over and over. Of course some of you compute your initial seed with some randomness (time of day, bits out of some system cell, a hash of the tweets by the President, etc.) that at least makes you get different sequences on different runs.
For x86-64, recent chips have real random number generators,
7.3.17.1 RDRAND
The RDRAND instruction returns a random number. All Intel processors that
support the RDRAND instruction indicate the availability of the RDRAND
instruction via reporting CPUID.01H:ECX.RDRAND[bit 30] = 1.
RDRAND returns random numbers that are supplied by a cryptographically secure,
deterministic random bit generator DRBG. The DRBG is designed to meet the NIST
SP 800-90A standard. The DRBG is re-seeded frequently from an on-chip
non-deterministic entropy source to guarantee data returned by RDRAND is statistically uniform, nonperiodic and non-deterministic.
7.3.17.2 RDSEED
The RDSEED instruction returns a random number. All Intel processors that
support the RDSEED instruction indicate the availability of the RDSEED
instruction via reporting CPUID.(EAX=07H, ECX=0H):EBX.RDSEED[bit 18] = 1.
RDSEED returns random numbers that are supplied by a cryptographically secure,
enhanced non-deterministic random bit generator (Enhanced NRBG). The NRBG is
designed to meet the NIST SP 800-90B and NIST SP800-90C
standards.
So should we switch MTH$RANDOM/MATH$RANDOM to use the on-chip, much-better, random number generator RDSEED? The downside is that even if start with the same seed value (we'd essentially ignore the seed value), you'll get different sequences every single time. We'll still have to extract bits to form the floating result of course.
29 **
30 ** This file is used to generate object modules for vms_rand.
31 ** The interface is: F_TYPE y = vms_rand(unsigned int *seed).
32 **
33 ** The routine updates seed with
34 ** seed = (69069 * seed + 1) mod (2 ^ 32)
35 ** and then uses the high order bits to form a floating
36 ** point value. The goal of this routine is compatibility
37 ** with the VAX/VMS mth$random routine.
Thanks for asking :)
1) What would Steve Lionel's outfit do (if it was Fortran)?
I believe Steve's no longer with his previous employer?
2) What would Steve Jobs outfit do (if it was Apple)?
I believe Steve's no longer with his previous employer?
3) What does the rest of the industry do (outside Intel)
with respect to the x86-specific RNGstuff? See e.g. (1) ?
Prime candidates for further reading might include AMD
(it's not immediately obvious that they have capability
directly matching the Intel x86 one) and (for entertainment)
ARM, both architectures and common implementations.
There seems to be a certain amount of existing discussion
on the subject in general (putting the VMS specifics
aside). This isn't as easy as it might at first appear,
either technically or otherwise.
I see Simon (Clubley) has already provided one starting
point for some further reading, some other further
reading is available on various stackexchange etc
discussions.
Are you sure you wanted to ask this :)
Aboslutely! I knew many of you know more about this area than I do and wanted feedback.
IanD
2017-04-01 03:50:11 UTC
Reply
Permalink
Raw Message
Post by John Reagan
Post by j***@yahoo.co.uk
Post by John Reagan
It is Friday and I'm in a good mood. I figured I'll toss something out for a friendly discussion.
For Alpha and Itanium, the MATH$RANDOMxxx functions use exactly the same algorithm as VAX. Is is a rather simple piece of code that is pseudo-random. If you have them the same seed value, you'll get the same sequence over and over. Of course some of you compute your initial seed with some randomness (time of day, bits out of some system cell, a hash of the tweets by the President, etc.) that at least makes you get different sequences on different runs.
For x86-64, recent chips have real random number generators,
7.3.17.1 RDRAND
The RDRAND instruction returns a random number. All Intel processors that
support the RDRAND instruction indicate the availability of the RDRAND
instruction via reporting CPUID.01H:ECX.RDRAND[bit 30] = 1.
RDRAND returns random numbers that are supplied by a cryptographically secure,
deterministic random bit generator DRBG. The DRBG is designed to meet the NIST
SP 800-90A standard. The DRBG is re-seeded frequently from an on-chip
non-deterministic entropy source to guarantee data returned by RDRAND is statistically uniform, nonperiodic and non-deterministic.
7.3.17.2 RDSEED
The RDSEED instruction returns a random number. All Intel processors that
support the RDSEED instruction indicate the availability of the RDSEED
instruction via reporting CPUID.(EAX=07H, ECX=0H):EBX.RDSEED[bit 18] = 1.
RDSEED returns random numbers that are supplied by a cryptographically secure,
enhanced non-deterministic random bit generator (Enhanced NRBG). The NRBG is
designed to meet the NIST SP 800-90B and NIST SP800-90C
standards.
So should we switch MTH$RANDOM/MATH$RANDOM to use the on-chip, much-better, random number generator RDSEED? The downside is that even if start with the same seed value (we'd essentially ignore the seed value), you'll get different sequences every single time. We'll still have to extract bits to form the floating result of course.
29 **
30 ** This file is used to generate object modules for vms_rand.
31 ** The interface is: F_TYPE y = vms_rand(unsigned int *seed).
32 **
33 ** The routine updates seed with
34 ** seed = (69069 * seed + 1) mod (2 ^ 32)
35 ** and then uses the high order bits to form a floating
36 ** point value. The goal of this routine is compatibility
37 ** with the VAX/VMS mth$random routine.
Thanks for asking :)
1) What would Steve Lionel's outfit do (if it was Fortran)?
I believe Steve's no longer with his previous employer?
2) What would Steve Jobs outfit do (if it was Apple)?
I believe Steve's no longer with his previous employer?
3) What does the rest of the industry do (outside Intel)
with respect to the x86-specific RNGstuff? See e.g. (1) ?
Prime candidates for further reading might include AMD
(it's not immediately obvious that they have capability
directly matching the Intel x86 one) and (for entertainment)
ARM, both architectures and common implementations.
There seems to be a certain amount of existing discussion
on the subject in general (putting the VMS specifics
aside). This isn't as easy as it might at first appear,
either technically or otherwise.
I see Simon (Clubley) has already provided one starting
point for some further reading, some other further
reading is available on various stackexchange etc
discussions.
Are you sure you wanted to ask this :)
Aboslutely! I knew many of you know more about this area than I do and wanted feedback.
^This

This is why a transformation of VMS could be successful

The custodians recognising an ivory tower is no place from which to project a future from

I'll use some Aussie lingo here

"I'm bloody impressed" :-)

I know sfa about random number generators other than a while ago I followed a youtube rabbit warren that eventually led me to a place on the internet that will generate random numbers based off quantum state particles

http://photonics.anu.edu.au/qoptics/Research/qrng.php

Random numbers are a big deal for security, like precision was years ago, which incidentally I think Vax used to great advantage to get an edge before VMS surrendered it's soul to IEEE being good enough

Maybe an extra VSI function, sys$vsi_near_true_random that calls VSI home that in turn verifies a call to a true random number source like a cosmic state capture or a call to one of these quantum sources? Pay for extended functionality if you need it

About 6 months ago I did do some reading about the battles that went on over math precision that happened way back in Digital and other industries - I actually found the reading invigorating and exciting (yeah, I know, I need to get out more!), but if VMS is going to hinge its comeback on things like security, then random number generation WILL factor into it, go for the best possible even if by today's standards it seems beyond what's needed
David Wade
2017-04-01 09:11:45 UTC
Reply
Permalink
Raw Message
Post by John Reagan
Post by j***@yahoo.co.uk
Post by John Reagan
It is Friday and I'm in a good mood. I figured I'll toss something out for a friendly discussion.
For Alpha and Itanium, the MATH$RANDOMxxx functions use exactly the same algorithm as VAX. Is is a rather simple piece of code that is pseudo-random. If you have them the same seed value, you'll get the same sequence over and over. Of course some of you compute your initial seed with some randomness (time of day, bits out of some system cell, a hash of the tweets by the President, etc.) that at least makes you get different sequences on different runs.
For x86-64, recent chips have real random number generators,
7.3.17.1 RDRAND
The RDRAND instruction returns a random number. All Intel processors that
support the RDRAND instruction indicate the availability of the RDRAND
instruction via reporting CPUID.01H:ECX.RDRAND[bit 30] = 1.
RDRAND returns random numbers that are supplied by a cryptographically secure,
deterministic random bit generator DRBG. The DRBG is designed to meet the NIST
SP 800-90A standard. The DRBG is re-seeded frequently from an on-chip
non-deterministic entropy source to guarantee data returned by RDRAND is statistically uniform, nonperiodic and non-deterministic.
7.3.17.2 RDSEED
The RDSEED instruction returns a random number. All Intel processors that
support the RDSEED instruction indicate the availability of the RDSEED
instruction via reporting CPUID.(EAX=07H, ECX=0H):EBX.RDSEED[bit 18] = 1.
RDSEED returns random numbers that are supplied by a cryptographically secure,
enhanced non-deterministic random bit generator (Enhanced NRBG). The NRBG is
designed to meet the NIST SP 800-90B and NIST SP800-90C
standards.
So should we switch MTH$RANDOM/MATH$RANDOM to use the on-chip, much-better, random number generator RDSEED? The downside is that even if start with the same seed value (we'd essentially ignore the seed value), you'll get different sequences every single time. We'll still have to extract bits to form the floating result of course.
29 **
30 ** This file is used to generate object modules for vms_rand.
31 ** The interface is: F_TYPE y = vms_rand(unsigned int *seed).
32 **
33 ** The routine updates seed with
34 ** seed = (69069 * seed + 1) mod (2 ^ 32)
35 ** and then uses the high order bits to form a floating
36 ** point value. The goal of this routine is compatibility
37 ** with the VAX/VMS mth$random routine.
Thanks for asking :)
1) What would Steve Lionel's outfit do (if it was Fortran)?
I believe Steve's no longer with his previous employer?
2) What would Steve Jobs outfit do (if it was Apple)?
I believe Steve's no longer with his previous employer?
3) What does the rest of the industry do (outside Intel)
with respect to the x86-specific RNGstuff? See e.g. (1) ?
Prime candidates for further reading might include AMD
(it's not immediately obvious that they have capability
directly matching the Intel x86 one) and (for entertainment)
ARM, both architectures and common implementations.
There seems to be a certain amount of existing discussion
on the subject in general (putting the VMS specifics
aside). This isn't as easy as it might at first appear,
either technically or otherwise.
I see Simon (Clubley) has already provided one starting
point for some further reading, some other further
reading is available on various stackexchange etc
discussions.
Are you sure you wanted to ask this :)
Aboslutely! I knew many of you know more about this area than I do and wanted feedback.
whilst it may be a good idea to make it available I would use a new
interface. MATH$RANDON is documented to return pseudorandom numbers, not
random numbers. These are different and pure random numbers may not be
better than random numbers...

Dave
Stephen Hoffman
2017-04-01 19:41:14 UTC
Reply
Permalink
Raw Message
Post by David Wade
whilst it may be a good idea to make it available I would use a new
interface. MATH$RANDON is documented to return pseudorandom numbers,
not random numbers. These are different and pure random numbers may not
be better than random numbers...
Absent external source(s) of randomness — a bank of lava lamps most
famously, or the Intel RDRAND instruction — software algorithms
implement a pseudo random number generator; a PRNG. Better generators
— which is what most folks should be using by default, absent specific
requirements — are CPRNGs; cryptographic...

Related: https://wiki.openssl.org/index.php/Random_Numbers
--
Pure Personal Opinion | HoffmanLabs LLC
Hans Vlems
2017-03-31 18:34:30 UTC
Reply
Permalink
Raw Message
In math functions with the same name are supposed to return the same result on identical input.
I'd expect the same with MTH$RANDOM, even if the current implementation doesn't really live up to expectations (it ain't random, right).
Make the the new hardware functionality available by another name: MTH$RANDOMNUMBER().
Hans
o***@gmail.com
2017-03-31 20:45:33 UTC
Reply
Permalink
Raw Message
I'd leave MTH$RANDOM as it is and provide a RANDOM: pseudo-device that applications needing a random seed can read (with appropriate function modifier to give it /dev/urandom behavior if that is sufficient).
Simon Clubley
2017-04-02 23:52:57 UTC
Reply
Permalink
Raw Message
Post by o***@gmail.com
I'd leave MTH$RANDOM as it is and provide a RANDOM: pseudo-device that
applications needing a random seed can read (with appropriate function
modifier to give it /dev/urandom behavior if that is sufficient).
That raises an interesting question: would some functionality, which
is intended for application level use, be better implemented as
a pseudo-device or is it better to stay with API calls ?

Simon.
--
Simon Clubley, ***@remove_me.eisner.decus.org-Earth.UFP
Microsoft: Bringing you 1980s technology to a 21st century world
Arne Vajhøj
2017-04-03 00:03:13 UTC
Reply
Permalink
Raw Message
Post by Simon Clubley
Post by o***@gmail.com
I'd leave MTH$RANDOM as it is and provide a RANDOM: pseudo-device that
applications needing a random seed can read (with appropriate function
modifier to give it /dev/urandom behavior if that is sufficient).
That raises an interesting question: would some functionality, which
is intended for application level use, be better implemented as
a pseudo-device or is it better to stay with API calls ?
The VMS way is API. Pseudo devices is the *nix way.

I would suggest staying with API and *maybe* have an optional
*nix compatibility package that exposes some pseudo devices.

Arne
John Reagan
2017-04-03 13:00:18 UTC
Reply
Permalink
Raw Message
Post by Arne Vajhøj
Post by Simon Clubley
Post by o***@gmail.com
I'd leave MTH$RANDOM as it is and provide a RANDOM: pseudo-device that
applications needing a random seed can read (with appropriate function
modifier to give it /dev/urandom behavior if that is sufficient).
That raises an interesting question: would some functionality, which
is intended for application level use, be better implemented as
a pseudo-device or is it better to stay with API calls ?
The VMS way is API. Pseudo devices is the *nix way.
I would suggest staying with API and *maybe* have an optional
*nix compatibility package that exposes some pseudo devices.
Arne
I wasn't suggesting that OpenVMS add pseudo-devices (that's a different thread, different rathole). I was observing that the hardware generator is at least indirectly available on Linux systems.
Stephen Hoffman
2017-04-03 15:39:55 UTC
Reply
Permalink
Raw Message
Post by o***@gmail.com
I'd leave MTH$RANDOM as it is and provide a RANDOM: pseudo-device that
applications needing a random seed can read (with appropriate function
modifier to give it /dev/urandom behavior if that is sufficient).
That raises an interesting question: would some functionality, which is
intended for application level use, be better implemented as a
pseudo-device or is it better to stay with API calls ?
OpenVMS traditionally doesn't use that approach, though it's certainly
possible, and it'll work. OPEN, READ/ERROR=err/END=eof, CLOSE would
work here, in place of the missing f$qiow or f$io_performw et al.
Opening up and closing the I/O channel does make for more glue code.
What to do in the undoubtedly rare case of errors from the generator?
What apps might then want or need to do with larger and inherently
ASCII/MCS string values in the 32-bit signed-integer (with
indirection-via-f$fao) environment of DCL, too?

There are some completely different alternatives to the
everything-is-a-file approach, too: https://pythonhosted.org/pyobjc/
This bridge allows Python to access the objects and interfaces
presented by the macOS Cocoa OO environment. This centrally if
alternatives to the typical programming interfaces presented by OpenVMS
are under discussion. Also see the Microsoft .NET integration, too.

Catching up to the everything-is-a-file approach does seem... not
really even catching up.
--
Pure Personal Opinion | HoffmanLabs LLC
Bob Koehler
2017-04-03 12:57:03 UTC
Reply
Permalink
Raw Message
Post by o***@gmail.com
I'd leave MTH$RANDOM as it is and provide a RANDOM: pseudo-device that applications needing a random seed can read (with appropriate function modifier to give it /dev/urandom behavior if that is sufficient).
RNA0:, URA0: ?
Stephen Hoffman
2017-03-31 22:08:12 UTC
Reply
Permalink
Raw Message
Post by John Reagan
It is Friday and I'm in a good mood. I figured I'll toss something out
for a friendly discussion.
For Alpha and Itanium, the MATH$RANDOMxxx functions use exactly the
same algorithm as VAX. Is is a rather simple piece of code that is
pseudo-random.
What VAX, Alpha and Itanium are using for random-number generation is
on par with the Purdy Polynomial. None of that would be used on any
modern system.

As for the Intel RDRAND implementation, various platforms use that —
when available — as part of the seed for the generator — as fodder for
the entropy pool — and not as the generator.

ChaCha20 is a recent and preferred choice, though some folks are still
using ARC4..

As a somewhat larger project, hauling over the OpenBSD crypto framework
would address this and other limitations.

https://www.schneier.com/blog/archives/2013/09/surreptitiously.html
http://openbsd-archive.7691.n7.nabble.com/RdRand-instruction-is-it-used-by-default-td234317.html

http://www.ieee-security.org/TC/SP2014/papers/Not-So-RandomNumbersinVirtualizedLinuxandtheWhirlwindRNG.pdf

https://www.openbsd.org/crypto.html
https://datatracker.ietf.org/doc/rfc7539/
http://blog.cr.yp.to/20140205-entropy.html
https://cve.circl.lu/cve/CVE-2008-0166

Now if that security marketing y'all have been spouting is just
cluelessness or feel-good-ism and the VSI technical goal is historic
preservation for a declining installed base, then leave it alone.

If the goal is to design ever-worse and more inscrutable
implementations, certainly add some arcane knobs to control what
happens. Logical name knobs make for some of the most non-modular and
ill-maintainable code and there are certainly enough implementation
choices for the names or the translations available to make those
implementations even more arcane (q.v. the C RTL dumpster fire), but
there are other good options for writing ever-more-obfuscated APIs.

TL;DR: There is little or no reason to use any implementation other
than a cryptographically secure random number generator; CPRNG. Even
for "regular" random numbers. Old arguments around the CPRNG
performance are moot in recent years, and attacks against lesser RNGs
are known. If some developer really needs a lesser RNG, one that's
more performance-sensitive than modern networking code, or an RNG that
can replay values — that for use in testing or in implementing latent
vulnerabilities, for instance — those are available from various
sources.
--
Pure Personal Opinion | HoffmanLabs LLC
t***@glaver.org
2017-04-01 02:48:40 UTC
Reply
Permalink
Raw Message
Post by Stephen Hoffman
What VAX, Alpha and Itanium are using for random-number generation is
on par with the Purdy Polynomial. None of that would be used on any
modern system.
As I asked when the X86-64 port was first mentioned, it is important to determine what the audience is - if the goal is to attract customers who are moving existing applications from older VMS architectures (and who may not have source code), then the fewer user-visible changes the better. If the goal is to attract new customers to VMS, then providing a modern environment with all the expected creature comforts is important. Sometimes those goals conflict with each other (like decisions about the filesystem and RMS) to varying extents, sometimes both goals can be accommodated easily.

I agree that yet more knobs and alternate code paths are unpleasant, but it should be possible to create functions with new names where needed and move the old ones into the deprecated features manual(s).

If the language compilers will support both older syntax (for moving older code over - and some of that code may well be in VMS itself) and modern syntax, it probably makes sense to do issue a COMPILER-W-DOYOUREALLYWANTTOUSETHISFUNCTION warning when in "modern" mode and to accept it without warning in "classic" mode. Extra points for a compiler switch to toggle the warning behavior.
Stephen Hoffman
2017-04-01 19:26:17 UTC
Reply
Permalink
Raw Message
Post by t***@glaver.org
Post by Stephen Hoffman
What VAX, Alpha and Itanium are using for random-number generation is
on par with the Purdy Polynomial. None of that would be used on any
modern system.
As I asked when the X86-64 port was first mentioned, it is important to
determine what the audience is - if the goal is to attract customers
who are moving existing applications from older VMS architectures (and
who may not have source code), then the fewer user-visible changes the
better. If the goal is to attract new customers to VMS, then providing
a modern environment with all the expected creature comforts is
important. Sometimes those goals conflict with each other (like
decisions about the filesystem and RMS) to varying extents, sometimes
both goals can be accommodated easily.
Following through on that question, the next one is whether that market
might grow, or whether it'll shrink. Some applications invariably
move off a platform. Without the arrival of sufficient new
applications to offset that, there's only one outcome: we're porting.
Post by t***@glaver.org
I agree that yet more knobs and alternate code paths are unpleasant,
but it should be possible to create functions with new names where
needed and move the old ones into the deprecated features manual(s).
For some cases it is. For others, it isn't. Following through on
that... Compatibility a wonderful idea, but it increases the costs,
and it inevitably and eventually ends up creating system code that's a
code sprawl — and even with the best coding practices — that becomes
unmanageable and unmaintainable, with indecipherable documentation, and
increasingly insecure or unstable. Compatibility incurs debt; future
costs both for the vendor, and for the end-user, and especially for new
adopters and new developers and new applications.

Yeah, I know, apparently enterprise product documentation and APIs are
apparently supposed to be awful.
Post by t***@glaver.org
If the language compilers will support both older syntax (for moving
older code over - and some of that code may well be in VMS itself) and
modern syntax, it probably makes sense to do issue a
COMPILER-W-DOYOUREALLYWANTTOUSETHISFUNCTION warning when in "modern"
mode and to accept it without warning in "classic" mode. Extra points
for a compiler switch to toggle the warning behavior.
Support for flagging deprecated and unsafe features has been another
long-requested platform feature. (The tools on some platforms offer
to refactor or migrate problematic code, but I digress.) Following
through on APU and feature deprecation, when does that "classic" stuff
ever go away? Never? If the results of API or feature deprecation
never go away, then the results are always going to be insecure if
there are insecure calls around, and the costs of supporting and
testing coverage and documentation and the rest — avoiding
instabilities, breaches, trying to describe or use 64-bit programming
on OpenVMS — all continue to accrue for the vendor and for the
end-users. TANSTAAFL.

Moving forward slowly enough for the oldest customer applications and
quickly enough for the actively-maintained applications and for
potential new applications and wholly new deployments is not an easy
balance for any vendor to maintain.
--
Pure Personal Opinion | HoffmanLabs LLC
Arne Vajhøj
2017-04-01 20:33:38 UTC
Reply
Permalink
Raw Message
Post by Stephen Hoffman
Post by t***@glaver.org
I agree that yet more knobs and alternate code paths are unpleasant,
but it should be possible to create functions with new names where
needed and move the old ones into the deprecated features manual(s).
For some cases it is. For others, it isn't. Following through on
that... Compatibility a wonderful idea, but it increases the costs,
and it inevitably and eventually ends up creating system code that's a
code sprawl — and even with the best coding practices — that becomes
unmanageable and unmaintainable, with indecipherable documentation, and
increasingly insecure or unstable. Compatibility incurs debt; future
costs both for the vendor, and for the end-user, and especially for new
adopters and new developers and new applications.
Sure there is overhead of keeping old stuff around.

But let us keep proportions.

A general API today will have 100000-250000 methods (functions) and
thousands of them being obsolete. Without becoming unmaintainable.

Keeping a few hundred obsolete VMS functions around should not
be a big problem.

Arne
Stephen Hoffman
2017-04-02 22:01:07 UTC
Reply
Permalink
Raw Message
Keeping a few hundred obsolete VMS functions around should not be a big
problem.
For many of the cases and when viewed individually, sure. Viewed over
time, the debt and the complexity and the testing effort only accrues.

So do the risks of the old code.

For example... Use a system with a flat address space, then go use
64-bit addressing on OpenVMS, and let me know what you think. Or go
look at what's involved with a secure cryptographic password hash, and
then look at what OpenVMS uses. The list continues.

While the OpenVMS 64-bit design is a fine and clever implementation, it
really reminds me of the era of dealing with TKB when I'm starting out
to develop a new 64-bit app. Most of us just don't do that, and we
all use 32-bit, which means we either ignore 64-bit APIs in our code
and our implementations, or we end up with more complexity to deal with
both 32- and 64-bit. All for compatibility. Going 64-bit native was
and is and will be disruptive, but that would be a future-focused
decision and not a backward-focused one.

Maintaining upward-compatibility completely — completely, utterly,
totally, entirely — blocks certain improvements, too. This whether
efforts to upgrade system security or otherwise.

The aggregate cost here is much higher — and much more subtle, and more
endemic — than many folks might realize.

To be clear, I'm not suggesting blindly breaking compatibility. I'm
suggesting breaking compatibility where there are direct and beneficial
improvements for users, and plans for and a path forward for new
development and new work, and with a migration path and deprecation
schedule for older applications. Because if you can't ever get rid of
the old and problematic APIs or tools or implementations, there's
rather less reason to even fix it. And rather less of a future for
wholly new applications and wholly new deployments.
--
Pure Personal Opinion | HoffmanLabs LLC
Arne Vajhøj
2017-04-03 00:15:51 UTC
Reply
Permalink
Raw Message
Post by Stephen Hoffman
Keeping a few hundred obsolete VMS functions around should not be a
big problem.
For many of the cases and when viewed individually, sure. Viewed over
time, the debt and the complexity and the testing effort only accrues.
I think real world experiences show otherwise.

Java, .NET and PHP seems to be thriving even though they have
accumulated obsolete API's over a couple of decades.
Post by Stephen Hoffman
For example... Use a system with a flat address space, then go use
64-bit addressing on OpenVMS, and let me know what you think. Or go
look at what's involved with a secure cryptographic password hash, and
then look at what OpenVMS uses. The list continues.
While the OpenVMS 64-bit design is a fine and clever implementation, it
really reminds me of the era of dealing with TKB when I'm starting out
to develop a new 64-bit app. Most of us just don't do that, and we all
use 32-bit, which means we either ignore 64-bit APIs in our code and our
implementations, or we end up with more complexity to deal with both 32-
and 64-bit. All for compatibility. Going 64-bit native was and is
and will be disruptive, but that would be a future-focused decision and
not a backward-focused one.
The drawback from that decision is much bigger than from keeping some
old functions around.

So this is a real discussion in my opinion.

But how difficult would it be to:
- have totally new API's that was only 64 bit
- have new images mapped into bottom of P2 and use top of P2 for stack
- have no compilers generate 64 bit code for that (unless /32bit used)
- have legacy applications use P0+P1 and current API's
?

Arne
t***@glaver.org
2017-04-02 08:39:30 UTC
Reply
Permalink
Raw Message
Post by Stephen Hoffman
For some cases it is. For others, it isn't. Following through on
that... Compatibility a wonderful idea, but it increases the costs,
and it inevitably and eventually ends up creating system code that's a
code sprawl — and even with the best coding practices — that becomes
unmanageable and unmaintainable, with indecipherable documentation, and
increasingly insecure or unstable. Compatibility incurs debt; future
costs both for the vendor, and for the end-user, and especially for new
adopters and new developers and new applications.
If it isn't in the code path of the "operating system guts" part of VMS (which this isn't, as it is in a library), there shouldn't be any execution overhead (you'd have a hard time convincing me that LIB$FIND_IMAGE_SYMBOL overhead on x86 is meaningful).

As far as the code maintenance overhead, that's up to VSI to decide on the trade-offs. If VMS on x86 includes something like the *BSD libmap.conf facility, this stuff could even be moved off into some separate library source file and image, never to be looked at again. Or VSI could decide that these routines are only available to translated images (which probably also moves them into the never looked at again area). Of course, doing that hinders migration for folks who want to migrate by translation and eventually recompilation. I'm sure VSI has done a lot of studies to determine where their market is and where it is likely to be going.
Post by Stephen Hoffman
[...] Following
through on APU and feature deprecation, when does that "classic" stuff
ever go away? Never? If the results of API or feature deprecation
never go away, then the results are always going to be insecure if
there are insecure calls around, and the costs of supporting and
testing coverage and documentation and the rest — avoiding
instabilities, breaches, trying to describe or use 64-bit programming
on OpenVMS — all continue to accrue for the vendor and for the
end-users. TANSTAAFL.
Ability to compile (and possibly link) stuff? "A long time"

Ability to execute stuff? "Never"

To expand on that, if (hypothetically) VMS on x86 offers DECC compatibility (minus /STANDARD=VAXC) and not VAX C compatibility, then there's no point in maintaining header files, etc. for things that were in VAX C and not DECC. But the executables should still work, assuming there's support in some form for translated images. If there's no translated image support and no compiler support, then all of those routines can be thrown overboard. Along with some percentage of the potential market. As a historical example, look at PDP-11 compatibility mode - it was standard in early VMS (because VMS itself partially ran in compatibility mode), then became optional (with free licenses to existing customers) and then "went away". I don't think it even made it as far as AXP, right?

As an example, consider C-Kermit. It is a relatively large application with a somewhat deep understanding of VMS stuff (it talks to RMS directly for file I/O, avoiding the CRTL routines because they simply don't work for a lot of file organizations). It comes with full source code and the compilation requirements are relatively small - pretty much the only hard requirement is a C compiler - a makefile system is not needed because you can use the provided DCL scripts and you only need to link against a TCP/IP library if you actually have TCP/IP on the system. Yet, even in the days of VAX, it was hard to get people to send binaries back to the project. You can still see this today - the C-Kermit download site lists 115 different binaries with combinations of architecture / network / VMS version.

There are software vendors that didn't port from VAX to AXP, and vendors who didn't port from AXP to Itanium. I'm sure there will be some vendors who don't port to x86, if for no other reason than they're no longer in business. So customers who want to migrate either need to obtain equivalent products or translate existing images.

One solution might be to offer compatibility as options (to use classic DEC-speak, either as SIPs or as layered products). I don't know if that is feasible with VMS. It is a solution that (for example) FreeBSD has used essentially forever - there are ports that provide compatibility for each version all the way back to FreeBSD 4.x (the current release version is 11.x). They're not perfect, but they support executing most of the older binaries out there.

As far as security goes, anybody who thinks that moving [potentially] insecure code (either by translation or recompilation) to a new platform will fix their security problems is pretty naive. Even if the compiler warns about "function x() is known to be insecure, consider alternatives" some people will ignore that. Only someone knowledgeable with the specific environment can say if something is "secure enough" for them (obligatory "the only secure computer is one that is turned off" comment here). Where security can be improved without impacting users, no question - do it if practical. If it impacts users, warn where possible, but do your best to avoid breaking things. The customer may not always be right, but they're the one with the checkbook.
Stephen Hoffman
2017-04-02 22:33:33 UTC
Reply
Permalink
Raw Message
Post by t***@glaver.org
Post by Stephen Hoffman
For some cases it is. For others, it isn't. Following through on
that... Compatibility a wonderful idea, but it increases the costs,
and it inevitably and eventually ends up creating system code that's a
code sprawl — and even with the best coding practices — that becomes
unmanageable and unmaintainable, with indecipherable documentation, and
increasingly insecure or unstable. Compatibility incurs debt; future
costs both for the vendor, and for the end-user, and especially for new
adopters and new developers and new applications.
If it isn't in the code path of the "operating system guts" part of VMS
(which this isn't, as it is in a library), there shouldn't be any
execution overhead (you'd have a hard time convincing me that
LIB$FIND_IMAGE_SYMBOL overhead on x86 is meaningful).
As far as the code maintenance overhead, that's up to VSI to decide on
the trade-offs. If VMS on x86 includes something like the *BSD
libmap.conf facility, this stuff could even be moved off into some
separate library source file and image, never to be looked at again. Or
VSI could decide that these routines are only available to translated
images (which probably also moves them into the never looked at again
area). Of course, doing that hinders migration for folks who want to
migrate by translation and eventually recompilation. I'm sure VSI has
done a lot of studies to determine where their market is and where it
is likely to be going.
For this math$random case, sure.

The fun is what happens as the cruft accumulates. Which it inevitably does.
Post by t***@glaver.org
Post by Stephen Hoffman
[...] Following through on APU and feature deprecation, when does that
"classic" stuff ever go away? Never? If the results of API or
feature deprecation never go away, then the results are always going to
be insecure if there are insecure calls around, and the costs of
supporting and testing coverage and documentation and the rest —
avoiding instabilities, breaches, trying to describe or use 64-bit
programming on OpenVMS — all continue to accrue for the vendor and for
the end-users. TANSTAAFL.
You're ignoring the difficulties that are almost inherent in the system
designs that choose compatibility. 64-bit is one. Compared with a
flat 64-bit address space, OpenVMS isn't nearly as easy as it should
be, which means that most applications are still 32-bit, and various
folks are dealing with maintaining both 32- and 64-bit APIs and
interfaces, testing and the rest.
Post by t***@glaver.org
Ability to compile (and possibly link) stuff? "A long time"
Ability to execute stuff? "Never"
To expand on that, if (hypothetically) VMS on x86 offers DECC
compatibility (minus /STANDARD=VAXC) and not VAX C compatibility, then
there's no point in maintaining header files, etc. for things that were
in VAX C and not DECC. But the executables should still work, assuming
there's support in some form for translated images. If there's no
translated image support and no compiler support, then all of those
routines can be thrown overboard. Along with some percentage of the
potential market. As a historical example, look at PDP-11 compatibility
mode - it was standard in early VMS (because VMS itself partially ran
in compatibility mode), then became optional (with free licenses to
existing customers) and then "went away". I don't think it even made it
as far as AXP, right?
As an example, consider C-Kermit. It is a relatively large application
with a somewhat deep understanding of VMS stuff (it talks to RMS
directly for file I/O, avoiding the CRTL routines because they simply
don't work for a lot of file organizations). It comes with full source
code and the compilation requirements are relatively small - pretty
much the only hard requirement is a C compiler - a makefile system is
not needed because you can use the provided DCL scripts and you only
need to link against a TCP/IP library if you actually have TCP/IP on
the system. Yet, even in the days of VAX, it was hard to get people to
send binaries back to the project. You can still see this today - the
C-Kermit download site lists 115 different binaries with combinations
of architecture / network / VMS version.
Nice example of the costs of compatibility, that.
Post by t***@glaver.org
There are software vendors that didn't port from VAX to AXP, and
vendors who didn't port from AXP to Itanium. I'm sure there will be
some vendors who don't port to x86, if for no other reason than they're
no longer in business. So customers who want to migrate either need to
obtain equivalent products or translate existing images.
Ayup. The other key question here is whether enough new folks are
willing to tangle with all this to offset the folks that don't port
forward, or that do port to other platforms.
Post by t***@glaver.org
One solution might be to offer compatibility as options (to use classic
DEC-speak, either as SIPs or as layered products). I don't know if that
is feasible with VMS. It is a solution that (for example) FreeBSD has
used essentially forever - there are ports that provide compatibility
for each version all the way back to FreeBSD 4.x (the current release
version is 11.x). They're not perfect, but they support executing most
of the older binaries out there.
Keeping old code alive directly offsets the efforts that can go into
new code and improvements.

Upward compatibility and stability is very important, but — when it
makes good sense to do so — sometimes breaking compatibility is more
important for the future of the platform.

I don't know how many developers are working on OpenBSD, but I'd wager
it's more than are working for VSI and HPE on OpenVMS. The equivalent
approach used by some other vendors is the so-called long-term support
release. Releases with five or maybe seven years of support and API
stability barring some really nasty bug, then move forward.
Post by t***@glaver.org
As far as security goes, anybody who thinks that moving [potentially]
insecure code (either by translation or recompilation) to a new
platform will fix their security problems is pretty naive. Even if the
compiler warns about "function x() is known to be insecure, consider
alternatives" some people will ignore that. Only someone knowledgeable
with the specific environment can say if something is "secure enough"
for them (obligatory "the only secure computer is one that is turned
off" comment here). Where security can be improved without impacting
users, no question - do it if practical. If it impacts users, warn
where possible, but do your best to avoid breaking things. The customer
may not always be right, but they're the one with the checkbook.
Sure. Up until they decide the existing and potential new customers
decide there are better alternatives, because changes weren't or can't
be made.

I really want to see VSI and OpenVMS succeed. The state of
math$random and such on OpenVMS is one of many smaller issues.
Individually, many of these can be addressed. Many can be ignored.
The larger issues with the state of OpenVMS — what compatibility often
leads to — are more difficult for wholly new deployments and wholly new
roll-outs to ignore. For the foreseeable future, VSI survives on the
installed base. But then the path that Mentec followed with the
PDP-11 installed base is not one that I want to see VSI and OpenVMS
follow. That's because newer users and newer deployments are needed
to offset those users and existing applications that don't port, or
that port off of OpenVMS. Again, there are no easy choices here.
But any sort of "always compatible" position — the inability to
judiciously and incrementally replace and remove technical debt, to use
a buzz-phrase — inevitably ends badly.
--
Pure Personal Opinion | HoffmanLabs LLC
Arne Vajhøj
2017-04-01 00:04:16 UTC
Reply
Permalink
Raw Message
Post by John Reagan
It is Friday and I'm in a good mood. I figured I'll toss something
out for a friendly discussion.
For Alpha and Itanium, the MATH$RANDOMxxx functions use exactly the
same algorithm as VAX. Is is a rather simple piece of code that is
pseudo-random. If you have them the same seed value, you'll get the
same sequence over and over. Of course some of you compute your
initial seed with some randomness (time of day, bits out of some
system cell, a hash of the tweets by the President, etc.) that at
least makes you get different sequences on different runs.
For x86-64, recent chips have real random number generators,
So should we switch MTH$RANDOM/MATH$RANDOM to use the on-chip,
much-better, random number generator RDSEED? The downside is that
even if start with the same seed value (we'd essentially ignore the
seed value), you'll get different sequences every single time. We'll
still have to extract bits to form the floating result of course.
The algorithm for MTH$RANDOM is documented.

Documented behavior should not be changed unless absolutely
necessary.

This change is not necessary.

I am not sure that MATH$RANDOM_x is documented using same
algorithm but changing behavior is still intrinsically bad.

So: a new function.

MATH$RNG([alg],[seed])

where alg is one of MATH$_RNGHW (CPU, default),
MATH$_RNGOLD (compatible), MATH$_RNGLECYUER (L'Ecuyer
MRG), MATH$RNG_MT (Mersenne Twister) and maybe more.

and seed is ignored, expect single longword, expect
multiple longwords etc. depending on alg.

Arne
Scott Dorsey
2017-04-01 10:14:11 UTC
Reply
Permalink
Raw Message
Post by Arne Vajhøj
The algorithm for MTH$RANDOM is documented.
Documented behavior should not be changed unless absolutely
necessary.
This change is not necessary.
Agreed. You're going to have folks who check the port of their monte carlo
simulation with each upgrade by making sure they get exactly the same results
with the same seed. Those people will get upset.

Adding a random device isn't a bad idea, but even if you do that I strongly
suggest a MTH$MORERANDOM function that is truly random, so that changing code
to use the more random function will only be a quick search and replace.
--scott
--
"C'est un Nagra. C'est suisse, et tres, tres precis."
John Reagan
2017-04-03 13:16:20 UTC
Reply
Permalink
Raw Message
Wow, I did get lots of feedback and learned quite a bit. Good thing I asked, eh?

So I'm convinced to leave MTH$RANDOM alone with its current algorithm.

Having access to the hardware generator would be a nice addition if people or other packages want to use it for additional entropy.

As for some other questions I think I remember...

Yes, you should be able to access the instructions via asm() through clang. However, our DEC C frontend (and others) does not have asm() and is like the Intel compiler in the sense that you access platform instructions via compiler intrinsics.

As for adding those "per-instruction" builtins to all frontends: We haven't done that in past. Mostly just C, BLISS, and Macro for our internal needs or explicit requests. People bringing open source code to OpenVMS are doing it in C and C++ (and other languages which we don't currently provide). Nobody has open source code written in COBOL, BASIC, Fortran, or Pascal that expects a mechanism to access the hardware generator. If somebody wanted to extend their current code base, a LIB$xxx or MATH$xxx routine can be added (I'll put that on the list). My reference to adding it to Macro was more a poke at Brian than anything else.
Richard Maher
2017-04-03 17:54:32 UTC
Reply
Permalink
Raw Message
Post by John Reagan
Wow, I did get lots of feedback and learned quite a bit. Good thing I asked, eh?
So I'm convinced to leave MTH$RANDOM alone with its current
algorithm.
Having access to the hardware generator would be a nice addition if
people or other packages want to use it for additional entropy.
Ok I give up :-( **WHY** won't you take the lead of standard/historical
VMS engineer solutions to this very same problem and add an optional
[flags] MATH$K_HARDWARE argument? You recalcitrant, bubble living, knob.
Loading...