Discussion:
RMS intro
(too old to reply)
Arne Vajhøj
2023-12-31 00:28:29 UTC
Permalink
So I took:
* what everybody knows about RMS
* relative file and direct access fixed length file examples
I recently posted
* the index-sequential file examples I have used numerous times before
* some file read and file creation test code I did a few years ago
and baked it into:

https://www.vajhoej.dk/arne/articles/rms.html

Arne
John Dallman
2023-12-31 07:43:00 UTC
Permalink
Post by Arne Vajhøj
https://www.vajhoej.dk/arne/articles/rms.html
Good work. You might want to explain why StreamCR is provided: I doubt it
is because of traditional Mac format, because VMS predates Mac by several
years. Was it used by older DEC OSes?

John
Neil Rieck
2023-12-31 10:19:50 UTC
Permalink
Post by John Dallman
Post by Arne Vajhøj
https://www.vajhoej.dk/arne/articles/rms.html
Good work. You might want to explain why StreamCR is provided: I doubt it
is because of traditional Mac format, because VMS predates Mac by several
years. Was it used by older DEC OSes?
John
IIRC, all Stream formats were used to support i/o operations from C/C++

Neil Rieck
Arne Vajhøj
2023-12-31 14:49:55 UTC
Permalink
Post by John Dallman
Post by Arne Vajhøj
https://www.vajhoej.dk/arne/articles/rms.html
Good work. You might want to explain why StreamCR is provided: I doubt it
is because of traditional Mac format, because VMS predates Mac by several
years. Was it used by older DEC OSes?
Good question. I don't know.

RSX-11? RT-11? RSTS/E? TOPS-10?

Anyone?

Arne
John Dallman
2023-12-31 21:34:00 UTC
Permalink
Post by Arne Vajhøj
Post by John Dallman
Good work. You might want to explain why StreamCR is provided: I
doubt it is because of traditional Mac format, because VMS
predates Mac by several years. Was it used by older DEC OSes?
Good question. I don't know.
RSX-11? RT-11? RSTS/E? TOPS-10?
On reflection, I think CP/M inherited using CR from a DEC OS, but I don't
know which one: the original author of CP/M was familiar with multiple
operating systems.

John
Lawrence D'Oliveiro
2023-12-31 21:50:36 UTC
Permalink
Post by John Dallman
On reflection, I think CP/M inherited using CR from a DEC OS, but I
don't know which one: the original author of CP/M was familiar with
multiple operating systems.
CP/M used CR/LF, which is why MS-DOS did the same, and Windows still does.

As I recall, Gary Kildall did his early cross-development of CP/M on a
PDP-10 machine running TOPS-10.

Clearly, DEC OSes gave him the idea of device names, though he simplified
them down to a single letter for his 8-bit OS, just for disks. But then
you needed ways of accessing non-disk devices like serial lines, printers,
etc; so either he or Microsoft introduced “reserved” file names for these.
And that’s where you get “PRN”, “COM1” and all the rest.

Then when Microsoft tried to copy some Unix features in MS-DOS 2.x, like
hierarchical pathnames, these interacted in rather unfortunate ways with
the “reserved” file names. And that bit of sadness persists in Windows to
this day. Along with those 8-bit-era single-letter drive names, for some
reason.

Microsoft: “26 drive letters ought to be enough for anybody!”
Chris Townley
2023-12-31 22:53:01 UTC
Permalink
Post by Lawrence D'Oliveiro
Post by John Dallman
On reflection, I think CP/M inherited using CR from a DEC OS, but I
don't know which one: the original author of CP/M was familiar with
multiple operating systems.
CP/M used CR/LF, which is why MS-DOS did the same, and Windows still does.
As I recall, Gary Kildall did his early cross-development of CP/M on a
PDP-10 machine running TOPS-10.
Clearly, DEC OSes gave him the idea of device names, though he simplified
them down to a single letter for his 8-bit OS, just for disks. But then
you needed ways of accessing non-disk devices like serial lines, printers,
etc; so either he or Microsoft introduced “reserved” file names for these.
And that’s where you get “PRN”, “COM1” and all the rest.
Then when Microsoft tried to copy some Unix features in MS-DOS 2.x, like
hierarchical pathnames, these interacted in rather unfortunate ways with
the “reserved” file names. And that bit of sadness persists in Windows to
this day. Along with those 8-bit-era single-letter drive names, for some
reason.
Microsoft: “26 drive letters ought to be enough for anybody!”
Along with 64k...
--
Chris
Lawrence D'Oliveiro
2023-12-31 23:04:18 UTC
Permalink
Post by Chris Townley
Post by Lawrence D'Oliveiro
Microsoft: “26 drive letters ought to be enough for anybody!”
Along with 64k...
You mean “640k” ... no, it’s unlikely Bill Gates would ever have said such
a thing.
Arne Vajhøj
2024-01-01 00:38:00 UTC
Permalink
Post by Lawrence D'Oliveiro
Along with those 8-bit-era single-letter drive names, for some
reason.
Microsoft: “26 drive letters ought to be enough for anybody!”
You can mount disks without using drive letters with NTFS.

It is just that nobody use it.

Probably because 26 letters actually are more than enough.

Arne
Lawrence D'Oliveiro
2024-01-01 02:05:59 UTC
Permalink
Post by Arne Vajhøj
Along with those 8-bit-era single-letter drive names, for some reason.
Microsoft: “26 drive letters ought to be enough for anybody!”
You can mount disks without using drive letters with NTFS.
As I recall from reading docs, that doesn’t work with network shares or
hot-pluggable storage. Or this thing called “storage spaces”.

And remember, it’s NTFS-specific. Windows lacks the equivalent of a fully
general VFS layer, for some totally unfathomable reason.
Arne Vajhøj
2024-01-01 02:36:33 UTC
Permalink
Post by Lawrence D'Oliveiro
Post by Arne Vajhøj
Along with those 8-bit-era single-letter drive names, for some reason.
Microsoft: “26 drive letters ought to be enough for anybody!”
You can mount disks without using drive letters with NTFS.
As I recall from reading docs, that doesn’t work with network shares or
hot-pluggable storage. Or this thing called “storage spaces”.
You don't need it for a network share - those can just be accessed
directly via UNC.

(it is said that one can make a symlink to UNC if one want to)
Post by Lawrence D'Oliveiro
And remember, it’s NTFS-specific.
It is specific to the file system used on practically all Windows
disks.
Post by Lawrence D'Oliveiro
Windows lacks the equivalent of a fully
general VFS layer, for some totally unfathomable reason.
I don't think many Windows users miss it.

Arne
Lawrence D'Oliveiro
2024-01-01 02:40:57 UTC
Permalink
Post by Arne Vajhøj
Post by Lawrence D'Oliveiro
Post by Arne Vajhøj
Along with those 8-bit-era single-letter drive names, for some reason.
Microsoft: “26 drive letters ought to be enough for anybody!”
You can mount disks without using drive letters with NTFS.
As I recall from reading docs, that doesn’t work with network shares or
hot-pluggable storage. Or this thing called “storage spaces”.
You don't need it for a network share - those can just be accessed
directly via UNC.
So how do you select from available UNC shares in an application’s file
picker?
Post by Arne Vajhøj
(it is said that one can make a symlink to UNC if one want to)
Windows symlinks have their own share of problems.
Post by Arne Vajhøj
Post by Lawrence D'Oliveiro
And remember, it’s NTFS-specific.
It is specific to the file system used on practically all Windows disks.
That’s a pretty shortsighted attitude, when you realize that NTFS is
starting to show its age.
Post by Arne Vajhøj
Post by Lawrence D'Oliveiro
Windows lacks the equivalent of a fully general VFS layer, for some
totally unfathomable reason.
I don't think many Windows users miss it.
That’s true of the Windows users/developers that are left. The ones that
understand the need, know where to get it. That’s why Microsoft is trying
desperately (with WSL2 and other things) to make Windows more like Linux.
Arne Vajhøj
2024-01-01 02:52:53 UTC
Permalink
Post by Lawrence D'Oliveiro
Post by Arne Vajhøj
Post by Lawrence D'Oliveiro
Post by Arne Vajhøj
Along with those 8-bit-era single-letter drive names, for some reason.
Microsoft: “26 drive letters ought to be enough for anybody!”
You can mount disks without using drive letters with NTFS.
As I recall from reading docs, that doesn’t work with network shares or
hot-pluggable storage. Or this thing called “storage spaces”.
You don't need it for a network share - those can just be accessed
directly via UNC.
So how do you select from available UNC shares in an application’s file
picker?
Post by Arne Vajhøj
(it is said that one can make a symlink to UNC if one want to)
Windows symlinks have their own share of problems.
If one is a Windows user that:
* need more than 26 drives
* need to use a file picker to choose network drives
* don't like to create symlinks
then one has a problem.

But my guess is that you can invite everyone in the world matching
that criteria to coffee in your dining room.
Post by Lawrence D'Oliveiro
Post by Arne Vajhøj
Post by Lawrence D'Oliveiro
And remember, it’s NTFS-specific.
It is specific to the file system used on practically all Windows disks.
That’s a pretty shortsighted attitude, when you realize that NTFS is
starting to show its age.
AFAIK there are no current plans to replace NTFS.

And when they eventually replace it, then it seems likely that
the NTFS feature set will be made requirements for the new
file system. Backwards compatibility is a big thing for Microsoft.
Post by Lawrence D'Oliveiro
Post by Arne Vajhøj
Post by Lawrence D'Oliveiro
Windows lacks the equivalent of a fully general VFS layer, for some
totally unfathomable reason.
I don't think many Windows users miss it.
That’s true of the Windows users/developers that are left. The ones that
understand the need, know where to get it. That’s why Microsoft is trying
desperately (with WSL2 and other things) to make Windows more like Linux.
WSL is not an attempt to make Windows like Linux.

WSL is a tool for developers that need to develop for both
Windows and Linux on Windows.

Arne
Lawrence D'Oliveiro
2024-01-01 03:09:31 UTC
Permalink
Post by Arne Vajhøj
* need more than 26 drives
Or even just wants something more meaningful than a single-letter name for
their volumes.
Post by Arne Vajhøj
AFAIK there are no current plans to replace NTFS.
Not that Microsoft hasn’t tried, somewhat. See ReFS.
Post by Arne Vajhøj
WSL is not an attempt to make Windows like Linux.
It is precisely that. It’s even in the name: guess what the “L” stands
for?
Arne Vajhøj
2024-01-01 03:35:34 UTC
Permalink
Post by Lawrence D'Oliveiro
Post by Arne Vajhøj
WSL is not an attempt to make Windows like Linux.
It is precisely that. It’s even in the name: guess what the “L” stands
for?
It stands for Linux, but that does not mean that they are trying to
move the general Windows user to a Linux experience.

It has a rather specific target group.

To quote from the WSL FAQ:

<quote>
Who is WSL for?

This is primarily a tool for developers, especially web developers,
those working on open source projects, or deploying to Linux server
environments. WSL is for anyone who likes using Bash, common Linux tools
(sed, awk, etc.) and Linux-first frameworks (Ruby, Python, etc.) but
also enjoys using Windows productivity tools
</quote>

https://learn.microsoft.com/en-us/windows/wsl/faq

Arne
Lawrence D'Oliveiro
2024-01-01 04:51:45 UTC
Permalink
It stands for Linux, but that does not mean that they are trying to move
the general Windows user to a Linux experience.
That seems inevitable, though. At some point it is going to become a
mandatory part of any Windows install. I’m not saying it was Microsoft’s
conscious intention when they introduced it, but it will become the path
of least resistance.
Chris Townley
2024-01-01 12:25:44 UTC
Permalink
Post by Lawrence D'Oliveiro
It stands for Linux, but that does not mean that they are trying to move
the general Windows user to a Linux experience.
That seems inevitable, though. At some point it is going to become a
mandatory part of any Windows install. I’m not saying it was Microsoft’s
conscious intention when they introduced it, but it will become the path
of least resistance.
If I wanted to create something in Linux, I would not use WSL, I would
use a proper distribution, either under a VM or on the various physical
linux devices I have
--
Chris
Arne Vajhøj
2024-01-01 14:55:44 UTC
Permalink
Post by Chris Townley
Post by Lawrence D'Oliveiro
It stands for Linux, but that does not mean that they are trying to move
the general Windows user to a Linux experience.
That seems inevitable, though. At some point it is going to become a
mandatory part of any Windows install. I’m not saying it was Microsoft’s
conscious intention when they introduced it, but it will become the path
of least resistance.
If I wanted to create something in Linux, I would not use WSL, I would
use a proper distribution, either under a VM or on the various physical
linux devices I have
You sort of get both with WSL.

You get a VM with a Linux distro of your choice (among the
approx. dozen that MS supports) that behaves pretty much like
any other Linux VM. Except that there are no visible startup time.

But you also get the ability to use Linux commands in
your CMD windows. Example:

wsl ls -la

They do some magic to utilize the Linux stuff in the VM.

I would not use WSL to run a server of any kind. But to
do a Linux build of some code, then why not.

Arne
Single Stage to Orbit
2024-01-02 11:10:11 UTC
Permalink
Post by Arne Vajhøj
You sort of get both with WSL.
No, WSL is terrible! Use WSL2 instead.
--
Tactical Nuclear Kittens
Arne Vajhøj
2024-01-02 12:28:37 UTC
Permalink
Post by Single Stage to Orbit
Post by Arne Vajhøj
You sort of get both with WSL.
No, WSL is terrible! Use WSL2 instead.
I guess I was not being precise enough.

I generally use the term WSL to cover both WSL1 and WSL2.

And here I was actually talking about WSL2 per the
"You get a VM with ...".

Arne
Andy Burns
2024-01-02 12:46:14 UTC
Permalink
Post by Single Stage to Orbit
Post by Arne Vajhøj
You sort of get both with WSL.
No, WSL is terrible! Use WSL2 instead.
Performance-wise sure, but there is a certain technical elegance to the
WSL1 mechanism.
Arne Vajhøj
2024-01-02 13:24:40 UTC
Permalink
Post by Andy Burns
Post by Single Stage to Orbit
Post by Arne Vajhøj
You sort of get both with WSL.
No, WSL is terrible! Use WSL2 instead.
Performance-wise sure, but there is a certain technical elegance to the
WSL1 mechanism.
Was the problem with WSL1 performance?

I always thought that the problem was compatibility
(that 99.9% compatibility was not god enough).

Arne
Craig A. Berry
2024-01-02 19:40:33 UTC
Permalink
Post by Arne Vajhøj
Post by Andy Burns
Post by Single Stage to Orbit
Post by Arne Vajhøj
You sort of get both with WSL.
No, WSL is terrible! Use WSL2 instead.
Performance-wise sure, but there is a certain technical elegance to
the WSL1 mechanism.
Was the problem with WSL1 performance?
I always thought that the problem was compatibility
(that 99.9% compatibility was not god enough).
The docs say that WSL2 performance is worse than WSL1 for operations
involving file system integration. Details here:

https://github.com/microsoft/WSL/issues/4197#issuecomment-604592340
Single Stage to Orbit
2024-01-03 00:30:41 UTC
Permalink
Post by Craig A. Berry
Post by Arne Vajhøj
Post by Andy Burns
Post by Single Stage to Orbit
Post by Arne Vajhøj
You sort of get both with WSL.
No, WSL is terrible! Use WSL2 instead.
Performance-wise sure, but there is a certain technical elegance to
the WSL1 mechanism.
Was the problem with WSL1 performance?
I always thought that the problem was compatibility
(that 99.9% compatibility was not god enough).
The docs say that WSL2 performance is worse than WSL1 for operations
https://github.com/microsoft/WSL/issues/4197#issuecomment-604592340
Eh? I thought it was WSL1 that had the performance issuses, not WSL2.
--
Tactical Nuclear Kittens
Dan Cross
2024-01-03 13:29:59 UTC
Permalink
Post by Single Stage to Orbit
Post by Craig A. Berry
Post by Arne Vajhøj
Post by Andy Burns
Post by Single Stage to Orbit
Post by Arne Vajhøj
You sort of get both with WSL.
No, WSL is terrible! Use WSL2 instead.
Performance-wise sure, but there is a certain technical elegance to
the WSL1 mechanism.
Was the problem with WSL1 performance?
I always thought that the problem was compatibility
(that 99.9% compatibility was not god enough).
The docs say that WSL2 performance is worse than WSL1 for operations
https://github.com/microsoft/WSL/issues/4197#issuecomment-604592340
Eh? I thought it was WSL1 that had the performance issuses, not WSL2.
Got a citation? I'd believe it's the other way around; a world
switch into Linux from Windows is going to be more expensive
than a system call, and accessing a filesystem remotely (surely
how they implement access to NTFS from WSL2) will mean
ping-ponging between Windows and Linux several times to satisfy
an IO request.

- Dan C.
Single Stage to Orbit
2024-01-03 14:45:07 UTC
Permalink
Post by Single Stage to Orbit
Post by Craig A. Berry
https://github.com/microsoft/WSL/issues/4197#issuecomment-604592340
Eh? I thought it was WSL1 that had the performance issuses, not WSL2.
Got a citation?  I'd believe it's the other way around; a world
switch into Linux from Windows is going to be more expensive
than a system call, and accessing a filesystem remotely (surely
how they implement access to NTFS from WSL2) will mean
ping-ponging between Windows and Linux several times to satisfy
an IO request.
With WSL2, both the Windows NT kernel and the Linux kernel run on top
of a hypervisor. It's a nice solution and performance differences
between the two WSL systems should be night and day.

https://thecodeblogger.com/2020/08/22/understanding-differences-between-wsl-1-and-wsl-2/
--
Tactical Nuclear Kittens
Dan Cross
2024-01-03 15:16:30 UTC
Permalink
Post by Single Stage to Orbit
Post by Single Stage to Orbit
Post by Craig A. Berry
https://github.com/microsoft/WSL/issues/4197#issuecomment-604592340
Eh? I thought it was WSL1 that had the performance issuses, not WSL2.
Got a citation?  I'd believe it's the other way around; a world
switch into Linux from Windows is going to be more expensive
than a system call, and accessing a filesystem remotely (surely
how they implement access to NTFS from WSL2) will mean
ping-ponging between Windows and Linux several times to satisfy
an IO request.
With WSL2, both the Windows NT kernel and the Linux kernel run on top
of a hypervisor. It's a nice solution and performance differences
between the two WSL systems should be night and day.
https://thecodeblogger.com/2020/08/22/understanding-differences-between-wsl-1-and-wsl-2/
That doesn't really explain why you'd think that WSL1 would be
appreciably slower (or slower at all) than WSL2. Consider that
Windows is still responsible for running IO devices and
providing access to e.g. the filesystem for Linux in the latter;
this will necessarily require lots of context switching between
the two (where in this context "context" means running in either
a Linux or Windows guest under a shared hypervisor) as Linux
accesses services provided by Windows. In WSL1 there would be
no such overhead.

- Dan C.
Single Stage to Orbit
2024-01-04 09:29:02 UTC
Permalink
Post by Dan Cross
Post by Single Stage to Orbit
With WSL2, both the Windows NT kernel and the Linux kernel run on top
of a hypervisor. It's a nice solution and performance differences
between the two WSL systems should be night and day.
https://thecodeblogger.com/2020/08/22/understanding-differences-between-wsl-1-and-wsl-2/
That doesn't really explain why you'd think that WSL1 would be
appreciably slower (or slower at all) than WSL2.  Consider that
Windows is still responsible for running IO devices and
providing access to e.g. the filesystem for Linux in the latter;
this will necessarily require lots of context switching between
the two (where in this context "context" means running in either
a Linux or Windows guest under a shared hypervisor) as Linux
accesses services provided by Windows.  In WSL1 there would be
no such overhead.
You've tickled my curiousity now. When I've time I'll spin up a couple
Windows VMs, one with WSL1, and one with WSL2, and do some benchmarks.
See who's right.
--
Tactical Nuclear Kittens
Lawrence D'Oliveiro
2024-01-02 19:11:40 UTC
Permalink
... there is a certain technical elegance to the
WSL1 mechanism.
When Windows NT was first created, it was supposed to support the idea of
“personalities” to allow different kinds of userland apps to run on top of
the same core kernel. For example, Win32 was just one kind of
“personality”, while the POSIX layer was implemented as another kind.

You would think the WSL1 would have been implemented as just another
“personality”. But no. It appears the whole extensibility of the
“personality” architecture has kind of bitrotted away in the years since
the introduction of NT, so WSL1 was implemented in its own unique kind of
way (don’t ask me how).

And, in spite of the fact that it was supposed to be implementing a well-
documented API, with plenty of example source code to refer to, they still
couldn’t get WSL1 to work right. So they had to chuck it away and bring in
a proper Linux kernel instead.
Andy Burns
2024-01-02 19:25:02 UTC
Permalink
Post by Lawrence D'Oliveiro
... there is a certain technical elegance to the
WSL1 mechanism.
When Windows NT was first created, it was supposed to support the idea of
“personalities” to allow different kinds of userland apps to run on top of
the same core kernel. For example, Win32 was just one kind of
“personality”, while the POSIX layer was implemented as another kind.
You would think the WSL1 would have been implemented as just another
“personality”. But no. It appears the whole extensibility of the
“personality” architecture has kind of bitrotted away in the years since
the introduction of NT,
They've tipped the nomenclature on its head now "windows subsystem for
linux" rather than "posix subsystem for windows"

I've read that the posix subsystem did the absolute bare minimum to pass
tests, without being in any way in any way useful, just so that WinNT
could tick the "posix" box and be allowed into govt contracts.

search for "ncommander posix"
Post by Lawrence D'Oliveiro
so WSL1 was implemented in its own unique kind of
way (don’t ask me how).
And, in spite of the fact that it was supposed to be implementing a well-
documented API, with plenty of example source code to refer to, they still
couldn’t get WSL1 to work right. So they had to chuck it away and bring in
a proper Linux kernel instead.
I didn't do anything other than "play" with WSL1, but I thought
performance was the issue.

I think I did more with MKS Toolkit and Cygwin than WSL.
Lawrence D'Oliveiro
2024-01-02 19:42:41 UTC
Permalink
Post by Andy Burns
I've read that the posix subsystem did the absolute bare minimum to pass
tests, without being in any way in any way useful, just so that WinNT
could tick the "posix" box and be allowed into govt contracts.
The true horror of it has to be seen to be believed

Post by Andy Burns
I think I did more with MKS Toolkit and Cygwin than WSL.
Interestingly, Cygwin seems to do a better job in some ways than Windows
can manage natively, like get select(2)/poll(2) to work with pipes. Maybe
that was the sort of thing that sank WSL1?
Andy Burns
2024-01-02 19:57:22 UTC
Permalink
Post by Lawrence D'Oliveiro
Post by Andy Burns
I've read that the posix subsystem did the absolute bare minimum to pass
tests, without being in any way in any way useful, just so that WinNT
could tick the "posix" box and be allowed into govt contracts.
The true horror of it has to be seen to be believed
http://youtu.be/BOeku3hDzrM
Yes, that's the "ncommander posix" search that I referred to
Arne Vajhøj
2024-01-03 00:18:01 UTC
Permalink
Post by Andy Burns
Post by Lawrence D'Oliveiro
And, in spite of the fact that it was supposed to be implementing a well-
documented API, with plenty of example source code to refer to, they still
couldn’t get WSL1 to work right. So they had to chuck it away and bring in
a proper Linux kernel instead.
I didn't do anything other than "play" with WSL1, but I thought
performance was the issue.
I think I did more with MKS Toolkit and Cygwin than WSL.
Note that Cygwin and WSL does completely different things.

Cygwin:

*nix and/or Windows source--Cygwin toolchain-->Windows executable

WSL:

*nix source--Linux toolchain-->Linux executable

Arne
Andy Burns
2024-01-03 09:03:17 UTC
Permalink
Post by Arne Vajhøj
Note that Cygwin and WSL does completely different things.
Sure, but they all fall into my "mixing windowsy things with unixy
things" slot ... also I forgot all about Interix/SFU
Lawrence D'Oliveiro
2024-01-03 19:03:22 UTC
Permalink
Post by Andy Burns
... also I forgot all about Interix/SFU
Apparently Interix/SFU was the result of a temporary lapse on the part of
Microsoft. In those days, it was possible to get documentation from them
for how to write your own “personality” on top of the NT kernel, which
Interix did.

Then Microsoft decided it didn’t want this knowledge becoming publicly
known. So they acquired Interix to stop it spreading further.
Dan Cross
2024-01-03 13:31:44 UTC
Permalink
Post by Arne Vajhøj
Post by Andy Burns
Post by Lawrence D'Oliveiro
And, in spite of the fact that it was supposed to be implementing a well-
documented API, with plenty of example source code to refer to, they still
couldn’t get WSL1 to work right. So they had to chuck it away and
bring in
a proper Linux kernel instead.
I didn't do anything other than "play" with WSL1, but I thought
performance was the issue.
I think I did more with MKS Toolkit and Cygwin than WSL.
Note that Cygwin and WSL does completely different things.
*nix and/or Windows source--Cygwin toolchain-->Windows executable
*nix source--Linux toolchain-->Linux executable
More like: "cygwin executes Windows executables that are built
from Unix-y sources using compatibility libraries, while WSL1
executes Linux-branded ELF binaries using a compatibility layer
in the Windows kernel." Toolchains are only tangentially
relevant.

- Dan C.
Dan Cross
2024-01-02 19:41:21 UTC
Permalink
Post by Lawrence D'Oliveiro
... there is a certain technical elegance to the
WSL1 mechanism.
When Windows NT was first created, it was supposed to support the idea of
"personalities" to allow different kinds of userland apps to run on top of
the same core kernel. For example, Win32 was just one kind of
"personality", while the POSIX layer was implemented as another kind.
You would think the WSL1 would have been implemented as just another
"personality". But no. It appears the whole extensibility of the
"personality" architecture has kind of bitrotted away in the years since
the introduction of NT, so WSL1 was implemented in its own unique kind of
way (don't ask me how).
With the caveat that I have no first-hand experience with the
WSL implementation in the Windows kernel, the mechanism is
likely to be fairly straight-forward. When the system goes to
invoke an executable program, it examines the image file; for
both PECOFF (e.g., Windows .EXE format) and Linux-styled ELF
binaries, the image formats are pretty well defined and the
kernel can easily distinguish between them. Depending on which
it is attempting to run, it can select the appropriate loader
and it can install a per-process system call handler. Traps
into the OS for system calls will then consult the local system
call table for that process and decide how to handle the call.

The interesting bit is inside the kernel, where the different
interfaces are handled using (presumably) a common set of
functionality.
Post by Lawrence D'Oliveiro
And, in spite of the fact that it was supposed to be implementing a well-
documented API, with plenty of example source code to refer to, they still
couldn't get WSL1 to work right. So they had to chuck it away and bring in
a proper Linux kernel instead.
This is a well-understood problem in the Linux space; gVisor
suffers from a similar issue.

The problem is not that Linux's behavior is well-documented, it
is also what is not documented. It's not enough to be
consistent with the documented interfaces, but one must also be
"bug-compatible" with the system. Further, Linux has a
long-standing policy of preserving userspace behaviors, even
when doing so means that the implementation contradicts a
documented interface in some way or another.

Furthermore, the rate of change in Linux is high; following
along from outside is fraught. And we haven't even gotten
to things like:
https://docs.kernel.org/process/stable-api-nonsense.html

- Dan C.
Lawrence D'Oliveiro
2024-01-02 20:36:24 UTC
Permalink
Furthermore, the rate of change in Linux is high; following along from
outside is fraught.
Strange, isn’t it: Microsoft must have access to at least a couple of
orders of magnitude greater development resources than that available to
the Linux kernel project, yet they cannot keep up with what the Linux
developers have achieved.
Dan Cross
2024-01-02 20:56:56 UTC
Permalink
Furthermore, the rate of change in Linux is high; following along from
outside is fraught.
Strange, isn't it: Microsoft must have access to at least a couple of
orders of magnitude greater development resources than that available to
the Linux kernel project,
Almost certainly the inverse is true. Indeed, this is _why_, if
I were MSFT, I would be looking at how I could leverage the rest
of the world's investment in Linux while minimizing my own cost
for maintaining Windows.
yet they cannot keep up with what the Linux developers have achieved.
Don't sell Microsoft short: their kernel engineers are
first-rate and incredibly sharp. But they only have so many,
and regardless of how many they have, they have fewer than
Linux.

On the other hand, don't look at Linux with rose-colored
glasses. The idea of the lone Linux kernel hacker independently
building something that rivals commercial vendors with nothing
more than a bag of dorittos and bottle of Jolt is a distant
memory. The reality is that _most_ Linux development is heavily
subsidized by commercial interests these days; getting something
non-trivial into the kernel requires lots of time and resources.
In theory an individual could do it, but in practice that's
pretty rare.

- Dan C.
Lawrence D'Oliveiro
2024-01-02 21:18:28 UTC
Permalink
Post by Dan Cross
Furthermore, the rate of change in Linux is high; following along from
outside is fraught.
Strange, isn't it: Microsoft must have access to at least a couple of
orders of magnitude greater development resources than that available to
the Linux kernel project,
Almost certainly the inverse is true.
Yet you yourself have already admitted the opposite.
Post by Dan Cross
Don't sell Microsoft short: their kernel engineers are first-rate and
incredibly sharp.
Maybe not quite so sharp; else why can’t they figure out how to get
select(2)/poll(2) working with pipes <https://docs.python.org/3/library/
select.html>? That is the one irritating reason why users of Python
asyncio on Windows have to make a choice between two different, partially-
overlapping event-loop implementations, neither of which is really a
general solution <https://docs.python.org/3/library/asyncio-
eventloop.html#event-loop-implementations>.
Post by Dan Cross
On the other hand, don't look at Linux with rose-colored glasses. The
idea of the lone Linux kernel hacker independently building something
that rivals commercial vendors with nothing more than a bag of dorittos
and bottle of Jolt is a distant memory.
It still happens. Look at who created WireGuard, just for a recent
example.
Dan Cross
2024-01-02 23:16:45 UTC
Permalink
Post by Lawrence D'Oliveiro
Post by Dan Cross
Furthermore, the rate of change in Linux is high; following along from
outside is fraught.
Strange, isn't it: Microsoft must have access to at least a couple of
orders of magnitude greater development resources than that available to
the Linux kernel project,
Almost certainly the inverse is true.
Yet you yourself have already admitted the opposite.
I don't believe I have. Perhaps you could tell me where you
think I did?
Post by Lawrence D'Oliveiro
Post by Dan Cross
Don't sell Microsoft short: their kernel engineers are first-rate and
incredibly sharp.
Maybe not quite so sharp;
I've worked closely with at least six engineers who worked on
the Windows kernel. I can assure you that they are excellent,
and based on what they told me, they were representative of
most Windows kernel people.

I recongize that's anecdotal, but I see no disconfirming
evidence.
Post by Lawrence D'Oliveiro
else why can’t they figure out how to get
select(2)/poll(2) working with pipes <https://docs.python.org/3/library/
select.html>?
Probably because they started out with a very different system
architecture and din't care about implementing Unix interfaces
at the time, and now have to retrofit it onto a very large
existing codebase and that's kind of a niche use case.

I'm curious: have you ever actually had a serious technical
conversation with someone who's worked on Windows? Did you ask
them this question, and if so, what was their answer?
Post by Lawrence D'Oliveiro
That is the one irritating reason why users of Python
asyncio on Windows have to make a choice between two different, partially-
overlapping event-loop implementations, neither of which is really a
general solution <https://docs.python.org/3/library/asyncio-
eventloop.html#event-loop-implementations>.
When you write that about select/poll and pipes and that being
an "irritating reason" for a more general issue, that seems
specious. Different systems do things differently; sadly Python
exposes that to the programmer. Oh well, but that's about par
for the course.
Post by Lawrence D'Oliveiro
Post by Dan Cross
On the other hand, don't look at Linux with rose-colored glasses. The
idea of the lone Linux kernel hacker independently building something
that rivals commercial vendors with nothing more than a bag of dorittos
and bottle of Jolt is a distant memory.
It still happens. Look at who created WireGuard, just for a recent
example.
What about WireGuard? The implementation was funded.

Anyway, anecdotal evidence, at best, I'm afraid. I'm using
conversations with primary sources for my assertion (Ts'o et
al).

- Dan C.
Lawrence D'Oliveiro
2024-01-02 23:30:53 UTC
Permalink
Post by Dan Cross
Post by Lawrence D'Oliveiro
Post by Dan Cross
Furthermore, the rate of change in Linux is high; following along from
outside is fraught.
Strange, isn't it: Microsoft must have access to at least a couple of
orders of magnitude greater development resources than that available to
the Linux kernel project,
Almost certainly the inverse is true.
Yet you yourself have already admitted the opposite.
I don't believe I have. Perhaps you could tell me where you
think I did?
Where you used the word “fraught”.
Post by Dan Cross
Post by Lawrence D'Oliveiro
else why can’t they figure out how to get
select(2)/poll(2) working with pipes <https://docs.python.org/3/library/
select.html>?
Probably because they started out with a very different system
architecture and din't care about implementing Unix interfaces
at the time, and now have to retrofit it onto a very large
existing codebase and that's kind of a niche use case.
That “niche” is the reason why they have had to resort to WSL2, to bring
Linux-type APIs to Windows. And why do they need Linux-type APIs on
Windows, anyway? Because that’s what the developers are increasingly
relying on. Why didn’t WSL1 work? Because the Windows kernel wasn’t up to
it.
Post by Dan Cross
When you write that about select/poll and pipes and that being
an "irritating reason" for a more general issue, that seems
specious.
Maybe you don’t realize how important Python has become in the computing
world lately, with its applications in data science and other areas.
Microsoft is even planning to offer Python access to Excel users (at least
cloud-based ones).

And it’s not just Python, of course: Windows limitations affect cross-
platform development across the board. To the point where Microsoft has to
do something to stem the tide of developers simply giving up on the
Windows platform. Hence WSL. Which is essentially a bag duct-taped on the
side of Windows.

Remember why Microsoft needs WSL, clunky as it is: it’s not something it
bestowed as a special favour on the Linux or open-source world or anything
like that: it created it because it had to, for sheer business survival.
Dan Cross
2024-01-03 00:03:36 UTC
Permalink
Post by Dan Cross
Post by Lawrence D'Oliveiro
Post by Dan Cross
Furthermore, the rate of change in Linux is high; following along
from
Post by Dan Cross
Post by Lawrence D'Oliveiro
Post by Dan Cross
outside is fraught.
Strange, isn't it: Microsoft must have access to at least a couple of
orders of magnitude greater development resources than that available
to
Post by Dan Cross
Post by Lawrence D'Oliveiro
Post by Dan Cross
the Linux kernel project,
Almost certainly the inverse is true.
Yet you yourself have already admitted the opposite.
I don't believe I have. Perhaps you could tell me where you
think I did?
Where you used the word "fraught".
I don't see it.

What I said is that it is a fraught proposition to try to keep
up with the rate of change in the Linux kernel from outside of
the Linux kernel.

On the other hand, you said,

|Microsoft must have access to at least a couple
|of orders of magnitude greater development resources than that
|available to the Linux kernel project

To which I replied that almost certainly the inverse is true;
i.e., that the Linux kernel folks probably have access to
several orders of magnitude more resources than Microsoft can
put to Windows.

The two are unrelated.
Post by Dan Cross
Post by Lawrence D'Oliveiro
else why can’t they figure out how to get
select(2)/poll(2) working with pipes <https://docs.python.org/3/library/
select.html>?
Probably because they started out with a very different system
architecture and din't care about implementing Unix interfaces
at the time, and now have to retrofit it onto a very large
existing codebase and that's kind of a niche use case.
That "niche" is the reason why they have had to resort to WSL2, to bring
Linux-type APIs to Windows. And why do they need Linux-type APIs on
Windows, anyway? Because that's what the developers are increasingly
relying on. Why didn't WSL1 work? Because the Windows kernel wasn't up to
it.
That seems like speculation on your part. Do you have any
special expertise in this area?

They surely moved to WSL2 because the best, cheapest way to be
"bug compatible" with Linux is to just run Linux. That really
doesn't say much about what the Windows kernel is, or is not,
capable of. It might say something about their development
priorities, though.
Post by Dan Cross
When you write that about select/poll and pipes and that being
an "irritating reason" for a more general issue, that seems
specious.
Maybe you don't realize how important Python has become in the computing
world lately, with its applications in data science and other areas.
Non sequitur. Python is important; this has does not imply that
Unix-style select/poll on a pipe under Windows in Python is
important. For more general types of asynchronous IO (storage,
networking) it says nothing at all, and these are certainly more
important than pipes which are just one kind of IPC mechanism
(and not super relevant to Python specifically).
Microsoft is even planning to offer Python access to Excel users (at least
cloud-based ones).
Again, this is conflating "python" (which I understand runs on
Windows just fine; I wouldn't know, as I don't do Windows) with
pipes+select and python.
And it's not just Python, of course: Windows limitations affect cross-
platform development across the board. To the point where Microsoft has to
do something to stem the tide of developers simply giving up on the
Windows platform. Hence WSL. Which is essentially a bag duct-taped on the
side of Windows.
This seems like pure speculation.
Remember why Microsoft needs WSL, clunky as it is: it's not something it
bestowed as a special favour on the Linux or open-source world or anything
like that: it created it because it had to, for sheer business survival.
I actually agree with this, but I think your argument here is
poor. MSFT needs Linux compatability because the world is
trending towards Linux and they can't keep up, yes. It does not
follow that their engineers are bad, or even that their kernel
is bad. It does mean that their kernel was designed with
different goals than Linux (which is a Unix knock-off).

- Dan C.
Lawrence D'Oliveiro
2024-01-03 02:56:25 UTC
Permalink
What I said is that it is a fraught proposition to try to keep up with
the rate of change in the Linux kernel from outside of the Linux kernel.
And that “outside of the Linux kernel” includes the part of the Universe
containing Microsoft. It, too, has a struggle, even with its vast
resources, keeping up with the pace of Linux kernel development.
To which I replied that almost certainly the inverse is true; i.e., that
the Linux kernel folks probably have access to several orders of
magnitude more resources than Microsoft can put to Windows.
Given how much revenue Microsoft is supposedly earning from Windows, don’t
you think it should be putting a proportionate level of investment back
into it?
That "niche" is the reason why they have had to resort to WSL2, to bring
Linux-type APIs to Windows. And why do they need Linux-type APIs on
Windows, anyway? Because that's what the developers are increasingly
relying on. Why didn't WSL1 work? Because the Windows kernel wasn't up
to it.
That seems like speculation on your part.
Microsoft’s own actions are all the evidence we need. Why abandon
something, after putting so much effort into it, if you could have got it
to work?
They surely moved to WSL2 because the best, cheapest way to be "bug
compatible" with Linux is to just run Linux.
After already putting all that investment into WSL1? That good old NT
kernel not versatile enough to emulate the little quirks as well as the
salient features?

As further evidence, consider how WINE is able to be bug-compatible with
Windows, on top of the Linux kernel. Why can’t Microsoft, with more
resources at its disposal than both the Linux kernel and WINE projects
*combined*, return the favour?
Python is important; this has does not imply that
Unix-style select/poll on a pipe under Windows in Python is important.
So how do you run that large installed base of Python code on Windows
without that compatibility?
For more general types of asynchronous IO (storage, networking) it says
nothing at all, and these are certainly more important than pipes which
are just one kind of IPC mechanism (and not super relevant to Python
specifically).
They’re a core feature of POSIX. They exist because they make for a very
convenient model for certain common kinds of IPC, as the original Unix
creators discovered in their experiments.
Remember why Microsoft needs WSL, clunky as it is: it's not something it
bestowed as a special favour on the Linux or open-source world or
anything like that: it created it because it had to, for sheer business
survival.
I actually agree with this, but I think your argument here is poor.
MSFT needs Linux compatability because the world is trending towards
Linux and they can't keep up, yes. It does not follow that their
engineers are bad, or even that their kernel is bad.
WSL1 would be evidence to the contrary.

And yes, engineers can be individually smart, yet due to a dysfunctional
and risk-averse corporate culture, they end up producing a mediocre net
product.
Dan Cross
2024-01-03 13:27:25 UTC
Permalink
What I said is that it is a fraught proposition to try to keep up with
the rate of change in the Linux kernel from outside of the Linux kernel.
And that "outside of the Linux kernel" includes the part of the Universe
containing Microsoft. It, too, has a struggle, even with its vast
resources, keeping up with the pace of Linux kernel development.
A) you are clearly speculating about the "resources" you think
that Microsoft can apply to Windows. B) you have not
acknowledged that, whatever resources MSFT can bring to bear,
Linux can _bring more_.
To which I replied that almost certainly the inverse is true; i.e., that
the Linux kernel folks probably have access to several orders of
magnitude more resources than Microsoft can put to Windows.
Given how much revenue Microsoft is supposedly earning from Windows, don't
you think it should be putting a proportionate level of investment back
into it?
A amateurish attempt to change the subject, but regardless,
that's a business, not a technical, decision at Microsoft. It
could be that they see the writing on the wall and don't feel
that it's a good investment.
That "niche" is the reason why they have had to resort to WSL2, to bring
Linux-type APIs to Windows. And why do they need Linux-type APIs on
Windows, anyway? Because that's what the developers are increasingly
relying on. Why didn't WSL1 work? Because the Windows kernel wasn't up
to it.
That seems like speculation on your part.
Microsoft's own actions are all the evidence we need. Why abandon
something, after putting so much effort into it, if you could have got it
to work?
Your logical fallacy is assuming that because they _dont't_ do
something that that implies that they _can't_ do that thing.
There are many reasons that they may choose not to do something
that are totally independent from their ability to do that
thing.
They surely moved to WSL2 because the best, cheapest way to be "bug
compatible" with Linux is to just run Linux.
After already putting all that investment into WSL1? That good old NT
kernel not versatile enough to emulate the little quirks as well as the
salient features?
Strawman. We've already discussed why they decided to move to
WSL2. Here, their inability to keep up with Linux is not about
technical ability, it's about resources, which (again) they have
fewer of than the rest of the world combined applies to Linux.
As further evidence, consider how WINE is able to be bug-compatible with
Windows, on top of the Linux kernel. Why can't Microsoft, with more
resources at its disposal than both the Linux kernel and WINE projects
*combined*, return the favour?
Again, you are assuming that Microsoft has "more resources at
its disposal" than Linux, but I see no evidence of that; in fact
I see significant evidence to the contrary.

Go ahead and browse the commit lots in the Linux kernel git
repo: how many author addresses end with meta.com, amazon.com,
google.com, ibm.com, intel.com, amd.com, redhat.com, nvidia.com,
adobe.come, etc? Or, for that matter, microsoft.com? And we're
not even talking about the smaller companies (altera, et al).

Even for those that don't, how many of the remaining offers work
for one of the FAANG companies? Do you really think that
Microsoft has more resources _combined_ working on Windows than
the rest of those companies are devoting to Linux?

As for WINE, it's compatibilty is not great.
Python is important; this has does not imply that
Unix-style select/poll on a pipe under Windows in Python is important.
So how do you run that large installed base of Python code on Windows
without that compatibility?
Since the vast bulk of Python code out there probably doesn't
try to mix pipes with select or poll, I'd imagine you just run
it.
For more general types of asynchronous IO (storage, networking) it says
nothing at all, and these are certainly more important than pipes which
are just one kind of IPC mechanism (and not super relevant to Python
specifically).
They're a core feature of POSIX. They exist because they make for a very
convenient model for certain common kinds of IPC, as the original Unix
creators discovered in their experiments.
"...in their experiments." Um, sure. Doug proposed it and Ken
implemented it in a weekend. It was relevatory in the context
of Unix, but not necessarily in other systems.

But so what? We're talking about Python code here, which mostly
insulates the programmer from something as low-level as POSIX.
Again, it seems like the thing that the vast, vast majority of
the tiny fraction of Python programmers who deal with such
things are going to care about is asynchronous IO for networking
and storage, not pipes. Honing in on that one aspect of
incompatibility doesn't really go very far to prove a general
point.
Remember why Microsoft needs WSL, clunky as it is: it's not something it
bestowed as a special favour on the Linux or open-source world or
anything like that: it created it because it had to, for sheer business
survival.
I actually agree with this, but I think your argument here is poor.
MSFT needs Linux compatability because the world is trending towards
Linux and they can't keep up, yes. It does not follow that their
engineers are bad, or even that their kernel is bad.
WSL1 would be evidence to the contrary.
Not really; see above.
And yes, engineers can be individually smart, yet due to a dysfunctional
and risk-averse corporate culture, they end up producing a mediocre net
product.
Maybe, but that's a different matter. I think the probability
that you know someone who's worked on both Linux and Windows at
the kernel level is approximately 0, as evidenced by your lack
of understanding of the technical and business issues involved.

- Dan C.
Scott Dorsey
2024-01-02 22:05:31 UTC
Permalink
Furthermore, the rate of change in Linux is high; following along from
outside is fraught.
Strange, isn’t it: Microsoft must have access to at least a couple of
orders of magnitude greater development resources than that available to
the Linux kernel project, yet they cannot keep up with what the Linux
developers have achieved.
This presumes that the high rate of change of Linux is actually good. I
rather suspect it is not.
--scott
--
"C'est un Nagra. C'est suisse, et tres, tres precis."
Lawrence D'Oliveiro
2024-01-02 23:03:30 UTC
Permalink
Post by Scott Dorsey
Post by Lawrence D'Oliveiro
Furthermore, the rate of change in Linux is high; following along from
outside is fraught.
Strange, isn’t it: Microsoft must have access to at least a couple of
orders of magnitude greater development resources than that available to
the Linux kernel project, yet they cannot keep up with what the Linux
developers have achieved.
This presumes that the high rate of change of Linux is actually good. I
rather suspect it is not.
You can think in terms of passive users of a project, versus active
contributors to the project.

In the open-source world, those passive users don’t matter so much: the
ones that matter are the active contributors.

The passive users also include the type who like to complain about things
and bemoan the fact that the project doesn’t cater to their needs.
Dan Cross
2024-01-02 23:21:23 UTC
Permalink
Post by Scott Dorsey
Furthermore, the rate of change in Linux is high; following along from
outside is fraught.
Strange, isn’t it: Microsoft must have access to at least a couple of
orders of magnitude greater development resources than that available to
the Linux kernel project, yet they cannot keep up with what the Linux
developers have achieved.
This presumes that the high rate of change of Linux is actually good. I
rather suspect it is not.
I think there's a lot of churn. What's missing is a lot of
principled engineering; Linux has evolved, and it's impressive
what it has evolved into (honestly the world's most important
operating system) but that doesn't necessarily mean that it is
_good_ in all respects (it is objectively not).

Take, for instance, the lack of internal kernel interfaces and
gregkh's insistence that, not only are they unnecessary, they
are _bad_: this can make things very hard to work with, with
multiple generations of partial interfaces all coresident at the
same time, and little incentive to clean up the resulting
complexity or mess. Honestly, I'm often surprised that it works
as well as it does. Compare to Solaris, on the other hand, that
has documented stable interfaces all over. Sure, there are some
cases where they're not used (or used well...) but this means
that binary modules compiled out-of-tree can work across
multiple versions of the operating system, which is pretty cool.
And neigh impossible for non-trivial things in the Linux world.

- Dan C.
Lawrence D'Oliveiro
2024-01-03 02:46:26 UTC
Permalink
Take, for instance, the lack of internal kernel interfaces and gregkh's
insistence that, not only are they unnecessary, they are _bad_: this can
make things very hard to work with, with multiple generations of partial
interfaces all coresident at the same time, and little incentive to
clean up the resulting complexity or mess.
Could be worse. You could be required to carry all that baggage around
forever.

GregKH made this point some years back, about the state of respective USB
support in Windows versus Linux. At that point, Microsoft had rewritten
its USB stack three times, and so it was carrying around three different
driver APIs, and as long as there were third-party drivers using the
obsolete interfaces, would have to continue doing that essentially
forever.

At the same time, Linux had also rewritten its USB stack three times. But
it had the luxury of starting pretty much from scratch each time, without
having to carry around a lot of legacy baggage, because all the drivers
included in the Linux kernel source tree could be updated at the same
time.

Just one of many reasons why Linux is a much trimmer and more efficient OS
than Windows.
Single Stage to Orbit
2024-01-03 08:54:23 UTC
Permalink
Post by Lawrence D'Oliveiro
At the same time, Linux had also rewritten its USB stack three times.
But it had the luxury of starting pretty much from scratch each time,
without having to carry around a lot of legacy baggage, because all
the drivers included in the Linux kernel source tree could be updated
at the same time.
That's true but it is a source of considerable pain for commercial
products such as VMware, VirtualBox, ZFS and others. They all have to
devote substantial engineering resources to release updates for every
kernel release.
--
Tactical Nuclear Kittens
Lawrence D'Oliveiro
2024-01-03 19:01:05 UTC
Permalink
Post by Single Stage to Orbit
Post by Lawrence D'Oliveiro
At the same time, Linux had also rewritten its USB stack three times.
But it had the luxury of starting pretty much from scratch each time,
without having to carry around a lot of legacy baggage, because all the
drivers included in the Linux kernel source tree could be updated at
the same time.
That's true but it is a source of considerable pain for commercial
products such as VMware, VirtualBox, ZFS and others. They all have to
devote substantial engineering resources to release updates for every
kernel release.
The usual recommendation is that they should contribute the necessary
drivers to the mainline Linux kernel--along with a commitment to keep
maintaining them.

Microsoft discovered this the hard way, when it got some Hyper-V drivers
accepted into the kernel, and then forgot about the part that every piece
of code in the kernel has to have a responsive maintainer. It took the
threat of the code being dropped from the kernel to make them sit up and
take notice.
Single Stage to Orbit
2024-01-04 09:30:20 UTC
Permalink
Post by Lawrence D'Oliveiro
Post by Single Stage to Orbit
That's true but it is a source of considerable pain for commercial
products such as VMware, VirtualBox, ZFS and others. They all have
to devote substantial engineering resources to release updates for
every kernel release.
The usual recommendation is that they should contribute the necessary
drivers to the mainline Linux kernel--along with a commitment to keep
maintaining them.
Microsoft discovered this the hard way, when it got some Hyper-V
drivers accepted into the kernel, and then forgot about the part that
every piece of code in the kernel has to have a responsive
maintainer. It took the threat of the code being dropped from the
kernel to make them sit up and take notice.
It's the same old story, costs != profit and beancounters are not
always right.
--
Tactical Nuclear Kittens
Lawrence D'Oliveiro
2024-01-04 19:28:27 UTC
Permalink
Post by Lawrence D'Oliveiro
Post by Single Stage to Orbit
That's true but it is a source of considerable pain for commercial
products such as VMware, VirtualBox, ZFS and others. They all have to
devote substantial engineering resources to release updates for every
kernel release.
The usual recommendation is that they should contribute the necessary
drivers to the mainline Linux kernel--along with a commitment to keep
maintaining them.
Microsoft discovered this the hard way, when it got some Hyper-V
drivers accepted into the kernel, and then forgot about the part that
every piece of code in the kernel has to have a responsive maintainer.
It took the threat of the code being dropped from the kernel to make
them sit up and take notice.
It's the same old story, costs != profit and beancounters are not always
right.
And yet you’d think those beancounters would notice the very opportunity
to save money mentioned above, and stop having to “devote substantial
engineering resources to release updates for every kernel release”.
Dan Cross
2024-01-03 13:35:40 UTC
Permalink
Post by Lawrence D'Oliveiro
Take, for instance, the lack of internal kernel interfaces and gregkh's
insistence that, not only are they unnecessary, they are _bad_: this can
make things very hard to work with, with multiple generations of partial
interfaces all coresident at the same time, and little incentive to
clean up the resulting complexity or mess.
Could be worse. You could be required to carry all that baggage around
forever.
GregKH made this point some years back, about the state of respective USB
support in Windows versus Linux. At that point, Microsoft had rewritten
its USB stack three times, and so it was carrying around three different
driver APIs, and as long as there were third-party drivers using the
obsolete interfaces, would have to continue doing that essentially
forever.
At the same time, Linux had also rewritten its USB stack three times. But
it had the luxury of starting pretty much from scratch each time, without
having to carry around a lot of legacy baggage, because all the drivers
included in the Linux kernel source tree could be updated at the same
time.
Just one of many reasons why Linux is a much trimmer and more efficient OS
than Windows.
You are conflating _internal_ interfaces and _external_
interfaces. Linux has none of the former; it has strong
backwards compatibility guarantees for the latter ("thou shalt
not break userspace").

The lack of internal interfaces mean that it's difficult to
evolve kernel internals without significant churn. E.g.,
filesystems are deeply entwined with the virtual memory
subsystem, as is networking; everything speaks directly to
the process code, etc.

Linux is extraordinarily bloated.

- Dan C.
Lawrence D'Oliveiro
2024-01-01 21:51:42 UTC
Permalink
Post by Chris Townley
If I wanted to create something in Linux, I would not use WSL, I would
use a proper distribution, either under a VM or on the various physical
linux devices I have
Certainly, and so would I. But WSL is the path by which the Windows
installed base will get there from here.
Arne Vajhøj
2024-01-01 14:48:53 UTC
Permalink
Post by Lawrence D'Oliveiro
It stands for Linux, but that does not mean that they are trying to move
the general Windows user to a Linux experience.
That seems inevitable, though. At some point it is going to become a
mandatory part of any Windows install. I’m not saying it was Microsoft’s
conscious intention when they introduced it, but it will become the path
of least resistance.
MS could roll it out to every Windows user tomorrow if they
wanted to.

But why on earth would they want to do that??

For servers the preference is real ESXi/Hyper-V/KVM not WSL.

For desktop/laptop the vast majorities of users has no interest
in Linux at all. Windows are facing serious challenges, but not
from (traditional) Linux. Windows usage is dropping because
people are switching to Android/iOS phones/tablets.

People are switching from a GUI centric OS (Windows) to
GUI only OS (Android & iOS) for casual use. Expecting them
to use WSL command line utilities is a non-starter.

Arne
Dan Cross
2024-01-01 16:12:41 UTC
Permalink
Post by Arne Vajhøj
Post by Lawrence D'Oliveiro
It stands for Linux, but that does not mean that they are trying to move
the general Windows user to a Linux experience.
That seems inevitable, though. At some point it is going to become a
mandatory part of any Windows install. I’m not saying it was Microsoft’s
conscious intention when they introduced it, but it will become the path
of least resistance.
MS could roll it out to every Windows user tomorrow if they
wanted to.
But why on earth would they want to do that??
To pave a way for MSFT to jetison Windows in favor of Linux.

Maintaining an OS like Windows is expensive and requires a
steady stream of talent. There is more talent working on that
kind of thing outside of Microsoft than inside, just not on
Windows: most of the work is happening around Linux. Being able
to leverage that investment would be a strategic win.
Post by Arne Vajhøj
For servers the preference is real ESXi/Hyper-V/KVM not WSL.
For desktop/laptop the vast majorities of users has no interest
in Linux at all. Windows are facing serious challenges, but not
from (traditional) Linux. Windows usage is dropping because
people are switching to Android/iOS phones/tablets.
People are switching from a GUI centric OS (Windows) to
GUI only OS (Android & iOS) for casual use. Expecting them
to use WSL command line utilities is a non-starter.
This conflates two things: WSL as a path for moving to Linux
as the kernel substrate for Microsoft's OS offerings, and using
WSL as an end user.

The latter is likely never going to happen outside of developer
communities. The former could well happen; WSL gives MSFT a
low-cost way to dip their toe into the waters and explore
interoperability between the traditional Windows API and Linux.

- Dan C.
Arne Vajhøj
2024-01-01 16:46:04 UTC
Permalink
Post by Dan Cross
Post by Arne Vajhøj
Post by Lawrence D'Oliveiro
It stands for Linux, but that does not mean that they are trying to move
the general Windows user to a Linux experience.
That seems inevitable, though. At some point it is going to become a
mandatory part of any Windows install. I’m not saying it was Microsoft’s
conscious intention when they introduced it, but it will become the path
of least resistance.
MS could roll it out to every Windows user tomorrow if they
wanted to.
But why on earth would they want to do that??
To pave a way for MSFT to jetison Windows in favor of Linux.
Maintaining an OS like Windows is expensive and requires a
steady stream of talent. There is more talent working on that
kind of thing outside of Microsoft than inside, just not on
Windows: most of the work is happening around Linux. Being able
to leverage that investment would be a strategic win.
It cost money to maintain Windows, but it also generates revenue.

A lot of revenue. MS sell Windows licenses in the magnitude of
20 B$ per year.

Becoming just another Linux distro vendor would loose
most of that revenue.

Does not make any business sense.
Post by Dan Cross
Post by Arne Vajhøj
For servers the preference is real ESXi/Hyper-V/KVM not WSL.
For desktop/laptop the vast majorities of users has no interest
in Linux at all. Windows are facing serious challenges, but not
from (traditional) Linux. Windows usage is dropping because
people are switching to Android/iOS phones/tablets.
People are switching from a GUI centric OS (Windows) to
GUI only OS (Android & iOS) for casual use. Expecting them
to use WSL command line utilities is a non-starter.
This conflates two things: WSL as a path for moving to Linux
as the kernel substrate for Microsoft's OS offerings, and using
WSL as an end user.
The latter is likely never going to happen outside of developer
communities. The former could well happen; WSL gives MSFT a
low-cost way to dip their toe into the waters and explore
interoperability between the traditional Windows API and Linux.
If MS wanted to switch to Linux kernel then Win32 API for Linux
would be very interesting.

But WSL does not provide anything for that.

WSL 1 provided the opposite direction - Linux API on Windows kernel.

WSL 2 is just a VM with a very smart/transparent integration.

Arne
Dan Cross
2024-01-01 16:52:50 UTC
Permalink
Post by Arne Vajhøj
Post by Dan Cross
Post by Arne Vajhøj
Post by Lawrence D'Oliveiro
It stands for Linux, but that does not mean that they are trying to move
the general Windows user to a Linux experience.
That seems inevitable, though. At some point it is going to become a
mandatory part of any Windows install. I’m not saying it was Microsoft’s
conscious intention when they introduced it, but it will become the path
of least resistance.
MS could roll it out to every Windows user tomorrow if they
wanted to.
But why on earth would they want to do that??
To pave a way for MSFT to jetison Windows in favor of Linux.
Maintaining an OS like Windows is expensive and requires a
steady stream of talent. There is more talent working on that
kind of thing outside of Microsoft than inside, just not on
Windows: most of the work is happening around Linux. Being able
to leverage that investment would be a strategic win.
It cost money to maintain Windows, but it also generates revenue.
A lot of revenue. MS sell Windows licenses in the magnitude of
20 B$ per year.
Becoming just another Linux distro vendor would loose
most of that revenue.
Does not make any business sense.
...right now.

The windows market share is, as you yourself mentioned earlier
in this thread, continuing to shrink.

People at Microsoft aren't stupid. They can see the long-term
direction themselves.
Post by Arne Vajhøj
Post by Dan Cross
Post by Arne Vajhøj
For servers the preference is real ESXi/Hyper-V/KVM not WSL.
For desktop/laptop the vast majorities of users has no interest
in Linux at all. Windows are facing serious challenges, but not
from (traditional) Linux. Windows usage is dropping because
people are switching to Android/iOS phones/tablets.
People are switching from a GUI centric OS (Windows) to
GUI only OS (Android & iOS) for casual use. Expecting them
to use WSL command line utilities is a non-starter.
This conflates two things: WSL as a path for moving to Linux
as the kernel substrate for Microsoft's OS offerings, and using
WSL as an end user.
The latter is likely never going to happen outside of developer
communities. The former could well happen; WSL gives MSFT a
low-cost way to dip their toe into the waters and explore
interoperability between the traditional Windows API and Linux.
If MS wanted to switch to Linux kernel then Win32 API for Linux
would be very interesting.
But WSL does not provide anything for that.
WSL 1 provided the opposite direction - Linux API on Windows kernel.
...and integration at the file level.
Post by Arne Vajhøj
WSL 2 is just a VM with a very smart/transparent integration.
As I said, it's a way for them to dip their toe in the water and
explore compatibility. It's obviously not the end state.

- Dan C.
Arne Vajhøj
2024-01-01 19:26:45 UTC
Permalink
Post by Dan Cross
Post by Arne Vajhøj
Post by Dan Cross
Post by Arne Vajhøj
Post by Lawrence D'Oliveiro
It stands for Linux, but that does not mean that they are trying to move
the general Windows user to a Linux experience.
That seems inevitable, though. At some point it is going to become a
mandatory part of any Windows install. I’m not saying it was Microsoft’s
conscious intention when they introduced it, but it will become the path
of least resistance.
MS could roll it out to every Windows user tomorrow if they
wanted to.
But why on earth would they want to do that??
To pave a way for MSFT to jetison Windows in favor of Linux.
Maintaining an OS like Windows is expensive and requires a
steady stream of talent. There is more talent working on that
kind of thing outside of Microsoft than inside, just not on
Windows: most of the work is happening around Linux. Being able
to leverage that investment would be a strategic win.
It cost money to maintain Windows, but it also generates revenue.
A lot of revenue. MS sell Windows licenses in the magnitude of
20 B$ per year.
Becoming just another Linux distro vendor would loose
most of that revenue.
Does not make any business sense.
...right now.
The windows market share is, as you yourself mentioned earlier
in this thread, continuing to shrink.
People at Microsoft aren't stupid. They can see the long-term
direction themselves.
Windows desktop market share is not shrinking much. The
problem for MS is that the desktop market itself is shrinking - a
lot of casual users are switching to phones and tablets. Most
still have a PC, but because it is not used so much then they
are kept way longer than they used to. So PC sales is dropping
and Windows sales is dropping.

If MS want to continue the big presence in the consumer segment,
then they need to address it. Somehow they need to get into
the phone and tablet market. This is why the MS CEO recently
said that he regretted killing Windows Phone.

Creating a Linux desktop distro is not going to help with this
problem, because it is not what the customers are looking for.
Post by Dan Cross
Post by Arne Vajhøj
Post by Dan Cross
Post by Arne Vajhøj
For desktop/laptop the vast majorities of users has no interest
in Linux at all. Windows are facing serious challenges, but not
from (traditional) Linux. Windows usage is dropping because
people are switching to Android/iOS phones/tablets.
People are switching from a GUI centric OS (Windows) to
GUI only OS (Android & iOS) for casual use. Expecting them
to use WSL command line utilities is a non-starter.
This conflates two things: WSL as a path for moving to Linux
as the kernel substrate for Microsoft's OS offerings, and using
WSL as an end user.
The latter is likely never going to happen outside of developer
communities. The former could well happen; WSL gives MSFT a
low-cost way to dip their toe into the waters and explore
interoperability between the traditional Windows API and Linux.
If MS wanted to switch to Linux kernel then Win32 API for Linux
would be very interesting.
But WSL does not provide anything for that.
WSL 1 provided the opposite direction - Linux API on Windows kernel.
...and integration at the file level.
Post by Arne Vajhøj
WSL 2 is just a VM with a very smart/transparent integration.
As I said, it's a way for them to dip their toe in the water and
explore compatibility. It's obviously not the end state.
"dip their toe in the water" and "explore compatibility" ????

It does not do anything for running Windows applications on a Linux
system.

Arne
Dan Cross
2024-01-01 20:15:27 UTC
Permalink
Post by Arne Vajhøj
[snip]
...right now.
The windows market share is, as you yourself mentioned earlier
in this thread, continuing to shrink.
People at Microsoft aren't stupid. They can see the long-term
direction themselves.
Windows desktop market share is not shrinking much. The
problem for MS is that the desktop market itself is shrinking - a
lot of casual users are switching to phones and tablets. Most
still have a PC, but because it is not used so much then they
are kept way longer than they used to. So PC sales is dropping
and Windows sales is dropping.
If MS want to continue the big presence in the consumer segment,
then they need to address it. Somehow they need to get into
the phone and tablet market. This is why the MS CEO recently
said that he regretted killing Windows Phone.
Creating a Linux desktop distro is not going to help with this
problem, because it is not what the customers are looking for.
You seem to conflate, "using the Linux kernel" with "creating
a Linux desktop distro."
Post by Arne Vajhøj
Post by Arne Vajhøj
Post by Dan Cross
Post by Arne Vajhøj
For desktop/laptop the vast majorities of users has no interest
in Linux at all. Windows are facing serious challenges, but not
from (traditional) Linux. Windows usage is dropping because
people are switching to Android/iOS phones/tablets.
People are switching from a GUI centric OS (Windows) to
GUI only OS (Android & iOS) for casual use. Expecting them
to use WSL command line utilities is a non-starter.
This conflates two things: WSL as a path for moving to Linux
as the kernel substrate for Microsoft's OS offerings, and using
WSL as an end user.
The latter is likely never going to happen outside of developer
communities. The former could well happen; WSL gives MSFT a
low-cost way to dip their toe into the waters and explore
interoperability between the traditional Windows API and Linux.
If MS wanted to switch to Linux kernel then Win32 API for Linux
would be very interesting.
But WSL does not provide anything for that.
WSL 1 provided the opposite direction - Linux API on Windows kernel.
...and integration at the file level.
Post by Arne Vajhøj
WSL 2 is just a VM with a very smart/transparent integration.
As I said, it's a way for them to dip their toe in the water and
explore compatibility. It's obviously not the end state.
"dip their toe in the water" and "explore compatibility" ????
It does not do anything for running Windows applications on a Linux
system.
Bluntly, you're long on opinion, but based on this and many
other answers in this newsgroup and others, I really don't think
you have a good handle on how any of the underlying technologies
actually work. If you did, perhaps you wouldn't be so quick to
dismiss how a kernel compatibility layer that implements a
kernel ABI (e.g., WSL1) could be leveraged to gain insight into
how two kernel interfaces could be made to work simultaneously,
or how virtualization across a kernel boundary with shared
services (e.g., WSL2) could be similarly leveraged.

- Dan C.
Arne Vajhøj
2024-01-02 00:34:58 UTC
Permalink
Post by Dan Cross
Post by Arne Vajhøj
[snip]
...right now.
The windows market share is, as you yourself mentioned earlier
in this thread, continuing to shrink.
People at Microsoft aren't stupid. They can see the long-term
direction themselves.
Windows desktop market share is not shrinking much. The
problem for MS is that the desktop market itself is shrinking - a
lot of casual users are switching to phones and tablets. Most
still have a PC, but because it is not used so much then they
are kept way longer than they used to. So PC sales is dropping
and Windows sales is dropping.
If MS want to continue the big presence in the consumer segment,
then they need to address it. Somehow they need to get into
the phone and tablet market. This is why the MS CEO recently
said that he regretted killing Windows Phone.
Creating a Linux desktop distro is not going to help with this
problem, because it is not what the customers are looking for.
You seem to conflate, "using the Linux kernel" with "creating
a Linux desktop distro."
(if they want to reduce cost significantly, then that would
be what they would need to do)

But no matter what they do to their desktop OS, then it does not
solve the problem of people moving from desktop OS
to phones and tablets.
Post by Dan Cross
Post by Arne Vajhøj
Post by Arne Vajhøj
Post by Dan Cross
Post by Arne Vajhøj
For desktop/laptop the vast majorities of users has no interest
in Linux at all. Windows are facing serious challenges, but not
from (traditional) Linux. Windows usage is dropping because
people are switching to Android/iOS phones/tablets.
People are switching from a GUI centric OS (Windows) to
GUI only OS (Android & iOS) for casual use. Expecting them
to use WSL command line utilities is a non-starter.
This conflates two things: WSL as a path for moving to Linux
as the kernel substrate for Microsoft's OS offerings, and using
WSL as an end user.
The latter is likely never going to happen outside of developer
communities. The former could well happen; WSL gives MSFT a
low-cost way to dip their toe into the waters and explore
interoperability between the traditional Windows API and Linux.
If MS wanted to switch to Linux kernel then Win32 API for Linux
would be very interesting.
But WSL does not provide anything for that.
WSL 1 provided the opposite direction - Linux API on Windows kernel.
...and integration at the file level.
Post by Arne Vajhøj
WSL 2 is just a VM with a very smart/transparent integration.
As I said, it's a way for them to dip their toe in the water and
explore compatibility. It's obviously not the end state.
"dip their toe in the water" and "explore compatibility" ????
It does not do anything for running Windows applications on a Linux
system.
Bluntly, you're long on opinion, but based on this and many
other answers in this newsgroup and others, I really don't think
you have a good handle on how any of the underlying technologies
actually work. If you did, perhaps you wouldn't be so quick to
dismiss how a kernel compatibility layer that implements a
kernel ABI (e.g., WSL1) could be leveraged to gain insight into
how two kernel interfaces could be made to work simultaneously,
or how virtualization across a kernel boundary with shared
services (e.g., WSL2) could be similarly leveraged.
I do find it difficult to see why running Linux apps
on Windows help with running Windows apps on Linux.

And they have known about both concepts for a long
time. WSL1 concept goes back to 90's - Win32 API on
top of NT kernel and Win9x kernel. WSL2 concept goes
back to late 00's - XP mode in Windows 7.

Arne

on Windows 7
Dan Cross
2024-01-02 17:41:20 UTC
Permalink
Post by Arne Vajhøj
Post by Dan Cross
Post by Arne Vajhøj
[snip]
...right now.
The windows market share is, as you yourself mentioned earlier
in this thread, continuing to shrink.
People at Microsoft aren't stupid. They can see the long-term
direction themselves.
Windows desktop market share is not shrinking much. The
problem for MS is that the desktop market itself is shrinking - a
lot of casual users are switching to phones and tablets. Most
still have a PC, but because it is not used so much then they
are kept way longer than they used to. So PC sales is dropping
and Windows sales is dropping.
If MS want to continue the big presence in the consumer segment,
then they need to address it. Somehow they need to get into
the phone and tablet market. This is why the MS CEO recently
said that he regretted killing Windows Phone.
Creating a Linux desktop distro is not going to help with this
problem, because it is not what the customers are looking for.
You seem to conflate, "using the Linux kernel" with "creating
a Linux desktop distro."
(if they want to reduce cost significantly, then that would
be what they would need to do)
Nope. Again, you're conflating replacing the Windows
kernel with a producing "desktop Linux distro."
Post by Arne Vajhøj
But no matter what they do to their desktop OS, then it does not
solve the problem of people moving from desktop OS
to phones and tablets.
Indeed. So as their market share continues to shrink,
support costs will continue to be significant. Moreover,
Windows revenue is not growing as fast as other parts of
their business, and as applications move to the cloud,
there is less incentive for users to run Windows.
Post by Arne Vajhøj
Post by Dan Cross
Post by Arne Vajhøj
Post by Arne Vajhøj
Post by Dan Cross
Post by Arne Vajhøj
For desktop/laptop the vast majorities of users has no interest
in Linux at all. Windows are facing serious challenges, but not
from (traditional) Linux. Windows usage is dropping because
people are switching to Android/iOS phones/tablets.
People are switching from a GUI centric OS (Windows) to
GUI only OS (Android & iOS) for casual use. Expecting them
to use WSL command line utilities is a non-starter.
This conflates two things: WSL as a path for moving to Linux
as the kernel substrate for Microsoft's OS offerings, and using
WSL as an end user.
The latter is likely never going to happen outside of developer
communities. The former could well happen; WSL gives MSFT a
low-cost way to dip their toe into the waters and explore
interoperability between the traditional Windows API and Linux.
If MS wanted to switch to Linux kernel then Win32 API for Linux
would be very interesting.
But WSL does not provide anything for that.
WSL 1 provided the opposite direction - Linux API on Windows kernel.
...and integration at the file level.
Post by Arne Vajhøj
WSL 2 is just a VM with a very smart/transparent integration.
As I said, it's a way for them to dip their toe in the water and
explore compatibility. It's obviously not the end state.
"dip their toe in the water" and "explore compatibility" ????
It does not do anything for running Windows applications on a Linux
system.
Bluntly, you're long on opinion, but based on this and many
other answers in this newsgroup and others, I really don't think
you have a good handle on how any of the underlying technologies
actually work. If you did, perhaps you wouldn't be so quick to
dismiss how a kernel compatibility layer that implements a
kernel ABI (e.g., WSL1) could be leveraged to gain insight into
how two kernel interfaces could be made to work simultaneously,
or how virtualization across a kernel boundary with shared
services (e.g., WSL2) could be similarly leveraged.
I do find it difficult to see why running Linux apps
on Windows help with running Windows apps on Linux.
Yes, you do, but it's not that complex. If you have to provide
compatibility for one system interface in terms of another, you
must necessarily understand what can be shared and how to share
it. Implementing the Linux system interface inside of the
Windows kernel (as one does with WSL1) gives MSFT engineers
insight into how to bridge between the two; if you do so in one
direction, you've necessarily learned a lot about how to do it
in the other direction. I'm sure in doing so, Microsoft also
took lessons from e.g. gVisor etc.

Of course, the issue of system interface fidelity is a big one,
which is why WSL2 simply boots the Linux kernel in a VM. But
if you do that, then conceptually you have two kernels running
in an orchestrated way on a single machine, and now you can
start to vary the interface between them and which takes
primacy. This allows you to transition from one to the other
in a principled way, while maintaining compatibility.

Note that the Windows API is, by design, opaque to application
programs: programs like with a DLL that interfaces with the
kernel as opposed to publishing how to do this directly (as
under Linux). This gives MSFT a lot of room in how they
implement their system while maintaining backwards compatibility
with application programs. There's nothing magical about the
Windows kernel that would preclude them providing the same
interface on a different kernel entirely, and there's lots of
prior art (e.g., plan 9 ports, Rosetta and the Mac and A/UX
on the Mac, even within Windows, etc).

I do not expect you to understand this.
Post by Arne Vajhøj
And they have known about both concepts for a long
time. WSL1 concept goes back to 90's - Win32 API on
top of NT kernel and Win9x kernel.
Not really, but the basic concept of emulating a system
interface on another system goes much further back than
that; e.g., PA1050 on TOPS-20, or the DTSS environment on
Multics.

- Dan C.
bill
2024-01-02 18:44:51 UTC
Permalink
Post by Dan Cross
Post by Arne Vajhøj
And they have known about both concepts for a long
time. WSL1 concept goes back to 90's - Win32 API on
top of NT kernel and Win9x kernel.
Not really, but the basic concept of emulating a system
interface on another system goes much further back than
that; e.g., PA1050 on TOPS-20, or the DTSS environment on
Multics.
And lets not forget The Software Tools Virtual Operating System
in 1980 that gave you a bunch of the Unix API on everything from
CP/M to Exec-8 on the Univac 1100 including all of the DEC systems
of the time. :-)

bill
Dan Cross
2024-01-02 19:42:29 UTC
Permalink
Post by bill
Post by Dan Cross
Post by Arne Vajhøj
And they have known about both concepts for a long
time. WSL1 concept goes back to 90's - Win32 API on
top of NT kernel and Win9x kernel.
Not really, but the basic concept of emulating a system
interface on another system goes much further back than
that; e.g., PA1050 on TOPS-20, or the DTSS environment on
Multics.
And lets not forget The Software Tools Virtual Operating System
in 1980 that gave you a bunch of the Unix API on everything from
CP/M to Exec-8 on the Univac 1100 including all of the DEC systems
of the time. :-)
Indeed! Though that was more about a suite of useful
functionality for source-level compatibility, more than running
unmodified binaries on compatible machines.

- Dan C.
bill
2024-01-02 20:34:19 UTC
Permalink
Post by Dan Cross
Post by bill
Post by Dan Cross
Post by Arne Vajhøj
And they have known about both concepts for a long
time. WSL1 concept goes back to 90's - Win32 API on
top of NT kernel and Win9x kernel.
Not really, but the basic concept of emulating a system
interface on another system goes much further back than
that; e.g., PA1050 on TOPS-20, or the DTSS environment on
Multics.
And lets not forget The Software Tools Virtual Operating System
in 1980 that gave you a bunch of the Unix API on everything from
CP/M to Exec-8 on the Univac 1100 including all of the DEC systems
of the time. :-)
Indeed! Though that was more about a suite of useful
functionality for source-level compatibility, more than running
unmodified binaries on compatible machines.
One of the big "selling" points for Unix was always source
compatibility. In the early Linux days there were compatibility
compatibility libraries that were supposed to make Linux stuff
run on BSD. I never had much luck with it and usually found it
easier to just recompile under BSD.

bill
Dan Cross
2024-01-02 21:01:17 UTC
Permalink
Post by bill
Post by Dan Cross
Post by bill
Post by Dan Cross
Post by Arne Vajhøj
And they have known about both concepts for a long
time. WSL1 concept goes back to 90's - Win32 API on
top of NT kernel and Win9x kernel.
Not really, but the basic concept of emulating a system
interface on another system goes much further back than
that; e.g., PA1050 on TOPS-20, or the DTSS environment on
Multics.
And lets not forget The Software Tools Virtual Operating System
in 1980 that gave you a bunch of the Unix API on everything from
CP/M to Exec-8 on the Univac 1100 including all of the DEC systems
of the time. :-)
Indeed! Though that was more about a suite of useful
functionality for source-level compatibility, more than running
unmodified binaries on compatible machines.
One of the big "selling" points for Unix was always source
compatibility. In the early Linux days there were compatibility
compatibility libraries that were supposed to make Linux stuff
run on BSD. I never had much luck with it and usually found it
easier to just recompile under BSD.
Yeah. It's much easier to take source and get it to run
somewhere else, though increasingly often I find Linux
assumptions baked into a lot of code. But Linux is the big boy
on the block, so they can get away with it.

I similarly found the Linux binary compatibility stuff in BSD
did a mediocre job. Similarly with other compability things
over the years: I brought up Torek's port of 4.4BSD-Lite on a
SPARCstation 1 by backporting some bits from -Encumbered, and
while it _could_ run SunOS 4.x binaries, they dumped core
disturbingly often.

- Dan C.
Lawrence D'Oliveiro
2024-01-02 19:20:40 UTC
Permalink
Post by Dan Cross
Note that the Windows API is, by design, opaque to application
programs ...
Worse than that, it is poorly specified. Compare POSIX, where it is not
just the functionality of each call that is documented, but also the
possible error conditions and what they mean. This is missing from the
Win32 API docs.

This is why the WINE project, for example, took 15 years to get to version
1.0: because they had to determine all the possible error paths in Win32
by exhaustive search.

Jeremy Allison goes into detail about this here
<https://www.samba.org/samba/news/articles/low_point/
tale_two_stds_os2.html>.
Dan Cross
2024-01-02 19:44:09 UTC
Permalink
Post by Lawrence D'Oliveiro
Post by Dan Cross
Note that the Windows API is, by design, opaque to application
programs ...
Worse than that, it is poorly specified. Compare POSIX, where it is not
just the functionality of each call that is documented, but also the
possible error conditions and what they mean. This is missing from the
Win32 API docs.
This is why the WINE project, for example, took 15 years to get to version
1.0: because they had to determine all the possible error paths in Win32
by exhaustive search.
Jeremy Allison goes into detail about this here
<https://www.samba.org/samba/news/articles/low_point/
tale_two_stds_os2.html>.
That's a separate issue. That is more about underspecification
of the documented programmer's interface, but I'm talking about
the opacity of the kernel/user interface.

- Dan C.
Andy Burns
2024-01-02 19:44:49 UTC
Permalink
Post by Lawrence D'Oliveiro
Compare POSIX, where it is not
just the functionality of each call that is documented, but also the
possible error conditions and what they mean
Yet NT *can* pass the NIST/FIPS/Posix tests, so is Posix that good a test?
Dan Cross
2024-01-02 19:48:50 UTC
Permalink
Post by Andy Burns
Post by Lawrence D'Oliveiro
Compare POSIX, where it is not
just the functionality of each call that is documented, but also the
possible error conditions and what they mean
Yet NT *can* pass the NIST/FIPS/Posix tests, so is Posix that good a test?
Not really. POSIX is sort of a least-common-denominator thing,
and has become increasingly less relevant over time. z/OS is
POSIX compliant, IIRC; make of that what you will. :-)

- Dan C.
John Dallman
2024-01-03 21:52:00 UTC
Permalink
Post by Arne Vajhøj
If MS want to continue the big presence in the consumer segment,
then they need to address it. Somehow they need to get into
the phone and tablet market. This is why the MS CEO recently
said that he regretted killing Windows Phone.
Then he still hasn't faced up to the reason for its failure. As far as I
can tell, that was because it was too much like desktop/laptop Windows.

iOS and Android both have pretty gentle learning curves. You don't need
to know much to start using them, and fairly small steps in learning open
up more activities quite quickly.

Windows Phone's GUI was ... kind of complicated. It makes sense to
someone who's been closely following the evolution of desktop Windows
since about the year 2000, and always sticking to the Microsoft defaults.
Like, say, Microsoft's UI designers and marketing people.

But it has been a pretty twisty path, and there's a lot of accumulated
strangeness. The result is not friendly to someone who has never used
Windows, and that's quite hard for someone who is very pro-Windows to
understand.

Me, I regarded Windows 2000 as the peak of their GUI design, and it's
been getting worse, on average, ever since.

John
Lawrence D'Oliveiro
2024-01-03 21:59:04 UTC
Permalink
Post by Arne Vajhøj
If MS want to continue the big presence in the consumer segment,
then they need to address it. Somehow they need to get into the phone
and tablet market. This is why the MS CEO recently said that he
regretted killing Windows Phone.
Then he still hasn't faced up to the reason for its failure.As far as I
can tell, that was because it was too much like desktop/laptop Windows.
There you have in a nutshell, this conflation between the core OS and the
GUI that should be a separate layer on top of the OS. Remember, the whole
“tiles” interface was developed as part of the emphasis on mobile, and
then Windows 8 tried to retrofit it onto the desktop, which people hated.

Imagine saying that Android is “too much like desktop/laptop/server/
embedded/supercomputer Linux”, because it is: they share essentially the
same kernel. Yet that didn’t stop Android from taking over the mobile
world: in fact, it probably helped.
Simon Clubley
2024-01-04 13:40:46 UTC
Permalink
Imagine saying that Android is ?too much like desktop/laptop/server/
embedded/supercomputer Linux?, because it is: they share essentially the
same kernel. Yet that didn?t stop Android from taking over the mobile
world: in fact, it probably helped.
Yes. I have the Alpine Linux userland running just fine on my Android
phone for some basic stuff (with the help of PRoot). Unfortunately,
I have to use vi on my phone because emacs requires a proper keyboard... :-)

Simon.
--
Simon Clubley, ***@remove_me.eisner.decus.org-Earth.UFP
Walking destinations on a map are further away than they appear.
Simon Clubley
2024-01-04 13:35:14 UTC
Permalink
Post by John Dallman
Me, I regarded Windows 2000 as the peak of their GUI design, and it's
been getting worse, on average, ever since.
Unfortunately, that seems to be the case with modern GUI designs in
general.

For example, compare the disaster that is GTK4 with the much cleaner GTK2,
including how there is no regard to backwards compatibility or fully
duplicating existing functionality. Also, just how arrogant and out of
touch modern GUI designers are.

How to move from GTK3 to GTK4 (basically, rewrite your code):

https://docs.gtk.org/gtk4/migrating-3to4.html

One very arrogant GUI designer example from a couple of years ago:

https://gitlab.gnome.org/GNOME/gtk/-/issues/3787

Basically, more blurry and hard to read text in GTK4 wasn't considered
a problem by the current GTK developers and users were initally told
outright it was not going to be fixed. :-(

Simon.
--
Simon Clubley, ***@remove_me.eisner.decus.org-Earth.UFP
Walking destinations on a map are further away than they appear.
Lawrence D'Oliveiro
2024-01-01 21:57:06 UTC
Permalink
Post by Arne Vajhøj
Post by Lawrence D'Oliveiro
Post by Arne Vajhøj
It stands for Linux, but that does not mean that they are trying to
move the general Windows user to a Linux experience.
That seems inevitable, though. At some point it is going to become a
mandatory part of any Windows install. I’m not saying it was
Microsoft’s conscious intention when they introduced it, but it will
become the path of least resistance.
But why on earth would they want to do that??
Because Windows is becoming increasingly complex and expensive for
Microsoft to maintain, and the profits from doing so are shrinking. Every
Windows user has been noticing how Microsoft is cutting corners on its QA
lately, with the deteriorating quality of its updates and patches.

So it will naturally take decisions (in its typical short-sighted fashion)
to reduce that support burden. The obvious, easy one is to rely more and
more on that Linux kernel for core stuff.

Exhibit A, what I consider to be the first step on this path:
<https://www.theregister.com/2023/12/14/windows_ai_studio_preview/>.
Arne Vajhøj
2024-01-02 00:54:38 UTC
Permalink
Post by Lawrence D'Oliveiro
Post by Arne Vajhøj
Post by Lawrence D'Oliveiro
Post by Arne Vajhøj
It stands for Linux, but that does not mean that they are trying to
move the general Windows user to a Linux experience.
That seems inevitable, though. At some point it is going to become a
mandatory part of any Windows install. I’m not saying it was
Microsoft’s conscious intention when they introduced it, but it will
become the path of least resistance.
But why on earth would they want to do that??
Because Windows is becoming increasingly complex and expensive for
Microsoft to maintain,
That problem is something almost all software has. Code bases grow over
time.
Post by Lawrence D'Oliveiro
and the profits from doing so are shrinking. Every
Windows user has been noticing how Microsoft is cutting corners on its QA
lately, with the deteriorating quality of its updates and patches.
That is a hypothesis but the numbers does not support it.

Windows sale = 20 B$/year.

10000 engineers @ 200 K$/year = 2 B$

It does not seem plausible that maintenance cost of Windows
is a real problem for MS.

(and if Windows development are actual done in India instead of
Seattle the cost would be way less)
Post by Lawrence D'Oliveiro
So it will naturally take decisions (in its typical short-sighted fashion)
to reduce that support burden. The obvious, easy one is to rely more and
more on that Linux kernel for core stuff.
I can't see that.

If they create another Linux desktop distro, then sale would plummmet.
There are no money in Linux desktop distros.

The compatibility API solution would not reduce maintenance significant.

From:

COM + .NET
Win32
NT kernel

to:

COM + .NET
Win32
Linux kernel

MS would still need to maintain all the high level stuff where most
of the work are done.
Post by Lawrence D'Oliveiro
<https://www.theregister.com/2023/12/14/windows_ai_studio_preview/>.
That is a developer tool not a general user tool. So it is not an
indication of WSL being more than development.

Arne
Dan Cross
2024-01-02 17:45:58 UTC
Permalink
Post by Arne Vajhøj
Post by Lawrence D'Oliveiro
Post by Arne Vajhøj
Post by Lawrence D'Oliveiro
Post by Arne Vajhøj
It stands for Linux, but that does not mean that they are trying to
move the general Windows user to a Linux experience.
That seems inevitable, though. At some point it is going to become a
mandatory part of any Windows install. I’m not saying it was
Microsoft’s conscious intention when they introduced it, but it will
become the path of least resistance.
But why on earth would they want to do that??
Because Windows is becoming increasingly complex and expensive for
Microsoft to maintain,
That problem is something almost all software has. Code bases grow over
time.
Post by Lawrence D'Oliveiro
and the profits from doing so are shrinking. Every
Windows user has been noticing how Microsoft is cutting corners on its QA
lately, with the deteriorating quality of its updates and patches.
That is a hypothesis but the numbers does not support it.
Windows sale = 20 B$/year.
It does not seem plausible that maintenance cost of Windows
is a real problem for MS.
...right now. But no one is talking about Right Now. People
are talking about 5 or 10 years down the road.
Post by Arne Vajhøj
(and if Windows development are actual done in India instead of
Seattle the cost would be way less)
Post by Lawrence D'Oliveiro
So it will naturally take decisions (in its typical short-sighted fashion)
to reduce that support burden. The obvious, easy one is to rely more and
more on that Linux kernel for core stuff.
I can't see that.
If they create another Linux desktop distro, then sale would plummmet.
There are no money in Linux desktop distros.
You keep talking about "Linux desktop distros", but that's your
failure of vision, not the rest of the world's.
Post by Arne Vajhøj
The compatibility API solution would not reduce maintenance significant.
COM + .NET
Win32
NT kernel
COM + .NET
Win32
Linux kernel
MS would still need to maintain all the high level stuff where most
of the work are done.
A significant amount of work is done maintaining drivers, but
inside MSFT and for OEMs. Linux has become the most important
OS in the world, and writing drivers for Windows is notoriously
tedious. Taking the burden off of both OEMs and MSFT to provide
drivers for disparate hardware by outsourcing that to vendors
who are already motivated (read: de facto required, if they want
to be relevant) to do so would be a significant win for
everyone: Microsoft wouldn't have to care, and the vendors would
only have to produce one driver.

- Dan C.
Lawrence D'Oliveiro
2024-01-02 19:15:20 UTC
Permalink
A significant amount of work is done maintaining drivers, but inside
MSFT and for OEMs. Linux has become the most important OS in the world,
and writing drivers for Windows is notoriously tedious.
In the words of the infamous “Halloween documents”
<http://www.catb.org/~esr/halloween/halloween2.html>:

An important attribute to note which has led to volume drivers is
the ease with which you can write drivers for linux, and the
relatively powerful debugging infrastructure that linux has.
Finding and installing the DDK, and trying to hook up the kernel
debugger and do any sort of interaction with user-mode without
tearing the NT system to bits is much more challenging than
writing the simple device-drivers for linux. Any idiot could write
a driver in 2 days with a book like "Linux Device Drivers" — there
is no such thing as a 2-day device-driver for NT
Dan Cross
2024-01-02 19:47:15 UTC
Permalink
A significant amount of work is done maintaining drivers, but inside
MSFT and for OEMs. Linux has become the most important OS in the world,
and writing drivers for Windows is notoriously tedious.
In the words of the infamous “Halloween documents”
An important attribute to note which has led to volume drivers is
the ease with which you can write drivers for linux, and the
relatively powerful debugging infrastructure that linux has.
Finding and installing the DDK, and trying to hook up the kernel
debugger and do any sort of interaction with user-mode without
tearing the NT system to bits is much more challenging than
writing the simple device-drivers for linux. Any idiot could write
a driver in 2 days with a book like "Linux Device Drivers" — there
is no such thing as a 2-day device-driver for NT
Yeah, NatBro's comment here is a bit dated. WinDbg is actually
amazing and trounces anything in the Linux/GNU world. The only
thing comparable might be `mdb` in the Solaris/illumos world.

- Dan C.
Dave Froble
2024-01-01 17:48:39 UTC
Permalink
Post by Arne Vajhøj
Post by Lawrence D'Oliveiro
Post by Arne Vajhøj
Post by Lawrence D'Oliveiro
Post by Arne Vajhøj
Along with those 8-bit-era single-letter drive names, for some reason.
Microsoft: “26 drive letters ought to be enough for anybody!”
You can mount disks without using drive letters with NTFS.
As I recall from reading docs, that doesn’t work with network shares or
hot-pluggable storage. Or this thing called “storage spaces”.
You don't need it for a network share - those can just be accessed
directly via UNC.
So how do you select from available UNC shares in an application’s file
picker?
Post by Arne Vajhøj
(it is said that one can make a symlink to UNC if one want to)
Windows symlinks have their own share of problems.
* need more than 26 drives
* need to use a file picker to choose network drives
* don't like to create symlinks
then one has a problem.
But my guess is that you can invite everyone in the world matching
that criteria to coffee in your dining room.
Even if your dining room is a phone booth ...

Does such even exist anymore?
--
David Froble Tel: 724-529-0450
Dave Froble Enterprises, Inc. E-Mail: ***@tsoft-inc.com
DFE Ultralights, Inc.
170 Grimplin Road
Vanderbilt, PA 15486
Single Stage to Orbit
2024-01-02 11:11:37 UTC
Permalink
Even  if your dining room is a phone booth ...
Well, if a Police box worked well for a Time Lord ...
--
Tactical Nuclear Kittens
Single Stage to Orbit
2024-01-02 11:09:06 UTC
Permalink
Post by Lawrence D'Oliveiro
So how do you select from available UNC shares in an application’s
file picker?
We don't let the users do that, we know it's too easy to mess up.
--
Tactical Nuclear Kittens
Simon Clubley
2024-01-02 13:41:09 UTC
Permalink
And remember, it?s NTFS-specific. Windows lacks the equivalent of a fully
general VFS layer, for some totally unfathomable reason.
So does VMS. Unfortunately.

Simon.
--
Simon Clubley, ***@remove_me.eisner.decus.org-Earth.UFP
Walking destinations on a map are further away than they appear.
Single Stage to Orbit
2024-01-02 11:08:16 UTC
Permalink
Post by Arne Vajhøj
You can mount disks without using drive letters with NTFS.
It is just that nobody use it.
Before I was made redundant last month I used UNC mounts a lot with the
ex-employer's business application to share data between workstations.
--
Tactical Nuclear Kittens
Lawrence D'Oliveiro
2023-12-31 21:29:35 UTC
Permalink
Post by John Dallman
You might want to explain why StreamCR is provided: I doubt
it is because of traditional Mac format, because VMS predates Mac by
several years.
Support for STREAM_xx formats in RMS were not added until VMS 4.0, near as
I can tell. And that dates from around the time of the release of the Mac.
Arne Vajhøj
2023-12-31 22:11:45 UTC
Permalink
Post by Lawrence D'Oliveiro
Post by John Dallman
You might want to explain why StreamCR is provided: I doubt
it is because of traditional Mac format, because VMS predates Mac by
several years.
Support for STREAM_xx formats in RMS were not added until VMS 4.0, near as
I can tell. And that dates from around the time of the release of the Mac.
That surprises me. I always thought they had been there since day 1.

But I started with VMS 4.4, so I don't know early VMS first hand.

And the RMS manual for VMS 2.0:

https://bitsavers.org/pdf/dec/vax/vms/2.0/AA-D031C-TE_VAX-11_2.0_Record_Management_Services_Reference_Manual_198003.pdf

does not list the stream formats, so it indeed look like a
later addition.

But that is sort of the benefit from RMS. It is possible to
add an entirely new record format and everything calling
SYS$GET directly or indirectly via language IO will not
need any change.

Arne
Lawrence D'Oliveiro
2023-12-31 23:13:27 UTC
Permalink
Post by Arne Vajhøj
Post by Lawrence D'Oliveiro
Support for STREAM_xx formats in RMS were not added until VMS 4.0, near
as I can tell. And that dates from around the time of the release of
the Mac.
That surprises me. I always thought they had been there since day 1.
But I started with VMS 4.4, so I don't know early VMS first hand.
I first got my hands on a VMS system in 1979. I was a first-year Comp Sci
student, and first-year students were not supposed to have access to time-
sharing accounts. But the difference between that and RSTS/E (which I also
was not supposed to have access to) was like night and day.

I’m not sure what version of VMS that was. It was likely 2.2, or possibly
earlier. There was an interesting privilege-escalation security hole in
2.2’s handling of message sections, which became quite clear if you read
the internals manual ... which I only got my hands on after we had
upgraded to 3.0.

I can remember quite a few other milestones off the top of my head. Like
some compatibility issues with the third-party serial drivers in VMS 2.x,
which went away in 3.0 because the terminal driver was split into a
(hardware-independent) “class” driver and a (hardware-specific) “port”
driver. This meant that third-party hardware vendors only needed to supply
their own “port” driver, and they automatically got all the standard QIO
and control-character support, just like DEC’s own terminal drivers.

VMS 4.0 was also the version that increased the length of file names from
9-dot-3 to 39-dot-39.
Arne Vajhøj
2024-01-01 00:39:09 UTC
Permalink
Post by Lawrence D'Oliveiro
Post by Arne Vajhøj
Post by Lawrence D'Oliveiro
Support for STREAM_xx formats in RMS were not added until VMS 4.0, near
as I can tell. And that dates from around the time of the release of
the Mac.
That surprises me. I always thought they had been there since day 1.
But I started with VMS 4.4, so I don't know early VMS first hand.
VMS 4.0 was also the version that increased the length of file names from
9-dot-3 to 39-dot-39.
That I did know.

Arne
Scott Dorsey
2023-12-31 22:18:06 UTC
Permalink
Post by Lawrence D'Oliveiro
Post by John Dallman
You might want to explain why StreamCR is provided: I doubt
it is because of traditional Mac format, because VMS predates Mac by
several years.
Support for STREAM_xx formats in RMS were not added until VMS 4.0, near as
I can tell. And that dates from around the time of the release of the Mac.
Most operating systems out there only used stream_lf and stream_cr. Whether
lf or crlf was used depended a lot on the whims of the original developers.
Most terminals had a switch that allowed you to toggle them.
--scott
--
"C'est un Nagra. C'est suisse, et tres, tres precis."
Lawrence D'Oliveiro
2023-12-31 23:15:51 UTC
Permalink
Post by Scott Dorsey
Most operating systems out there only used stream_lf and stream_cr.
Actually I think the timing of addition of those STREAM_xx formats was a
little too early to be credited/blamed on the Mac. STREAM_CRLF would have
been because of the massive popularity of MS-DOS and the IBM PC by that
time, and STREAM_LF was similarly a sign of feeling the pressure from
Unix. STREAM_CR ... maybe that was just stuck in for completeness. I
really don’t know of any other OSes that might have used it, apart from
old MacOS.
Dan Cross
2023-12-31 23:07:49 UTC
Permalink
Post by John Dallman
Post by Arne Vajhøj
https://www.vajhoej.dk/arne/articles/rms.html
Good work. You might want to explain why StreamCR is provided: I doubt it
is because of traditional Mac format, because VMS predates Mac by several
years. Was it used by older DEC OSes?
Perhaps for direct terminal input. By default, DEC terminals
like the VT100 only send a CR when the RETURN key is pressed.
(They can be configured to send CRLF via an option on the setup
screen, however.) Anyway, I'm speculating, but I can imagine a
mode in which data is copied directly from a terminal to a file
preserving the single CR character.

- Dan C.
mjos_examine
2024-01-02 02:15:23 UTC
Permalink
Post by John Dallman
Good work. You might want to explain why StreamCR is provided: I doubt it
is because of traditional Mac format, because VMS predates Mac by several
years. Was it used by older DEC OSes?
According to a couple of search hits, the Apple II used CR line ending
before the Mac did.

You could save space on those 360K 5¼ floppy disks by eliminating the LF
on every line in a text file. ;-)

Here's one reference:

"... and Classic MacOS inherited the CR line ending convention from
ProDOS on the Apple II (and presumably SOS on the Apple ///, which was
the genesis of ProDOS.)"

https://stardot.org.uk/forums/viewtopic.php?p=381225&sid=5b7d6dac7407c615aaad1c711fa3fda9#p381225

That would be somewhere after 1980 (Apple /// SOS).
Simon Clubley
2024-01-02 13:39:52 UTC
Permalink
Post by John Dallman
Post by Arne Vajhøj
https://www.vajhoej.dk/arne/articles/rms.html
Good work. You might want to explain why StreamCR is provided: I doubt it
is because of traditional Mac format, because VMS predates Mac by several
years. Was it used by older DEC OSes?
John
Second-system effect ? :-)

Simon.
--
Simon Clubley, ***@remove_me.eisner.decus.org-Earth.UFP
Walking destinations on a map are further away than they appear.
Simon Clubley
2024-01-02 13:39:03 UTC
Permalink
Post by Arne Vajhøj
* what everybody knows about RMS
* relative file and direct access fixed length file examples
I recently posted
* the index-sequential file examples I have used numerous times before
* some file read and file creation test code I did a few years ago
https://www.vajhoej.dk/arne/articles/rms.html
Some feedback:

About your VFC comments: what does Fortran carriage control use on VMS ?
You say there is no use for VFC, but then you go on to say Fortran uses
VFC for carriage control.

You may wish to make explicit what the maximum record size is with RMS.

Do you want to discuss the history of ISAM files with their different
prolog versions ?

You may wish to make it clear that RMS is a part of VMS in that it is
within VMS itself and not just some user-mode library linked in to each
user program.

You may wish to make the byte ordering explicit by giving offsets to
each field. IOW, make it clear that the control data is at the start
of each record and that fields are in little-endian and not big-endian
format. This matters because different operating systems have both
different documentation and endian conventions.

Simon.
--
Simon Clubley, ***@remove_me.eisner.decus.org-Earth.UFP
Walking destinations on a map are further away than they appear.
Arne Vajhøj
2024-01-02 14:12:11 UTC
Permalink
Post by Simon Clubley
Post by Arne Vajhøj
* what everybody knows about RMS
* relative file and direct access fixed length file examples
I recently posted
* the index-sequential file examples I have used numerous times before
* some file read and file creation test code I did a few years ago
https://www.vajhoej.dk/arne/articles/rms.html
Thanks. Apperciated.
Post by Simon Clubley
About your VFC comments: what does Fortran carriage control use on VMS ?
You say there is no use for VFC, but then you go on to say Fortran uses
VFC for carriage control.
DCL use record format VFC with carriage control PRN .

Fortran default use record format VAR with carriage control FTN.

I hope that is what I communicate:

<quote>
Nobody seems to see a purpose with VFC record format. But DCL create
files in VFC record format, so it is used.
...
Fortran carriage control use the first column in file for control. A
space ' ' in first column means ordinary new line. A one '1' in first
column means new page. This is a very old Fortran convention. And
usually it is only files generated by Fortran programs that use Fortran
carriage control, but VMS and RMS support sit.

Print carriage control use the content of the control information for
VFC files to manage output.
</quote>

(but I just noted that I can't count to 3)
Post by Simon Clubley
You may wish to make explicit what the maximum record size is with RMS.
Yes. That is important. It may not have been a severe limitation
in 1977, but it sure is today.

And I would need to mention Fortran segmented files.
Post by Simon Clubley
Do you want to discuss the history of ISAM files with their different
prolog versions ?
I want to but I can't.

:-)

I don't have the knowledge.
Post by Simon Clubley
You may wish to make it clear that RMS is a part of VMS in that it is
within VMS itself and not just some user-mode library linked in to each
user program.
I should probably mention EXEC mode
Post by Simon Clubley
You may wish to make the byte ordering explicit by giving offsets to
each field. IOW, make it clear that the control data is at the start
of each record and that fields are in little-endian and not big-endian
format. This matters because different operating systems have both
different documentation and endian conventions.
Yes.

Not as big a problem today as 25 years ago. But still relevant.

Arne
Arne Vajhøj
2024-01-03 01:32:54 UTC
Permalink
Post by Arne Vajhøj
Post by Simon Clubley
Post by Arne Vajhøj
* what everybody knows about RMS
* relative file and direct access fixed length file examples
    I recently posted
* the index-sequential file examples I have used numerous times before
* some file read and file creation test code I did a few years ago
https://www.vajhoej.dk/arne/articles/rms.html
Thanks. Apperciated.
Post by Simon Clubley
You may wish to make explicit what the maximum record size is with RMS.
Yes. That is important. It may not have been a severe limitation
in 1977, but it sure is today.
And I would need to mention Fortran segmented files.
Post by Simon Clubley
You may wish to make it clear that RMS is a part of VMS in that it is
within VMS itself and not just some user-mode library linked in to each
user program.
I should probably mention EXEC mode
Post by Simon Clubley
You may wish to make the byte ordering explicit by giving offsets to
each field. IOW, make it clear that the control data is at the start
of each record and that fields are in little-endian and not big-endian
format. This matters because different operating systems have both
different documentation and endian conventions.
Yes.
Not as big a problem today as 25 years ago. But still relevant.
Updated:
* add Groovy examples
* add examples using byte offset seek to mimic access by
record number to fixed length records
* add a lot about record size
* add note about RMS being exec mode and lengths being little endian

Still:

https://www.vajhoej.dk/arne/articles/rms.html

Arne
Hans Bachner
2024-01-03 17:21:01 UTC
Permalink
Post by Arne Vajhøj
[...]
<quote>
Nobody seems to see a purpose with VFC record format. But DCL create
files in VFC record format, so it is used.
...
 Fortran carriage control use the first column in file for control. A
space ' ' in first column means ordinary new line. A one '1' in first
column means new page. This is a very old Fortran convention. And
usually it is only files generated by Fortran programs that use Fortran
carriage control, but VMS and RMS support sit.
[...]
In addition, '+' means no line feed (overprint) and '0' means two new lines.

(from my Fortran programming days back in the early 80s...)

Hans.
Arne Vajhøj
2024-01-03 21:10:37 UTC
Permalink
Post by Hans Bachner
Post by Arne Vajhøj
<quote>
Nobody seems to see a purpose with VFC record format. But DCL create
files in VFC record format, so it is used.
...
  Fortran carriage control use the first column in file for control. A
space ' ' in first column means ordinary new line. A one '1' in first
column means new page. This is a very old Fortran convention. And
usually it is only files generated by Fortran programs that use
Fortran carriage control, but VMS and RMS support sit.
[...]
In addition, '+' means no line feed (overprint) and '0' means two new lines.
(from my Fortran programming days back in the early 80s...)
Yes. I guess I was lazy and just noted those that I use.

+ and 0 still works fine.

$ type z.txt
AAA
+B
0CC
$ set file/attr=rat=ftn z.txt
$ type z.txt
BAA

CC

I will update.

Arne
Lawrence D'Oliveiro
2024-01-02 19:31:05 UTC
Permalink
Post by Simon Clubley
You may wish to make it clear that RMS is a part of VMS in that it is
within VMS itself and not just some user-mode library linked in to each
user program.
The symbols have to come from some file that is input to the linker
(.OLB, .OBJ, .STB). All the system entry points (RMS and the kernel
proper) reside at absolute locations; this used to be at 80000168 hex
(note: system space), then later (in either VMS V3 or V4) that was moved
to 7FFEDF68 (note: P1 space). Not sure why: to allow per-process
interception of system calls? Of course the old location continued to be
valid forever for existing services, but all new services added thereafter
were only available at the new place.

RMS was one of those things that was quite difficult to explain to new
users. I was the micro support guy, whom people came to if they had
trouble moving files between these newfangled PC things and the VAX
machines. And of course VMS software would typically refuse to work
properly with files unless they had the right record format attributes
etc. And files transferred from other systems tended to end up as “fixed-
length 512-byte records”, which was just about the least useful format
imaginable. So fixing the RMS attributes was a very common need. Assuming
the users could understand the right settings to use.

Linus “Mr Linux” Torvalds did apparently use VMS for a while, and he hated
it. The reason had to do with RMS.
Loading...