Discussion:
What is your favorite tool lo look at open batch log files? I use a vested ICR LIST from 1988 (binary translated from VAX to Alpha)
Add Reply
Jon Pinkley
2020-11-08 07:04:50 UTC
Reply
Permalink
William P. Wood, Jr. submitted LIST on the VAX82B tape [VAX82B.ICR.LIST]

It is written in RATFIV and VAX MACRO (1982 VAX Macro before Alpha)

In 1988, I resubmitted LIST with two tiny changes, one performance related (return to pre-VMS 4.4 locking behavior) and one to support 39 character file extensions. This was in the [VAX88A2.JLP.LIST] directory, and it included a backup saveset that had the original ICR VAX82B submission including RATFIV.

When I started using the Alpha in 1995, I just vested the VAX LIST.EXE from 1988 and that's what I've been using since.

----

Why I like LIST. It's a combination of a pager like more/less, with the ability to search forward and backward, jump to specified line numbers (or $ for end of file), and extract with more general pattern matching than VMS SEARCH (but not regex), it caches RFAs (I think every 100th record), so once you have read the file, it is quite quick to reposition to any line number in the file.

The main thing I like about it is that it can read open batch log files, unlike an editor. And even if the file isn't currently open, if there is a runaway batch job, with a GB log file, it has no problem, as it doesn't try to read the whole file into virtual memory (like TPU).

Even the VESTed image is pretty efficient.

So the question is, what tools do other people use to look at log files that are opened by long running batch jobs?
Jan-Erik Söderholm
2020-11-08 07:32:24 UTC
Reply
Permalink
Post by Jon Pinkley
William P. Wood, Jr. submitted LIST on the VAX82B tape [VAX82B.ICR.LIST]
It is written in RATFIV and VAX MACRO (1982 VAX Macro before Alpha)
In 1988, I resubmitted LIST with two tiny changes, one performance related (return to pre-VMS 4.4 locking behavior) and one to support 39 character file extensions. This was in the [VAX88A2.JLP.LIST] directory, and it included a backup saveset that had the original ICR VAX82B submission including RATFIV.
When I started using the Alpha in 1995, I just vested the VAX LIST.EXE from 1988 and that's what I've been using since.
----
Why I like LIST. It's a combination of a pager like more/less, with the ability to search forward and backward, jump to specified line numbers (or $ for end of file), and extract with more general pattern matching than VMS SEARCH (but not regex), it caches RFAs (I think every 100th record), so once you have read the file, it is quite quick to reposition to any line number in the file.
The main thing I like about it is that it can read open batch log files, unlike an editor. And even if the file isn't currently open, if there is a runaway batch job, with a GB log file, it has no problem, as it doesn't try to read the whole file into virtual memory (like TPU).
Even the VESTed image is pretty efficient.
So the question is, what tools do other people use to look at log files that are opened by long running batch jobs?
TYPE. With /TAIL, if that works.
Jon Pinkley
2020-11-08 11:58:46 UTC
Reply
Permalink
Post by Jan-Erik Söderholm
TYPE. With /TAIL, if that works.
TYPE/PAGE works well for small log files, and TAIL is good to see the end of the file, but neither of those allow you to search within the file if you are looking for a specific string. And if you are debugging a problem you often need to be able to search forward and backward.

For example you may have a runaway job caused by a failure that is then causing your job to loop, and tail can quickly show you that. But it isn't very good in letting you know when it started.

All this assumes verification is turned on, or there are useful messages being written to sys$output by the batch job.

I had a co-worker that would make a copy of the log file with convert/share and then use an editor to view/debug, which is ok for small log files, but doesn't scale well with large log files.

I am not sure when I started using LIST, probably shortly after it was on the DECUS tapes in 1982, and once I learned the syntax and commands, which at the time was quite foreign to me, I was hooked, and never found anything else that made me want to switch. (Like Teco to Andy Goldstein)

There's an SMG based file viewer (LOOK) by Serge Kovalyovon and enhancements by Patrick Ellis on Hunter Goatley's FILESERV and mirrors, and it is quite good. It is more like using an editor, with single key commands for navigation. It is definitely easier to learn than list. And much better than type/page

I find myself using it more frequently for simple things, and it is more likely to be accepted by new users than LIST is. (EVE vs Teco, nano vs vim)

http://vms.process.com/ftp/vms-freeware/fileserv/look.zip

But I still prefer LIST for most things, because I have learned how to use it. On OpenVMS I still prefer EDT over EVE, so that shows how people tend to like what they first learned. List can also be used like search to extract records, or a column range from a file, although in general, gawk or perl would probably be better choices at this time. One VMS specific thing that LIST can do is open an indexed file and read in index key order, not something I use often, but I have used it.

Here's a link to the ICR LIST documentation:
https://www.digiater.nl/openvms/decus/vax82b/icr/list/list.doc (runoff text output, not word document)
https://www.digiater.nl/openvms/decus/vax82b/icr/list/list.rno runoff source
Phillip Helbig (undress to reply)
2020-11-08 14:40:12 UTC
Reply
Permalink
On Sunday, November 8, 2020 at 2:32:26 AM UTC-5, Jan-Erik S=C3=B6derholm wr=
Post by Jan-Erik Söderholm
TYPE. With /TAIL, if that works.
TYPE/PAGE works well for small log files, and TAIL is good to see the end
of the file, but neither of those allow you to search within the file if you
are looking for a specific string.
TYPE/PAGE=SAVE[/TAIL] allows one to search.
Phillip Helbig (undress to reply)
2020-11-08 07:52:10 UTC
Reply
Permalink
Post by Jon Pinkley
The main thing I like about it is that it can read open batch log files,
unlike an editor.
EDIT/TPU/READ will allow one to view open batch log files...
Post by Jon Pinkley
And even if the file isn't currently open, if there is a runaway batch job,
with a GB log file, it has no problem, as it doesn't try to read the whole
file into virtual memory (like TPU).
...but does read big files into memory before displaying anything (one
thing I like about EDT, though usually not an issue with batch log
files).
Post by Jon Pinkley
So the question is, what tools do other people use to look at log files
that are opened by long running batch jobs?
BACKUP/IGNORE=INTERLOCK <log-file> <temp-file> then use whatever on
<temp-file>.
Jon Pinkley
2020-11-08 09:39:19 UTC
Reply
Permalink
Post by Phillip Helbig (undress to reply)
Post by Jon Pinkley
The main thing I like about it is that it can read open batch log files,
unlike an editor.
EDIT/TPU/READ will allow one to view open batch log files...
Phillip, thanks for pointing that out, I thought I tried using EDIT/TPU/READ, but I just verified that EDIT/TPU/READ does work, but EDIT/EDT/READ does not.

$ edt/read SYS$SYSDEVICE:[TCPIP$FTP]TCPIP$FTP_RUN.LOG;1008
%EDT-F-OPENIN, error opening SYS$SYSDEVICE:[TCPIP$FTP]TCPIP$FTP_RUN.LOG;1008 as input
-RMS-E-FLK, file currently locked by another user
$

My guess is that I don't like EVE for large files, and when I tried EDIT/EDT/READ, I just assumed that EDIT/TPU/READ would be the same.
Jan-Erik Söderholm
2020-11-08 11:24:04 UTC
Reply
Permalink
Post by Jon Pinkley
Post by Phillip Helbig (undress to reply)
Post by Jon Pinkley
The main thing I like about it is that it can read open batch log files,
unlike an editor.
EDIT/TPU/READ will allow one to view open batch log files...
Phillip, thanks for pointing that out, I thought I tried using EDIT/TPU/READ, but I just verified that EDIT/TPU/READ does work, but EDIT/EDT/READ does not.
$ edt/read SYS$SYSDEVICE:[TCPIP$FTP]TCPIP$FTP_RUN.LOG;1008
%EDT-F-OPENIN, error opening SYS$SYSDEVICE:[TCPIP$FTP]TCPIP$FTP_RUN.LOG;1008 as input
-RMS-E-FLK, file currently locked by another user
$
My guess is that I don't like EVE for large files, and when I tried EDIT/EDT/READ, I just assumed that EDIT/TPU/READ would be the same.
Or, to get a constantly updated display:

TYPE/CONT/INT=1 <batch-logfile>

Together with $ SET OUTPUT_RATE=00:00:01 in the batch job.
Phillip Helbig (undress to reply)
2020-11-08 11:39:50 UTC
Reply
Permalink
Post by Jan-Erik Söderholm
TYPE/CONT/INT=1 <batch-logfile>
Together with $ SET OUTPUT_RATE=00:00:01 in the batch job.
That is often useful, though TYPE/TAIL sometimes doesn't work (I'm not
sure exactly when). Before TYPE/TAIL was available, I used the
following C code which I got from somewhere (probably comp.os.vms):

#include descrip /* descriptor details */
#include stdio /* for printf and stuff */
#include rms /* for the "real" I/O stuff */
#include dvidef /* stuff for the GETDVI ss */
#include dcdef /* device classes */
#include iodef /* QIO stuff */
#include msgdef /* mailbox stuff */
#include ssdef /* completion codes */

struct FAB inp_fab; /* Input fab/rab/xab */
struct RAB inp_rab;
struct XABFHC inp_xab;

struct FAB out_fab; /* Output fab/rab */
struct RAB out_rab;

typedef struct { /* define an item list struct */
short length; /* length of buffer */
short code; /* item code */
void *ptr; /* ptr to buffer */
void *retlen; /* ptr to return length */
} Item;

typedef struct { /* Genernic IOSB struct */
short status;
short length;
long devdepend;
} Iosb;

typedef struct {
unsigned long block; /* block # */
unsigned short offset; /* offset within block */
unsigned short length; /* record length */
} Rfa;

typedef struct { /* VMS 64 bit quadword time */
long lsb;
long msb;
} VMSTime;

unsigned char inp_buf[65536-512]; /* 127 blocks of buffer space */
int the_safe_way=0; /* If != then read var files front to back */
int monitor; /* if != then loop on display of tail */
int sec=5; /* delay time in seconds */
int tti_chan; /* channel to use to look for stop code */
char tti_text[8]; /* room for terminal input */
Iosb tti_iosb; /* IOSB for terminal input */
int tti_class; /* SYS$INPUT device class */
int tto_class; /* SYS$OUTPUT device class */
int tto_page; /* SYS$OUTPUT page size (if tto_class == DC$_TERM) */
int were_done; /* if !=, signals monitor complete */
int last_rfa_blk; /* saved rfa of last record read */
int last_rfa_off;

char default_string[] = "SYS$DISK:[].LOG"; /* default input filename */

Item tt_dvi[] = { /* An item list used to get SYS$xxx class */
{4,DVI$_DEVCLASS,&tto_class,0},
{4,DVI$_TT_PAGE,&tto_page,0},
{0,0,0,0}
};

$DESCRIPTOR(sysin,"SYS$INPUT");
$DESCRIPTOR(sysout,"SYS$OUTPUT");

/* rfa stands for "record file address" */

Rfa *rfas; /* ptr to array of rfa structs */
int record_count=23; /* number of records to output */
int maxargs; /* records size of next_file array */
char **next_file; /* pts to array of char ptrs to filenames */
VMSTime delay = {-5*10*1000*1000, /* 64 bit VMS delta time format for monitor timer... */
-1}; /* ...initialized to 5 seconds */

char *mini_help_msg[] = {
"TAIL version 2.4, 09/10/91. D. Shepperd, ***@dms.UUCP\n",
"Usage: TAIL [/record_count] [/S] [/F] [/T secs] input_file [output_file]\n",
0
};

char *help_msg[] = {
"where: \"[]\" indicates optional data\n",
" \"/record_count\" is decimal number of records desired\n",
" \"/S\" indicates to use the \"safer\" mode\n",
" \"/F\" monitor tail end of file (5 second sample rate)\n",
" \"/T secs\" monitor tail end of file with sample rate of \"secs\"\n",
" \"input_file\" is the input filename (can have wildcards)\n",
" \"output_file\" is output filename (bogus if wildcards on input)\n",
"Note that a \"-\" can be used in place of the \"/\" to delimit options.\n",
"White space is required between all arguments (including the /T and secs).\n",
"Options may appear in any order, but all must preceed filename(s).\n",
0
};


void mini_help()
{
char **s;
for (s=mini_help_msg;*s;++s) fputs(*s,stderr); /* show mini help */
return;
}

void show_help()
{
char **s;
mini_help();
for (s = help_msg;*s;++s) fputs(*s,stderr); /* display help message */
return;
}

int main(int argc,char *argv[])
{
int param; /* Parameter counter */
int err; /* place to hold rms errors */
int rfm; /* loaded with record format code */
int i, files;

err = sys$getdviw(0,0,&sysin,&tt_dvi,0,0,0,0);
if ((err&1) == 0) return err;
tti_class = tto_class;

err = sys$getdviw(0,0,&sysout,&tt_dvi,0,0,0,0);
if ((err&1) == 0) return err;
if (tto_class == DC$_TERM) record_count = tto_page > 2 ? tto_page-1 : 2;

inp_fab = cc$rms_fab; /* init the input fab/rab/xab */
inp_fab.fab$b_shr = FAB$M_GET|FAB$M_PUT|FAB$M_UPI;
inp_fab.fab$b_fac = FAB$M_GET|FAB$M_BRO;
inp_rab = cc$rms_rab;
inp_rab.rab$l_ubf = inp_buf;
inp_rab.rab$w_usz = sizeof(inp_buf); /* assume to read the max */
inp_xab = cc$rms_xabfhc;
inp_rab.rab$l_fab = &inp_fab; /* tell rab where fab is */
inp_fab.fab$l_xab = &inp_xab; /* tell fab where xab is */
inp_fab.fab$l_dna = default_string; /* say input file defaults */
inp_fab.fab$b_dns = sizeof(default_string)-1;

param = 1; /* start looking at argv[1] */
--argc; /* skip the image name */
while(1) { /* get all the command options */
char c,*s;
if (argc < 1) break;
s = argv[param];
c = *s++;
if (c == '/' || c == '-') {
c = *s++;
c = toupper(c);
switch (c) {
int secs;
case '?':
case 'H':
show_help();
return 0x10000003;
case 'S': /* safer mode */
the_safe_way = 1;
break;
case 'T': /* set monitor rate in seconds */
if (*s == 0) {
if (--argc < 1) {
fputs("Missing delay time parameter\n",stderr);
mini_help();
return 0x10000002;
}
s = argv[++param];
}
if (sscanf(s,"%d",&secs) != 1 || secs <= 0 || secs > 2000) {
fprintf(stderr,"Invalid delay time parameter: %s\n",s);
fputs("Time value must be between 1 and 2000 secs\n",stderr);
mini_help();
return 0x10000002;
}
delay.lsb = -secs*10*1000*1000;
/* fall through to /F to default to monitor mode if /T specified */
case 'F': /* set monitor mode */
monitor = 1;
break;
default: /* assume the param is a record count */
--s; /* backup to the first char of the record count */
if (sscanf(s,"%d",&record_count) != 1 || record_count <= 0) {
fprintf(stderr,"Invalid record count parameter: %s\n",s);
mini_help();
return 0x10000002;
}
}
++param;
--argc;
} else {
break;
}
}
record_count++; /* Fix of wrong record_count. KAR - 6-mar-1990 */
if (argc < 1) {
show_help();
exit(0x10000003);
}
files = fgen(argv[param], &next_file, &maxargs); /* deal with wildcards */
if (files < 1) {
fputs("No input file(s) found\n",stderr);
return 0x10000002;
}
if (monitor) {
if (tti_class != DC$_TERM) {
fputs("Cannot monitor files if SYS$INPUT is not a terminal\n",stderr);
monitor = 0;
} else {
err = sys$assign(&sysin,&tti_chan,0,0);
if ((err&1) == 0) {
fputs("Error assigning channel to SYS$INPUT, monitor mode disabled\n",stderr);
monitor = 0;
}
que_ttiread(); /* que up a read to the terminal */
}
}
for (i = 0; i < files; i++) {
if (files > 1) {
if (tto_class == DC$_TERM) {
printf("\r\n \033[7m**************** %s ****************\033[0m",
next_file[i]);
} else {
printf("\r\n **************** %s ****************",
next_file[i]);
}
}
inp_fab.fab$l_fna = next_file[i]; /* input filename is next param */
inp_fab.fab$b_fns = strlen(next_file[i]);
if (((err=sys$open(&inp_fab))&1) == 0) {
fputs("Error opening input file\n",stderr);
exit(err);
}
if (((err=sys$connect(&inp_rab))&1) == 0) {
fputs("Error connecting input rab\n",stderr);
exit(err);
}
if (inp_fab.fab$b_org != FAB$C_SEQ) {
char *oldtype="UNKNOWN";
if (inp_fab.fab$b_org == FAB$C_REL) oldtype = "RELATIVE";
if (inp_fab.fab$b_org == FAB$C_IDX) oldtype = "INDEXED";
if (inp_fab.fab$b_org == FAB$C_HSH) oldtype = "HASHED";
fprintf(stderr,"Input file organization is %s. This program only supports SEQUENTIAL.\n",
oldtype);
exit(0x10000002);
}
if (inp_fab.fab$b_rfm == FAB$C_UDF || inp_fab.fab$b_rfm > FAB$C_STMCR) {
fputs("Input file has undefined or unsupported record format.\n",stderr);
exit(0x10000002);
}
out_fab = cc$rms_fab; /* init the output fab */
out_fab.fab$b_bks = inp_fab.fab$b_bks; /* make rest same as input */
out_fab.fab$w_bls = inp_fab.fab$w_bls;
out_fab.fab$w_deq = inp_fab.fab$w_deq;
out_fab.fab$b_fac = FAB$M_PUT;
out_fab.fab$b_fsz = inp_fab.fab$b_fsz;
out_fab.fab$w_mrs = inp_fab.fab$w_mrs;
out_fab.fab$b_rat = inp_fab.fab$b_rat;
out_fab.fab$b_rfm = inp_fab.fab$b_rfm;
out_rab = cc$rms_rab;
out_rab.rab$l_fab = &out_fab; /* tell rab where fab is */
if (--argc >= 1) {
out_fab.fab$l_fna = argv[++param]; /* output filename is next param */
} else {
out_fab.fab$l_fna = "SYS$OUTPUT:"; /* else default */
}
out_fab.fab$b_fns = strlen(out_fab.fab$l_fna);
if (((err=sys$create(&out_fab))&1) == 0) {
fputs("Error opening output file\n",stderr);
exit(err);
}
if (((err=sys$connect(&out_rab))&1) == 0) {
fputs("Error connecting output rab\n",stderr);
exit(err);
}
rfm = inp_fab.fab$b_rfm; /* pickup rfm code */
last_rfa_blk = last_rfa_off = 0; /* start at top of file */
were_done = 0;
while (1) {
long oldebk;
int oldffb;
oldebk = inp_xab.xab$l_ebk; /* remember the old end of file mark */
oldffb = inp_xab.xab$w_ffb;
if (last_rfa_blk != 0) { /* if we've already been through the file */
err = skip_through(); /* then just start where we left off */
} else {
switch (inp_fab.fab$b_rfm) {
case FAB$C_FIX:
err = do_fixed();
break;
case FAB$C_VAR:
err = do_var();
break;
case FAB$C_VFC:
err = do_var();
break;
case FAB$C_STM:
err = do_stream(2);
break;
case FAB$C_STMCR:
err = do_stream(1);
break;
case FAB$C_STMLF:
err = do_stream(0);
break;
default:
err = 0x10000004;
fputs("Unknown record format\n",stderr);
exit(err);
}
}
if ((err&1) == 0) {
if (err != RMS$_EOF) {
fputs("Error reading input\n",stderr);
exit(err);
}
}
if (!monitor || were_done == 1) break;
while (!were_done) {
sys$schdwk(0,0,&delay,0);
sys$hiber();
err = sys$display(&inp_fab); /* see if stuff has been added to file */
if ((err&1) == 0) {
fputs("Error doing $DISPLAY on input, monitor mode cancelled\n",stderr);
were_done = 1;
break;
}
if (inp_xab.xab$l_ebk == oldebk && inp_xab.xab$w_ffb == oldffb) {
continue; /* eof hasn't moved, continue waiting */
}
/* This part is goofy. One would think that a simple sys$display would be all that */
/* should be required, but nooooooo.... DEC has set some internal flags that will */
/* not let me read past the old end of file regardless of the fact the the eof has */
/* moved. What's really goofy is that sys$display notices that the eof has moved */
/* but it won't change those internal flags. Closing/reopening the file every few */
/* seconds seems like an expensive proposition to me. Grrrr. */
err = sys$close(&inp_fab); /* close the file */
if ((err&1) == 0) {
fputs("Error closing the input file\n",stderr);
exit(err);
}
err = sys$open(&inp_fab); /* reopen the file */
if ((err&1) == 0) {
fputs("Error reopening the input file\n",stderr);
exit(err);
}
err = sys$connect(&inp_rab); /* reconnect the rab */
if ((err&1) == 0) {
fputs("Error reconnecting the input rab\n",stderr);
exit(err);
}
break;
}
}
sys$close(&inp_fab);
sys$close(&out_fab);
}
if ((err&1) != 0) return err;
if (err != RMS$_EOF) return err;
return SS$_NORMAL;
}

/**************************************************************************
* do_seqout - sequentially output the data from the input file
*
* At entry:
* rfa_blk - starting block number
* rfa_off - offset within block to start of record
* At exit:
* last_rfa_blk and last_rfa_off are set to the rfa of the last record read
* input file has been dumped to output
**************************************************************************/

do_seqout(long rfa_blk,int rfa_off) /* seek to desired record and output */
{
int err,skip=0;
if (rfa_blk == 0) rfa_blk = 1; /* start at the beginning */
if (rfa_blk < last_rfa_blk || (rfa_blk == last_rfa_blk && rfa_off <= last_rfa_off)) {
rfa_blk = last_rfa_blk; /* seek to record last displayed */
rfa_off = last_rfa_off;
skip = 1; /* and skip it */
}
inp_rab.rab$l_rfa0 = rfa_blk; /* starting block # */
inp_rab.rab$w_rfa4 = rfa_off; /* byte offset within block */
inp_rab.rab$b_rac = RAB$C_RFA; /* change to RFA access mode */
inp_rab.rab$l_bkt = 0; /* make sure bkt field is clear */
inp_rab.rab$l_rhb = inp_buf; /* init the ptrs */
inp_rab.rab$l_ubf = inp_buf + inp_fab.fab$b_fsz; /* in case making VFC file */
out_rab.rab$l_rhb = inp_buf;
out_rab.rab$l_rbf = inp_buf + inp_fab.fab$b_fsz;
while(1) {
last_rfa_blk = inp_rab.rab$l_rfa0; /* save rfa of last record read */
last_rfa_off = inp_rab.rab$w_rfa4;
err=sys$get(&inp_rab); /* read the record */
if ((err&1) == 0) {
if (err != RMS$_EOF) {
fputs("Error reading input\n",stderr);
exit(err);
}
break;
}
inp_rab.rab$b_rac = RAB$C_SEQ; /* switch back to sequential reads */
out_rab.rab$w_rsz = inp_rab.rab$w_rsz;
if (skip == 0) { /* if not to skip the record */
if (((err=sys$put(&out_rab))&1) == 0) { /* write it */
fputs("Error writing output\n",stderr);
exit(err);
}
}
skip = 0; /* at most, we skip 1 record */
}
return err;
}

char end_mark[] = "\n\r\n";

/**************************************************************************
* do_stream - figure out the record structure for one of the 3 types of
* stream files there are.
* At entry:
* type - record type. 0=stmlf, 1=stmcr, 2=stmcrlf
* At exit:
* has called do_seqout with the computed block and offset of the
* desired starting record.
**************************************************************************/

do_stream(type)
int type; /* 0=stmlf, 1=stmcr, 2=stmcrlf */
{
unsigned long block,rfa_blk=0;
int rcd_num= 0,err,rfa_off,part1=0,end_char;
unsigned char *s;

inp_rab.rab$l_bkt = inp_xab.xab$l_ebk+1;
end_char = end_mark[type];

while(1) { /* as long as there's data in the file */
if (inp_rab.rab$l_bkt <= 1) { /* quit if we already read blk 1 */
rfa_blk = 1; /* give 'em the whole file */
rfa_off = 0;
break;
}
block = inp_rab.rab$l_bkt-(inp_rab.rab$w_usz>>9);
if (block == 0 || block > inp_rab.rab$l_bkt) block = 1; /* but can't start before 1 */
inp_rab.rab$l_bkt = block; /* rememebr starting block number */
err = sys$read(&inp_rab);
if ((err&1) == 0) {
if (err == RMS$_EOF) break;
fprintf(stderr,"Error (%08X) trying to read %d bytes at block %d\n",
err,inp_rab.rab$w_usz,block);
continue;
}
s = inp_rab.rab$l_ubf+inp_rab.rab$w_rsz;

if (part1) {
if (*--s == '\r') {
rfa_blk = block+((s+2)-inp_rab.rab$l_ubf>>9);
rfa_off = ((s+2)-inp_rab.rab$l_ubf)&511;
if (++rcd_num >= record_count) break;
}
++s;
}
part1 = 0;
while (s >= inp_rab.rab$l_ubf) {
if (*--s == end_char) {
char *nrp;
nrp = s+1; /* next record starts here */
if (type == 2) { /* if strmcrlf */
if (s <= inp_rab.rab$l_ubf) { /* if on the cusp */
part1 = 1; /* defer */
break; /* get somemore data */
}
if (*--s != '\r') { /* else if next char is not cr */
++s; /* then this is not a record */
continue;
}
}
rfa_blk = block+(nrp-inp_rab.rab$l_ubf>>9);
rfa_off = (nrp-inp_rab.rab$l_ubf)&511;
if (++rcd_num >= record_count) break;
}
}
if (rcd_num >= record_count) break; /* we got everything */
}
if (rfa_blk == 0) return RMS$_EOF;
return do_seqout(rfa_blk,rfa_off); /* write out records */
}

/************************************************************************
* do_varfast. This procedure trys to figure out the record structure
* of a variable length file while reading from the end of the file.
*
* At entry:
* (called by do_var)
* At exit:
* returns a 0 if it couldn't determine a valid record structure.
* called do_seqout with a computed block and offset if it did
* successfuly find an appropriate record boundary.
* returns with RMS status in all cases.
*************************************************************************/

do_varfast()
{
unsigned long block,rfa_blk=0;
int rcd_num= 0,err,rec_len,rfa_off,min_recsiz;
union {
unsigned char *s;
unsigned short *len;
unsigned int align;
} rp;

inp_rab.rab$l_bkt = inp_xab.xab$l_ebk+1;
rec_len = 0;
min_recsiz = inp_fab.fab$b_fsz;
inp_rab.rab$w_usz = ((inp_xab.xab$w_lrl+3)*record_count+511)&~511;

while(1) {
if (inp_rab.rab$l_bkt <= 1) break; /* quit if we already read blk 1 */
block = inp_rab.rab$l_bkt-(inp_rab.rab$w_usz>>9);
if (block == 0 || block > inp_rab.rab$l_bkt) block = 1; /* but can't start before 1 */
inp_rab.rab$l_bkt = block; /* remember starting block number */
err = sys$read(&inp_rab);
if ((err&1) == 0) {
if (err == RMS$_EOF) break;
fprintf(stderr,"Error (%08X) trying to read %d bytes at block %d, record %d\n",
err,inp_rab.rab$w_usz,block,rcd_num);
continue;
}
rp.s = inp_rab.rab$l_ubf+inp_rab.rab$w_rsz;
if ((rp.align&1) == 1) ++rp.align; /* ffb must be even */

/*************************************************************************
* The following procedure loops through the buffer picking up pairs of bytes (a
* short) on even byte boundaries from the end. If this pair of bytes forms a
* integer whose value points within 1 of the the next record or the current eof,
* then the pair is assumed to be the count field of a valid record and is so
* recorded. It can easily get screwed up if there is binary data in the records,
* but for ascii files such as log files, it ought to work well enough most of
* the time. The integer value represents the length of the record. If this value
* is greater than the maxium record length recorded for the file (in the
* XAB$W_LRL field of the XABFHC), then this routine assumes to have gotten lost,
* so it rolls over and dies.
*************************************************************************/

while (1) {
int len,t;
char *tp;
rp.s -= 2; /* backup 2 bytes */
if (rp.s < inp_rab.rab$l_ubf) break; /* too far, get more data */
len = *rp.len;
if (rec_len >= min_recsiz && /* if currently at or greater than min */
(len == rec_len || /* and lengths match exactly */
(rec_len > 0 && /* or less by 1 */
len == rec_len-1))) {
rfa_blk = block+(rp.s-inp_rab.rab$l_ubf>>9); /* remember this point */
rfa_off = (rp.s-inp_rab.rab$l_ubf)&511;
rec_len = 0;
if (++rcd_num >= record_count) break;
continue;
}
rec_len += 2; /* increase size of record */
if (rec_len > inp_xab.xab$w_lrl) return 0; /* we're lost, give up */
}
if (rcd_num >= record_count) break; /* we got everything */
}
if (rfa_blk == 0) return RMS$_EOF;
return do_seqout(rfa_blk,rfa_off); /* write out records */
}

/************************************************************************
* skip_through - this routine positions to the desired record begining
* with a known starting position. It is used in monitor mode to skip
* records that may have been added to the input file since the last
* time it was looked at it.
*
* At entry:
* last_rfa_blk and last_rfa_off point to the last record read.
* At exit:
* called do_seqout with a computed block and offset
* last_rfa_blk and last_rfa_off point to a new last record read.
*
************************************************************************/

int skip_through()
{
int err;
int first_rfa=0; /* index to first rfa */
int next_rfa=0; /* index to next available rfa */

if (rfas == (Rfa *)0) rfas = (Rfa *)calloc(record_count,sizeof(Rfa));
inp_rab.rab$l_rfa0 = last_rfa_blk; /* where we left off */
inp_rab.rab$w_rfa4 = last_rfa_off;
inp_rab.rab$b_rac = RAB$C_RFA; /* change to RFA access mode */
inp_rab.rab$l_bkt = 0; /* make sure bkt field is off */
inp_rab.rab$l_rhb = inp_buf; /* init the ptrs */
inp_rab.rab$l_ubf = inp_buf + inp_fab.fab$b_fsz; /* in case making VFC file */
while (1) {
(rfas+next_rfa)->block = inp_rab.rab$l_rfa0;
(rfas+next_rfa)->offset = inp_rab.rab$w_rfa4;
next_rfa += 1;
next_rfa %= record_count;
if (next_rfa == first_rfa) {
first_rfa += 1;
first_rfa %= record_count;
}
err=sys$get(&inp_rab);
if ((err&1) == 0) {
if (err != RMS$_EOF) {
fputs("Error reading input\n",stderr);
exit(err);
}
break;
}
inp_rab.rab$b_rac = RAB$C_SEQ; /* switch back to sequential */
}
return do_seqout((rfas+first_rfa)->block,(rfas+first_rfa)->offset);
}

/************************************************************************
* do_var. This procedure positions to the nth record from the end of
* a file by reading the whole file into memory 127 blocks at a time
* and skipping through the records recording the block and offset of
* each as it goes.
*
* At entry:
*
* At exit:
* called do_seqout with the computed block and offset
* last_rfa_blk and last_rfa_off updated to point to the last record read.
* returns with RMS status.
*************************************************************************/

do_var()
{
long block,rcd_num= 0;
int err;
int first_rfa=0; /* index to first rfa */
int next_rfa=0; /* index to next available rfa */
unsigned char *ebp;
union {
char *s;
unsigned short *len;
unsigned int align;
} rp;

/* Unless otherwise indicated, trys to do the fast way first. If that fails,
* it'll do the hard way. */

if (the_safe_way == 0 && inp_xab.xab$l_ebk > (inp_rab.rab$w_usz>>9)) {
if ((err=do_varfast()) != 0) return err;
fputs("Unable to determine record structure from backend of file.\n",stderr);
fputs("Doing it the 'safer' way instead.\n",stderr);
inp_rab.rab$w_usz = sizeof(inp_buf); /* assume to read the max */
}
rp.s = inp_rab.rab$l_ubf;
inp_rab.rab$l_bkt = 1;
if (rfas == (Rfa *)0) rfas = (Rfa *)calloc(record_count,sizeof(Rfa));
while(1) {
block = inp_rab.rab$l_bkt;
err = sys$read(&inp_rab);
inp_rab.rab$l_bkt += (inp_rab.rab$w_rsz+511) >> 9;
if ((err&1) == 0) {
if (err == RMS$_EOF) break;
fprintf(stderr,"Error (%08X) trying to read %d bytes at block %d, record %d\n",
err,inp_rab.rab$w_usz,block,rcd_num);
continue;
}
ebp = inp_rab.rab$l_ubf+inp_rab.rab$w_rsz;
while (1) {
int len,t;
char *tp;
if ((rp.align&1) == 1) ++rp.align;
if (rp.s >= ebp) {
t = rp.s - ebp;
inp_rab.rab$l_bkt += t>>9;
rp.s = inp_rab.rab$l_ubf+(t&511);
break;
}
++rcd_num;
len = *rp.len;
if (len > 32767) {
if (len == 0xFFFF) {
rp.s = inp_rab.rab$l_ubf;
break; /* eof */
}
fprintf(stderr,"Warning: Record %d (blk=0x%08X,off=0x%04X): cnt %04X greater than 0x7FFF\n",
rcd_num,(rp.s-inp_rab.rab$l_ubf>>9)+block,(rp.s-inp_rab.rab$l_ubf)&511,len);
}
t = rp.s-inp_rab.rab$l_ubf;
(rfas+next_rfa)->block = block + (t >> 9);
(rfas+next_rfa)->offset = t & 511;
(rfas+next_rfa)->length = len;
next_rfa += 1;
next_rfa %= record_count;
if (next_rfa == first_rfa) {
first_rfa += 1;
first_rfa %= record_count;
}
rp.s += len+2;
}
}
if ((rfas+first_rfa)->block == 0) return RMS$_EOF;
return do_seqout((rfas+first_rfa)->block,(rfas+first_rfa)->offset);
}

/************************************************************************
* do_fixed. This procedure positions to the nth record from the end of
* a fixed length record file by computing the desired rfa from the
* required record number, the size of the file and the size of each
* record. This is by far the simplest of the three decoding routines.
*
* At entry:
*
* At exit:
* called do_seqout with the computed block and offset
* last_rfa_blk and last_rfa_off updated to point to the last record read.
* returns with RMS status.
*************************************************************************/

do_fixed()
{
int reccnt;
unsigned long size,sbn;
long start,rfa_blk,rfa_off;

size = inp_xab.xab$l_ebk*512+inp_xab.xab$w_ffb; /* file size in bytes */
reccnt = size/inp_xab.xab$w_mrz; /* total record count */
start = reccnt - record_count; /* starting record # */
if (start < 0) start = 0; /* can't go past beginning */
sbn = start*inp_xab.xab$w_mrz; /* starting byte # */
rfa_blk = (sbn >> 9) + 1; /* starting block number */
rfa_off = sbn & 511; /* offset in block */
if (rfa_blk < last_rfa_blk || (rfa_blk == last_rfa_blk && rfa_off < last_rfa_off)) {
rfa_blk = last_rfa_blk; /* don't display records already displayed */
rfa_off = last_rfa_off+inp_xab.xab$w_mrz; /* advance 1 record */
if (rfa_off >= 512) { /* and adjust pointers if appropriate */
rfa_off -= 512;
rfa_blk += 1;
}
}
return do_seqout(rfa_blk,rfa_off); /* write the end of the file */
}

int tti_ast()
{
int sts;
if (tti_iosb.status == SS$_ABORT) {
return SS$_NORMAL; /* aborts are ok, since they'll happen at exit */
}
if (tti_iosb.status == SS$_NORMAL || tti_iosb.status == SS$_ENDOFFILE) {
were_done = 1; /* signal that we're done */
sys$canwak(0,0); /* cancel our schwk */
sys$wake(0,0); /* wake up from our hiber */
} else {
if ((tti_iosb.status&1) == 0) {
fputs("Error reading from maillbox\n",stderr);
exit(tti_iosb.status);
}
}
return que_ttiread();
}

int que_ttiread()
{
int sts;
sts = sys$qio( 0, /* efn */
tti_chan, /* channel */
IO$_READVBLK, /* function */
&tti_iosb, /* iosb addr */
tti_ast, /* astadr */
0, /* astparam */
tti_text, /* p1 (buffer ptr) */
sizeof(tti_text), /* p2 (buffer size) */
0,0,0,0); /* p3-p6 not used */

if ((sts&1) == 0) {
fputs("unable to do QIO to SYS$INPUT, monitor mode disabled\n",stderr);
monitor = 0;
}
return sts;
}

/* ==============================fgen.c============================== */

#include <descrip.h>
/* #include <rms.h> ...defined in the program top... */

/*
* fgen(pattern, result_array, array_length)
* fgen generates filenames of files, matching a VMS pattern,
* the results are stored in an array, space for the strings is allocated
* by using malloc.
* Return values:
* -1 : error in pattern
* 0 : no files found
* n(>0) : number of matching filenames. (Stored in result_array [0] - [n-1])
*
* Wildcard expansion for VMS is easy; we just use a run-time library call.
*/

fgen(pat,resarry,len)
char *pat,**resarry[];
int *len;
{
struct dsc$descriptor_s file_spec, result, deflt;
long context;
int count, slen, status, plen;
char *pp, *rp, result_string[256], *strchr();
char *fnp,**fnpp;

file_spec.dsc$w_length = strlen(pat);
file_spec.dsc$b_dtype = DSC$K_DTYPE_T;
file_spec.dsc$b_class = DSC$K_CLASS_S;
file_spec.dsc$a_pointer = pat;

result.dsc$w_length = sizeof result_string;
result.dsc$b_dtype = DSC$K_DTYPE_T;
result.dsc$b_class = DSC$K_CLASS_S;
result.dsc$a_pointer = result_string;

deflt.dsc$w_length = sizeof(default_string)-1;
deflt.dsc$b_dtype = DSC$K_DTYPE_T;
deflt.dsc$b_class = DSC$K_CLASS_S;
deflt.dsc$a_pointer = default_string;

count = 0;
context = 0;
pp = strrchr(pat, ']');
if ( !pp ) pp = strrchr(pat, ':');
if ( !pp ) plen = 0;
else plen = pp - pat + 1;
fnpp = *resarry;
while ((status = LIB$FIND_FILE(&file_spec, &result, &context, &deflt))
== RMS$_NORMAL) {
if (count >= *len) {
if (*len == 0) *len = 256;
if (*resarry == (char **)0) {
*resarry = (char **)malloc(*len*sizeof(char *));
} else {
*len += *len/2; /* increase length by 1/2 again */
*resarry = (char **)realloc(*resarry,*len*sizeof(char *));
}
if (*resarry == (char **)0) {
perror("Unable to malloc/realloc memory");
exit(0x10000004);
}
fnpp = *resarry + count;
}
rp = strrchr(result_string, ']') + 1;
if( !rp )
rp = result_string;
slen = strchr(rp, ' ') - rp;
fnp = *fnpp++ = malloc(slen + plen + 1);
if (plen != 0)
strncpy(fnp, pat, plen);
strncpy(fnp + plen, rp, slen);
fnp[slen + plen] = '\0';
++count;
}
#ifdef DVI$_ALT_HOST_TYPE
lib$find_file_end(&context); /* Only on V4 and later */
#endif
if (status == RMS$_FNF) return(0);
if (status == RMS$_NMF) return(count);
return(-1);
}
Jon Pinkley
2020-11-08 23:47:28 UTC
Reply
Permalink
Post by Phillip Helbig (undress to reply)
That is often useful, though TYPE/TAIL sometimes doesn't work (I'm not
sure exactly when). Before TYPE/TAIL was available, I used the
followed by c source code for TAIL version 2.4, 09/10/91. D. Shepperd

For future readers, TAIL 2.4 with some additions by Ehud Gavron (help file and some changes to support Alpha) is on Hunter Goatley's FILESERV and mirrors.

For example, here: http://vms.process.com/ftp/vms-freeware/fileserv/tail.zip

Other file viewing tools there. LOOK, EXTRACT, EPYT (type backwards)

Another is VLOG on the VAX92A decus tape (on Eisner.decus.org in disk$repository:[MASTER.VMS.SIG_TAPES.VAX92A.NSWC]) or decus archives for example at digiater.nl in https://www.digiater.nl/openvms/decus/vax92a/nswc/ (Fortran sources are in an lz compressed vms backup saveset).

I have only used LOOK and TAIL (but currently have TAIL == "TYPE/TAIL" in my login.com)

Another thread on comp.os.vms for reference https://groups.google.com/d/msg/comp.os.vms/9IzdS82lYxk/dUi07WmWVcMJ
Phillip Helbig (undress to reply)
2020-11-08 11:36:39 UTC
Reply
Permalink
Post by Jon Pinkley
Post by Phillip Helbig (undress to reply)
Post by Jon Pinkley
The main thing I like about it is that it can read open batch log files,
unlike an editor.
EDIT/TPU/READ will allow one to view open batch log files...
Phillip, thanks for pointing that out, I thought I tried using
EDIT/TPU/READ, but I just verified that EDIT/TPU/READ does work, but
EDIT/EDT/READ does not.
Right. That is one reason I sometimes use TPU. Others are when there
are long lines (though I think that this has been fixed in the
meantime), when more than 65,535 commands need to be processed at once,
or when I need the "learn" functionality. Otherwise, I prefer EDT since
it starts up more quickly, macros can be written in a compact style, it
doesn't load the entire file unless necessary, the cursor movement is
fast and more to my liking, and so on.
Post by Jon Pinkley
My guess is that I don't like EVE for large files,
Same here.
Post by Jon Pinkley
and when I tried
EDIT/EDT/READ, I just assumed that EDIT/TPU/READ would be the same.
I'm not sure why EDT (which DOES have a /READ_ONLY mode) has a problem
with open batch log files. One can get around it with a simple script
which uses BACKUP/IGNORE=INTERLOCK (a good example of a case where it
makes since to use this qualifier) to create a temporary file (though
that could take a while if the file is huge) and then run EDT on that.
Simon Clubley
2020-11-09 13:23:26 UTC
Reply
Permalink
Post by Jon Pinkley
So the question is, what tools do other people use to look at log files that are opened by long running batch jobs?
pip filename.log/mo:4096

(Sorry, wrong operating system :-), but you have just prompted a new
VMS enhancement suggestion).

Simon.
--
Simon Clubley, ***@remove_me.eisner.decus.org-Earth.UFP
Walking destinations on a map are further away than they appear.
Hunter Goatley
2020-11-09 16:20:55 UTC
Reply
Permalink
Post by Jon Pinkley
So the question is, what tools do other people use to look at log files that are opened by long running batch jobs?
I use my FLIST to look at everything, including open batch log files. It
is written in TPU, so it does read the whole file into memory, so it's
not always practical for really large log files. Without a doubt, FLIST
is my most used daily utility on VMS.

FLIST is a TPU-based directory and file manager. You can use it to
peruse directories, view files, delete files, rename files, etc.

http://vms.process.com/scripts/fileserv/fileserv.com?FLIST

http://vms.process.com/ftp/vms-freeware/fileserv/flist.zip
ftp://ftp.process.com/vms-freeware/fileserv/flist.zip
--
Hunter
------
Hunter Goatley, Process Software, http://www.process.com/
***@goatley.com http://hunter.goatley.com/
Loading...