[Search for users] [Overall Top Noters] [List of all Conferences] [Download this site]

Conference vaxaxp::vmsnotes

Title:VAX and Alpha VMS
Notice:This is a new VMSnotes, please read note 2.1
Moderator:VAXAXP::BERNARDO
Created:Thu Jan 23 1997
Last Modified:Fri Jun 06 1997
Last Successful Update:Fri Jun 06 1997
Number of topics:703
Total number of notes:3722

301.0. "Using indexed files?" by CHEFS::WILLIAMSA (I wanna be Luke) Mon Mar 10 1997 13:47

    Hi all...
    
    I'm currently producing some tools (in C) for monitoring customers
    applications. One of the things I need to do is record jobstep
    information for these applications. I have written a procedure
    and the cld to do this, basically it records process names, accounting
    information when the command was called etc. Now comes my query. I need
    to store this information and have it easily accessible, searching by
    process name, the customer will use the querying mechanism for both
    reporting and possibly synchronisation purposes. Ie has process LO1
    finished step 25???
    
    The query is how best to store the information and access it. I suspect
    I am limiting myself to indexed files, however my knowledge of indexed
    files is pretty small (miniscule). Could someone suggest any other
    mechanisms or better still is there an 'idiots guide to indexed files',
    I've been through the RMS manuals, but they tend to assume you know
    what ya doing!!!
    
    Any ideas/pointers gratefully received.
    
    Alen.
T.RTitleUserPersonal
Name
DateLines
301.1RMS Examples Available...XDELTA::HOFFMANSteve, OpenVMS EngineeringMon Mar 10 1997 15:10314
   The process name is not a unique value, and there can be multiple
   processes with the same process name active at one time, and many
   more duplicates over the life over the system.

   The process PID is a better choice to identify a process over the
   life of the system, and the process slot is a better choice at any
   particular instant.  (I would tend to use the process PID, and/or
   an application-constructed "lock resource name" here.)

   File-based synchronization is a classic design found on PC and UNIX
   systems, and is workable, but tends to leave a lot to be desired.
   Under OpenVMS, one likely wants to look at a tool such as a "lock"
   -- "locks" are explictly intended for multi-process synchoronization,
   and for recovery from errors and unexpected process exits.

   As for how to program an application that uses an indexed file, see
   the RMS and DEC C manuals.  (Once one figures out and gets the RMS
   FAB, RAB, XABSUM, and XABKEY structures in shape, reading and writing
   an indexed file is trivial.  :-)

   To write up an application that works with an indexed file, you will
   want to learn about fdl$create and FDL files (to create the initial
   baseline file), and then about calling sys$open with a FAB, and XABSUM
   block, then -- based on the number of keys found referenced in
   the XABSUM -- chain off a sufficient number of XABKEY blocks and call
   sys$display.  Then create a FAB and tie it into the FAB, and call
   sys$connect.  Now you can start calling sys$get, sys$put, and
   sys$update, with records.

   To get fancier, place just the file name in the FAB file name
   field, and place the device and directory (or logical name), and
   file extension in the FAB default file name field.  (This is how
   one can access SYSUAF be defining a SYSUAF logical -- the primary
   specification in the file open is "SYSUAF", and the default is
   "SYS$SYSTEM:.DAT".  If the logical is found, the fields from the
   logical override the fields from the default.

   There are examples in the RMS Guide to Files, in the DEC C manuals,
   in the VMSZOO::RMS_OPENVMS and TURRIS::DECC notes conferences, and
   on the Freeware CD-ROMs, among other places.  See note 5 in the
   RMS conference, and see below for a basic sequential file read
   program...  (This is very similar to the sequence used to perform
   a sequential read on an RMS indexed file key...)

   You will also want to take a look at the RMS Programming Concepts
   manual, for information on programming with the sys$enq[w] and
   sys$deq[w] services and with the lock manager in general.  (The
   lock manager is *far* more than "just for file and record locks";
   it's *much* more powerful...  It's very useful for process control
   applications, particularly in a VMScluster environment...)
    
:    The query is how best to store the information and access it. I suspect
:    I am limiting myself to indexed files, however my knowledge of indexed
:    files is pretty small (miniscule). Could someone suggest any other
:    mechanisms or better still is there an 'idiots guide to indexed files',
:    I've been through the RMS manuals, but they tend to assume you know
:    what ya doing!!!

   Please remember to log one or more QARs against the RMS Reference
   Manual and the Guide to File Applications.  (If you do not understand
   this and are looking for an "idiot's guide", there are likely other
   folks with similar problems and confusion caused by these manuals.)

   --


/*
** COPYRIGHT (c) 1992 BY
** DIGITAL EQUIPMENT CORPORATION, MAYNARD, MASSACHUSETTS.
** ALL RIGHTS RESERVED.
**
** THIS SOFTWARE IS FURNISHED UNDER A LICENSE AND MAY BE USED AND COPIED
** ONLY  IN  ACCORDANCE  OF  THE  TERMS  OF  SUCH  LICENSE  AND WITH THE
** INCLUSION OF THE ABOVE COPYRIGHT NOTICE. THIS SOFTWARE OR  ANY  OTHER
** COPIES THEREOF MAY NOT BE PROVIDED OR OTHERWISE MADE AVAILABLE TO ANY
** OTHER PERSON.  NO TITLE TO AND  OWNERSHIP OF THE  SOFTWARE IS  HEREBY
** TRANSFERRED.
**
** THE INFORMATION IN THIS SOFTWARE IS  SUBJECT TO CHANGE WITHOUT NOTICE
** AND  SHOULD  NOT  BE  CONSTRUED  AS A COMMITMENT BY DIGITAL EQUIPMENT
** CORPORATION.
**
** DIGITAL ASSUMES NO RESPONSIBILITY FOR THE USE  OR  RELIABILITY OF ITS
** SOFTWARE ON EQUIPMENT WHICH IS NOT SUPPLIED BY DIGITAL.
*/

/*
**++
**  Facility:
**
**	Examples
**
**  Version: V1.1
**
**  Abstract:
**
**	Example of working with RMS calls from C
**
**  Author:
**	Steve Hoffman (XDELTA::HOFFMAN)
**
**  Creation Date:  1-Jan-1990
**
**  Modification History:
**	    Hoffman	15-Oct-1994	Updates for DEC C
**--
*/

/*
//  RMS_EXAMPLES.C
//
//  Program displays some RMS calls made from the c language.  Types out
//  the calling process's SYS$LOGIN:LOGIN.COM file to SYS$OUTPUT.
//
//  Included is a main and three subroutines.  The subroutines open,
//  read a record, and close the file.  Several hooks, such as the
//  use of the NAM block to obtain the specification of the file that
//  was actually opened, are included but are not currently used.
//
//  To build:
//
//    $ CC [/DECC] [/DEBUG/NOOPTIM] RMS_EXAMPLES
//    $ LINK [/DEBUG] RMS_EXAMPLES
//    $ RUN [/[NO]DEBUG] RMS_EXAMPLES
*/

#include <lib$routines.h>
#include <rms.h>
#include <starlet.h>
#include <string.h>
#include <ssdef.h>
#include <stdio.h>
#include <stsdef.h>

/*
// RMS_MRS is the maximum record size that can be read (and thus
// displayed) by this program.
*/
#define RMS_MRS	255

/*
// The following is the core data structure for the program.
// The various RMS subroutines all communicate via a pointer
// referencing this struct.
*/
struct RmsFileContext
    {
    struct FAB fab;
    struct RAB rab;
    struct NAM nam;
    char rss[NAM$C_MAXRSS];
    short max_rec_siz;
    char *data_buffer;
    };

RmsFileOpen( void **CtxArg, 
char *FileName, char *DefFileName, int flags, int rss )
    {
    int RetStat;
    struct RmsFileContext *Ctx;
    int howbig = sizeof( struct RmsFileContext );

    /*
    // acquire some space for a Context block.
    */
    RetStat = lib$get_vm( &howbig, &Ctx, 0 );

    if ( !$VMS_STATUS_SUCCESS( RetStat ) )
	return RetStat;

    *CtxArg = (void *) Ctx;

    /*
    // Fill in the various fields of the Context block.
    // -- Builds the File Access Block (FAB), the Record Access
    // Block (RAB) and the Name (NAM) Block.  Along with some
    // other miscellaneous housekeeping stuff.
    */
    Ctx->fab = cc$rms_fab;
    Ctx->rab = cc$rms_rab;
    Ctx->nam = cc$rms_nam;

    Ctx->fab.fab$l_nam = &Ctx->nam;
    Ctx->fab.fab$l_fop = FAB$M_NAM;
    Ctx->fab.fab$b_fac = FAB$M_GET;

    Ctx->fab.fab$l_fna = FileName;
    Ctx->fab.fab$b_fns = strlen( FileName );
    Ctx->fab.fab$l_dna = DefFileName;
    Ctx->fab.fab$b_dns = strlen( DefFileName );

    Ctx->rab.rab$l_fab = &Ctx->fab;

    Ctx->nam.nam$b_rss = NAM$C_MAXRSS;
    Ctx->nam.nam$l_rsa = Ctx->rss;

    Ctx->rab.rab$b_rac = RAB$C_SEQ;

    /*
    // Attempt to open the file...
    */
    RetStat = sys$open( &Ctx->fab, 0, 0 );
    if ( !$VMS_STATUS_SUCCESS( RetStat ) )
	return RetStat;

    /*
    // Allocate a buffer large enough for the biggest record.
    */
    RetStat = lib$get_vm( &RMS_MRS, &Ctx->data_buffer, 0 );
    if ( !$VMS_STATUS_SUCCESS( RetStat ) )
	return RetStat;

    /*
    // Attempt to connect the record stream to the file...
    */
    RetStat = sys$connect( &Ctx->rab, 0, 0 );
    if ( !$VMS_STATUS_SUCCESS( RetStat ) )
	return RetStat;

    return RetStat;
    }

RmsFileRead( void **CtxArg, char **BufAdr, int *BufLen )
    {
    int RetStat;
    struct RmsFileContext *Ctx = *CtxArg;

    Ctx->rab.rab$l_ubf = Ctx->data_buffer;
    Ctx->rab.rab$w_usz = RMS_MRS;

    RetStat = sys$get( &Ctx->rab, 0, 0 );

    if ( !$VMS_STATUS_SUCCESS( RetStat ) )
	{
        *BufLen = (char) 0;
	*BufAdr = (char) 0;
	return RetStat;
	}

    *BufAdr = Ctx->rab.rab$l_rbf;
    *BufLen = Ctx->rab.rab$w_rsz;

    return RetStat;
    }

RmsFileClose( void **CtxArg )
    {
    int RetStat;
    struct RmsFileContext *Ctx = *CtxArg;

    /*
    // Free up the record buffer...
    */
    RetStat = lib$free_vm( &RMS_MRS, &Ctx->data_buffer, 0 );
    if ( !$VMS_STATUS_SUCCESS( RetStat ) )
	return RetStat;

    /*
    // Be nice and clean up the record stream...
    */
    RetStat = sys$disconnect( &Ctx->rab, 0, 0 );
    if ( !$VMS_STATUS_SUCCESS( RetStat ) )
	return RetStat;

    /*
    // And close the file...
    */
    RetStat = sys$close( &Ctx->fab, 0, 0 );
    if ( !$VMS_STATUS_SUCCESS( RetStat ) )
	return RetStat;

    /*
    // And free up the allocated memory...
    */
    RetStat = lib$free_vm( &sizeof( struct RmsFileContext ), CtxArg, 0 );
    if ( !$VMS_STATUS_SUCCESS( RetStat ) )
	return RetStat;

    return RetStat;

    }
main()
    {
    int RetStat;
    void *Context;
    char *BufAdr;
    int BufLen;

    /*
    // Open the file.  Minimal checking is performed.  Read access only.
    */
    RetStat = RmsFileOpen( &Context, "LOGIN", "SYS$LOGIN:.COM", 0, 0 );

    /*
    // Read the file.  Minimal checking is performed.
    */
    for (;;)
	{
	RetStat = RmsFileRead( &Context, &BufAdr, &BufLen );
	if ( $VMS_STATUS_SUCCESS( RetStat ) )
	    printf("%*.*s\n", BufLen, BufLen, BufAdr );
	else
	    break;
	}

    /*
    // Close up shop.
    */
    RetStat = RmsFileClose( &Context );

    return RetStat;
    }

301.2AUSS::GARSONDECcharity Program OfficeTue Mar 11 1997 00:1316
re .0
    
    For recording this information an indexed file is a possibility. If the
    reporting is complex and the requirements dynamic though you might
    consider a relational database. If all you are doing is retrieving by
    one key (e.g. process name) then an indexed file is fine.
    
    An indexed file is probably not the best means of synchronising jobs.
    You might want to look at DECscheduler (or whatever its nom du jour is)
    or just the vanilla DCL SYNCH command however both of those (I suspect)
    don't go to "job step" level. If you need that level of synchronisation
    then you would have to roll your own and the VMS lock manager could be
    a part of the solution.

    Reading between the lines a little, it sounds as if you would be better
    off using a caller supplied "job name" than the process name.
301.3DCL example using an indexed file.EPS::VANDENHEUVELHeinTue Mar 11 1997 20:36118
    
    I agree with the other expressed sentiments of it being more
    desirable to go for a relational DB solution. Still, If you'd
    also like to do this stuff from DCL, then you may want to peek 
    over the include command file. It ain't pretty, but it should 
    should give you the required hints as to how to get going.
    The example is a single keyed hack to provide simple file
    checkin / checkout. It is not pretty because it hard codes
    the field offsets in the record layout chosen. Should be easy
    enough to add more key to provide alternate lookups (btw, SEARCH
    with clever /MATCH=AND can often be a nice key-less lookup tool
    for ascii formatted files).
    
    perhaps it'll help some,
    
    				Hein.
    
$
$! 
$! This procedure is a basic example of a file check-in check-out facility.
$! It could easily be modified to remember last-editted by information
$! and or a file 'wait' list, notably exploiting duplicate key names
$! The record can be grown for example to hold a comments field.
$! Hope this get you going. Hein van den Heuvel, Oct-1994, Digital.
$!
$labels = "|LOCK|UNLOCK|FORCE_UNLOCK|LIST"
$reservations = "user9$:[vandenheuvel.examples]reservations_file.dat" ! Use, common, accessible filespec
$if p1.eqs."" then inquire p1 "Function"
$IF p1.EQS."" THEN p1 = 0
$IF F$TYPE(p1).NES."INTEGER"            ! By name or by number ?
$  THEN label  = F$ELEM(1,"|",F$EXT(F$LOC("|"+p1,labels),99,labels))
$  ELSE label  = F$ELEM(p1,"|",labels)
$ENDIF
$IF label.EQS."|".OR.label.EQS."" THEN GOTO function_help
$if p2.eqs."" then inquire p2 "File name"
$if p2.eqs."" then exit
$record_not_found = 98994
$record_locked = 98986
$name = f$parse(p2,,,"NAME")+f$parse(p2,,,"TYPE")
$key_name = f$fao("!40AS",name)
$open/read/write/share=write/error=open_error file 'reservations'
$goto 'label
$
$lock:
$! Try to read specified record, getting an error is goodness.
$!
$ read/key=&key_name/index=0/error=lock_read_error/time_out=255 file record
$ write sys$output "Sorry, file ", name, " locked by ", -
		f$ext(40,13,record), " since ", f$ext(53,20,record)
$close_file:
$ save_status = $status
$ close file
$ exit $status
$
$lock_read_error:
$ IF $status.eq.record_not_found
$ then
$    WRITE file F$FAO("!40AS!13AS!%D",name,F$GETJPI("","USERNAME"),0)
$ ELSE 
$    IF $status.EQ.record_locked 
$    THEN WRITE sys$output -
	"Sorry reservation record for ", name, " being accessed now."
$    ELSE
$    ENDIF
$ ENDIF
$ GOTO close_file
$
$force_unlock:
$unlock:
$ READ/KEY=&key_name/INDEX=0/ERROR=unlock_read_error/TIME_OUT=255 file record
$ locked_by = F$EXT(40,12,record)
$ IF locked_by.EQS.F$GETJPI("","USERNAME") .OR. function.EQS."FORCE_UNLOCK"
$ THEN
$	READ/DELETE/KEY=&key_name/INDEX=0/ERROR=oops file record
$ ELSE
$ 	WRITE sys$output "Sorry, file ", name, " locked by ", -
		F$EXT(40,13,record), " since ", F$EXT(53,20,record)
$ ENDIF
$ GOTO close_file
$
$list:
$ CLOSE file
$ SEARCH 'reservations' 'p2'
$ EXIT
$
$unlock_read_error:
$ if $status.eq.record_not_found then -
	write sys$output "Sorry, file ", name, " not locked."
$ if $status.eq.record_locked then write sys$output -
	"Sorry reservation record for ", name, " being accessed now."
$ goto close_file
$
$function_help:
$i = 1	
$WRITE SYS$OUTPUT "Function """,p1,""" is not valid. Choices are:"
$help_loop:
$label = F$ELEM(i,"|",labels)
$IF label.EQS."|" THEN EXIT
$WRITE SYS$OUTPUT "  ",i,"      ",label
$i = i + 1
$GOTO help_loop
$
$open_error:
$create/fdl=sys$input 'reservations
file; org indexed; prot (world:rwe);
area 0; bucket 6
key 0; seg0_pos 0; seg0_len 40; duplicates yes
key 1; seg0_pos 40; seg0_len 12; duplicates yes
$open/read/write/share=write/error=ooops file 'reservations'
$goto 'function
$exit
$ooops:
$write sys$output "Sorry, I give up. Can not access: ", reservations
$exit
$help:
$write sys$output "Valid functions are: ", functions
$exit
    
301.4Found guide, now an official idiot!CHEFS::WILLIAMSAI wanna be LukeThu Mar 13 1997 08:4927
    Now what was that about woods and trees and blindness???
    
    I've found my 'idiots guide' disguised as the "Guide to OpenVMS File
    Applications", after reading this the other RMS type manuals become
    much clearer, as well as the examples I've managed to find.
    
    Re .1 Thanks for the example, very useful.
    
    Re .2 Err, spot on with the reading between the lines, the main purpose
    of this procedure is simply recording application progress, the fact
    that it 'could' be used for some simple DCL level synchronisation is an
    added bonus, not it's primary use. I've already included the ability to
    use an alternate user supplied name, if however no value is supplied it
    defaults to the process name.
    
    The question of a relational database is easier to deal with, the
    querying is merely based on things such as "Has process x finished step
    10 yet?" and nothing more complex, using a database would appear to be
    cracking a small nut with a large sledge hammer, besides which 'tis
    probably beyond the abilities of a mere Systems Mangler spud like me!!!
    
    Now 'all' (ha) I have to do is code the stuff up, but hopefully with
    the examples I've managed to find I should be able to give it a go.
    
    Thanks to all for your input.
    
    Alen.
301.5EPS::VANDENHEUVELHeinThu Mar 13 1997 13:5316
 >   querying is merely based on things such as "Has process x finished step
 >   10 yet?" and nothing more complex, using a database would appear to be
    
    
    	$set proc/name="xxx munging"
    	$munge munge munge
    	$set proc/name="xxx digesting"
    	$digest digest digest
    	$set proc/name="xxx output phase"
    	$cra..sh
    	
    grins,
    	Hein.
    
    
    
301.6BATCH$RESTARTXDELTA::HOFFMANSteve, OpenVMS EngineeringThu Mar 13 1997 14:547
: >   querying is merely based on things such as "Has process x finished step
: >   10 yet?" and nothing more complex, using a database would appear to be
: 	$set proc/name="xxx munging"
:    	$munge munge munge

   And don't forget the support for the DCL command SET RESTART_VALUE...
    
301.7Oh if it were that simple!CHEFS::WILLIAMSAI wanna be LukeFri Mar 14 1997 08:189
    re .5 & .6
    
    Of course the other query they want to do is:
    
    Query/before=tod/since=13-jan-1902/proc=bonzo/step=xxx
    
    So it has to record the information, on the fly isn't enough!
    
    Alen.
301.8Sounds Like One Of Several Existing Process ControllersXDELTA::HOFFMANSteve, OpenVMS EngineeringFri Mar 14 1997 12:367
:                        -< Oh if it were that simple! >-
:    Of course the other query they want to do is:
:    Query/before=tod/since=13-jan-1902/proc=bonzo/step=xxx
:    So it has to record the information, on the fly isn't enough!

   What you are starting to describe is a re-implementation of DECscheduler.