[Search for users] [Overall Top Noters] [List of all Conferences] [Download this site]

Conference noted::hackers_v1

Title:-={ H A C K E R S }=-
Notice:Write locked - see NOTED::HACKERS
Moderator:DIEHRD::MORRIS
Created:Thu Feb 20 1986
Last Modified:Mon Aug 03 1992
Last Successful Update:Fri Jun 06 1997
Number of topics:680
Total number of notes:5456

297.0. "Table validation and translations" by BARAKA::LASTOVICA (Norm Lastovica) Wed Aug 27 1986 00:48

    here's a topic for thought (now don't get scared).
    
    Have a OLTP system that performs table validations on a number of
    fields.  There are about 40 tables and 100 entries in each (these
    numbers are representitive, not exact.  table sizes vary greatly).
    This system must run blazingly fast.  And so, the table validations
    must be quick!  A table in my context is a list of "words" (each
    up to 50 bytes, though variable) and associated "translations" (same
    length restrictions).  Some tables are for validation only and some
    are for translation as well (though this is not really important).
    
    The current implimentaion is with a global section containing balanced
    binary trees.  It is quite fast.  Problem is, the tables must be
    changable.  On line.  On the fly.  And, the application must run
    on a cluster.  Also, the table section file is not small the way
    it is implimented.
    
    I bet that there is someone who can offer some ideas!  :-)
T.RTitleUserPersonal
Name
DateLines
297.1Use a hash table & keep it <50% fullOCKER::PUCKETTFortran will never dieWed Aug 27 1986 07:130
297.258378::READBobWed Aug 27 1986 19:1017
    We're facing the same problem with two global sections we use for
    validation and translation.  Our plans are to have two locks associated
    with each section.  Lock one is used to indicate ownership of the
    editing function.  Only one editor of a global section at a time
    across a cluster.  Lock two is used to fire a blocking AST on a
    background process running on each system in the cluster.
    
    When a user wants to edit a section, grab the "edit" lock, perform
    the edit, and flush the section out to disk.  Grab the "AST" lock,
    to notify all the background processes that they should update.
    Each of them force an update of the section from disk.  Voila! 
    A cluster-wide global section update.
    
    At least that's what I'm thinking ...
    
    thanks,
    bob.
297.3Yet another approach2132::KLEINThu Aug 28 1986 14:3044
    You could do it with one lock:
    
    	CR mode:	reader
    	PW mode:	editor
    	EX mode:	editor "broadcast" of updates
    
    When the editor is done editing, it flushes the section, then
    promotes the lock to EX mode.  This sends a blocking AST to
    all readers, who demote to NL mode, re-read the section, and
    then re-request the lock in CR mode.
    
    When the editor gets the EX lock granted, it knows that all
    the readers have gotten the message, and releases the lock.
    Then, then readers' re-requests (CR mode) are granted, and
    they can continue.
    
    The real problems with this whole approach are that VMS may
    decide to page in some of the section while it is actually
    in the process of being flushed by the editor.  This can
    cause the section to be inconsistent (unless you can guarantee
    atomicity of updates at a disk-block level).  For example,
    a "pointer" from one block to another may appear to be broken
    because a "new" block has been paged in prior to the sending of
    the "broadcast" message.
    
    In VAX DBMS and Rdb/VMS, we implemented "cluster-wide global
    sections", but the basic difference is that we use page-file
    sections on each node, and "manually" do $QIOs to read and
    write the data in the page-file to the actual data file.
    By preventing VMS from paging directly to the data file, we
    are able to control the refreshing to ensure consistency.
    
    We then use the lock manager to synchronize copies in the cluster,
    much as you described.
    
    One difference, though, is that users on the same node as the
    "editor" should not be required to re-read the section.  We
    avoid this by using the lock value block in the lock to save
    a "version number" for the current copy of the section.  This version
    number is also "stashed" in the page-file section (local to each
    node).  If this is the same as the reader's, the reader doesn't have to
    refresh the section, since his node has the current one already.
            
    -steve-
297.4nice solutionBARAKA::LASTOVICANorm LastovicaSun Aug 31 1986 23:417
    RE: .-1
    	Humm, I like.  My goal was to get some ideas on the translations
    without using a section.  The problems that I see include inability
    to grow the section on the fly, leading to wasted space.  As it
    is, the section is 6700 pages big.  Are there better ways?  An indexed
    file with mucho global buffers?  Too much overhead I think, and
    hardly blazing speed.  Any other thoughts?  I am looking for ideas.
297.5Years later..MDVAX3::COARA wretched hive of bugs and flamers.Mon Nov 23 1987 21:0610
    SEC$M_PAGFIL sections combined with LIB$mumble_TREE calls?
    
    If you do your locking right, you should never have to worry about
    a page being brought in on another processor when you're doing a
    flush.  Always lock the section before doing anything to it, and
    grab it EX mode before a flush.  Nothing will get paged in on ANY
    processor until the requestor can get a lock, and he won't get the
    lock until the flush is complete.  Or am I missing something?
    
    #ken	:-)}