[Search for users] [Overall Top Noters] [List of all Conferences] [Download this site]

Conference iosg::all-in-1

Title:ALL-IN-1 (tm) Support Conference
Notice:Please spell ALL-IN-1 correctly - all CAPITALS!
Moderator:IOSG::PYECE
Created:Fri Jul 01 1994
Last Modified:Fri Jun 06 1997
Last Successful Update:Fri Jun 06 1997
Number of topics:2716
Total number of notes:12169

2557.0. "Script file taking a long time" by KERNEL::BURDENI () Thu Mar 06 1997 12:55

    I have a customer query,
    
    This customer says that he is trying to use an ALL-IN-1 script to
    update a data file of approximately 100,000 blocks. The data file is
    Indexed with 1 key (the first field).  Though the script (only in
    Fax format at present), does not use Indexing, only sequential reads.
    
    His question : Why, when he updates a single field in each of the
    records, does the script take 3 hours to run, but when he updates two
    fields it takes 12 hours and 3 fields can take a full day.
    
    This appears to be disproportionate.  Has anyone got any ideas on what
    is at present very little information.  I think the data file
    origated from ALL-IN-1.
    
    Ivan
    
T.RTitleUserPersonal
Name
DateLines
2557.1Additional info.KERNEL::BURDENIThu Mar 06 1997 13:099
    Additional information,
    
    The ALL-IN-1 version is v3.1, The file is a data file not created by
    ALL-IN-1.  Also, the procedure has been run on a clear system with
    initialized disks and no other applications running and the timings are
    still disproportionate.
    
    Many Thanks
    Ivan.
2557.2WAGsIOSG::PYEGraham - ALL-IN-1 Sorcerer's ApprenticeThu Mar 06 1997 17:027
    It's a little difficult to say anything without seeing the script...
    
    However, is the file indexed? If so, why is the script doing sequential
    access? How optimal is the file? Have they done ANAL/RMS/FDL on it?
    
    Sensible SYSUAF parameters for the account, which might use a lot of VM
    to read and write it? Disc fragmented?
2557.3SCRIPT in questionKERNEL::BURDENIThu Mar 06 1997 18:4562
    Thanks for the reply, I realise it was a little vague to say the least.
    I have obtained a copy of the script, it is as follows.  There is no
    fragmentation problem, I believe this file is used for other purposes,
    which is where the index comes into play.
    
    I will post the account information when I get it.
!
! test4.scp copy of test3.scp
!
GET #START_TIME = OA$DATE_NBS

DECIMAL I
GET #COUNT_UPDATE = 0

!OPEN FORM LIBRARIES
!
	OA$FLO_OPEN CTS$LIB:DEVELOP

GET #WPF_KEY = "199611"
GET #WPF_SAVESET = "199611"
GET #WPF_TAPE_LABEL = "TEST1NOV"

GET #NEW_DOC_STATUS = "A"

! REMOVED BETWEEN IF AND WRITE
! GET #DOC_STATUS = .CC_DOCUMENT_STATUS \\\\ - 
! GET #NEW_DOC_STATUS = "A" #DOC_STATUS:1:1 \\\\ -

FOR NIB$CCF$ENTRY1 DO -
	IF .CC_DOC_SUBMIT_NBS_DT = #WPF_KEY THEN - 
		WRITE CHANGE NIB$CCF$ENTRY1 -
			%KEY = %KEY, -
			CC_DOCUMENT_STATUS = #NEW_DOC_STATUS, -
			CC_ARCH_TAPE_ID = #WPF_TAPE_LABEL, -
			CC_ARCH_SAVESET = #WPF_SAVESET, -
			CC_ARCH_STORE_DT = oa$date_full \\\\ -
			DECIMAL I \\\\ -
			INCREMENT #COUNT_UPDATE

!	GET #TOTAL = .%COUNT

GET #END_TIME = OA$DATE_NBS
	
	GET #LOG_FILE = 'CTS$DATA:ARCHIVE_KT.log'
	TEXT_FILE OPEN F #LOG_FILE /WRITE
!
! Common fields
!
	GET #MAIL_TEXT = "Start Time 	= " #START_TIME
	TEXT_FILE WRITE F #MAIL_TEXT

	GET #MAIL_TEXT = 'Date		= " #WPF_SAVESET
	TEXT_FILE WRITE F #MAIL_TEXT

	GET #MAIL_TEXT = "End Time	= " #END_TIME
	TEXT_FILE WRITE F #MAIL_TEXT

	GET #MAIL_TEXT = "Total update Archive CCF records = " #COUNT_UPDATE
	TEXT_FILE WRITE F #MAIL_TEXT

	TEXT_FILE CLOSE F
    
2557.4IOSG::PYEGraham - ALL-IN-1 Sorcerer's ApprenticeFri Mar 07 1997 10:5232
    I'm not sure why this should take longer for more updates, but it's not
    a very sensibly written piece of code...
    
    First, taking the 'DECIMAL I' out of the loop might be a good idea!
    
    Why isn't the test of .CC_DOC_SUBMIT_NBS_DT part of the DO? This would
    also mean that OA$SEL_COUNT could be used rather than a separate count:
    
    	FOR NIB$CCF$ENTRY1 WITH .CC_DOC_SUBMIT_NBS_DT = #WPF_KEY DO -
    	    WRITE CHANGE etc.....
    
        GET #MAIL_TEXT = "Total update Archive CCF records = " OA$SEL_COUNT
    
    Since they're using '=' (i.e. begins with) for the comparison, then
    this will always be a serial search.
    
    Also, if they're planning on doing this sort of thing often, it would
    make a lot of sense to make the CC_DOC_SUBMIT_NBS_DT field an
    alternate key, or better, a short version of it that only contains the
    Month and year they're searching for. This would enable them to use an
    '==' comparison, which would then be a keyed lookup.
    
    I guess you're going to have to turn tracing on to see what's going on
    here, although that's going to make a very big log file. I would also
    suggest you look into page faulting or other things happening to
    the process.
    
    Finally, I tend to use this sort of syntax for the key field:
    
    	FOR Form WITH .... DO WRITE CHANGE Form K = .%KEY, ....
    
    I'm not sure if there are any implications of the syntax they're using.
2557.5I shall look at process performance.KERNEL::BURDENIFri Mar 07 1997 11:497
    Thanks very much for all the info, this should at least help with his
    main problem and that is the time taken to run the procedure.  I will
    post any further information I can get from him, and I will have a good
    look at the performance side of things.
    
    Many Thanks
    Ivan.
2557.6Did the trickKERNEL::BURDENIFri Mar 07 1997 19:134
    The changes offered improved the turnaround times considerably, and the
    customer is happy.
    
    Many Thanks.
2557.7Where do I send my consultancy bill? :-)IOSG::PYEGraham - ALL-IN-1 Sorcerer's ApprenticeMon Mar 10 1997 11:090
2557.8ChequeKERNEL::BURDENIMon Mar 10 1997 12:281
    Would you believe that a check is in the post ????
2557.9Nope :-)IOSG::PYEGraham - ALL-IN-1 Sorcerer's ApprenticeMon Mar 10 1997 20:580
2557.10GIDDAY::BACOTWed Mar 12 1997 01:263
    Ahh Graham, I do believe you are becoming a bit of a cynic... ;')
    
    
2557.11(OK, it's a pint on Monday,Graham!)ZUR01::ASHGGrahame Ash @RLEWed Mar 12 1997 10:486
>    Ahh Graham, I do believe you are becoming a bit of a cynic... ;')
    
Something seems to have gone wrong with Notes here - wasn't this Note 
originally posted in about 1984?!

g
2557.12Boom, boom!IOSG::PYEGraham - ALL-IN-1 Sorcerer's ApprenticeWed Mar 12 1997 11:113
    <<<< OK, it's a pint on Monday,Graham!
    
    Alas no, I'll be on holiday (again)...