[Search for users] [Overall Top Noters] [List of all Conferences] [Download this site]

Conference azur::mcc

Title:DECmcc user notes file. Does not replace IPMT.
Notice:Use IPMT for problems. Newsletter location in note 6187
Moderator:TAEC::BEROUD
Created:Mon Aug 21 1989
Last Modified:Wed Jun 04 1997
Last Successful Update:Fri Jun 06 1997
Number of topics:6497
Total number of notes:27359

2788.0. "Performance: Record vs. Export" by ICS::WOODCOCK () Fri Apr 17 1992 19:20

Hello all MCC implementers/developers,

This note is an attempt to understand performance for RECORD and
EXPORT activities with respect to Disk I/O. In the past most references to
performance have been towards MEMORY and the need for it. I suspect there
may also be problems in the area of disk I/O which can cause major performance
problems/frustration and it's a bit tougher to detect.

System performance is *not* my forte' so if any assumptions made here look
wrong please pipe in. I'm making my own decisions based on this data and if
it's wrong I also suffer with improper choices in the future.

History:

ANCHOR was sunk to the bottom of the ocean using MCC because we weren't 
careful how we used our disks. MCC_COMMON was placed on the system disk.
Generally, this probably isn't the best idea if you can possibly avoid it but
still wasn't our major problem. The problem was with the user disk which
also contained the historical files. Polling 70 circuits each half hour 
(counters) using 12 batch processes (because it was convenient, it matched 
the map) with RECORD statements bottlenecked the disk I/O. This made MCC 
very slooooooooooooow!

The solution was to reduce the polling to every hour, reduce the processes
polling to 4 by creating seperate domains, and even move all log files off the
disk when possible to simply reduce the total number of open files on the user
disk. MCC_COMMON was also moved to its own disk.

This was quite some time ago but I have now found the time to play with the
numbers to try and understand the disk limitations. What we saw back then
with "$ monitor disk" was an I/O at a constant value of 28-30. This was the
saturation point of the RA82 disk. From what I have heard it doesn't get much
better than this for most disks (usually a bit less).

Current:

We are still using 4 polling processes to RECORD 73 data circuits. Polling
is set for:

	Node4 Circuit counters - hourly
	Node4 Circuit char.    - daily (default)
        Node4 Line char.       - daily (default)
                                 ------
                                 26 partitions/day/circuit

These are the 'minimum' partitions needed to produce node4 circuit 
statistics for hourly intervals, the desired endpoint. Doing a "$ monitor 
disk" for a total of an hour typically lists the average around 7-8 and the 
peak around 35 for this disk. It doesn't seem to matter too much if the map 
is running and alarms are polling (but not firing often). For convienience 
we'll call the average "7.3". This is for 73 circuits therefore a per circuit 
average would be 7.3/73 = 0.1 I/O. Because most of the partitions (24 of 26) 
are for the hourly counters you could also approximate that each partition is 
0.1 also (polling hourly).

Comparison to EXPORT: Using EXPORT to produce the same end result (hourly
utilization figures) you would have to set the EXPORT poll period to 1
hour. This results in writing the following partitions to disk per hour per
circuit: Counters, Characteristics, Identifiers, Reference Attributes, Status,
Statistics, Initial Attributes. This would be 7 partitions/hour as compared
to 1 partition/hour for RECORD. Comparitively, this may not be a 1 for 1
because there could be less activity for some partitions which EXPORT tries
to write but which aren't needed for RECORD. At this point one could say that
EXPORT appears to take at least "6" times the I/O rate compared to RECORD,
*in theory*. Therefore, EXPORT requires 0.6 I/O per circuit for hourly
polling. Also, EXPORT requires calling the PA module to continually calculate
statistics. Using RECORD means only calling PA when statistics are required
to be calulated when issuing PA commands (less continual CPU).

Conclusion: 

If a disk saturates at a consistent I/O rate of 30, then one probably doesn't
want to "plan" on anything more than 75-80% of this rate (22-24). Being the
good guys that we are (you don't have to be as nice) we'll call it 24.

MCC can manage 240 circuits with this disk by RECORDing the partitions as 
described above (24/0.1=240). This can also be equated to about 240 partitions
per hour. 

MCC can manage 40 circuits with this disk by EXPORTing as described above 
(24/0.6=40). This could also be equated to 40 circuits per hour. This number
appears a bit low but this could explain the complaints, any experience out 
there???

General Comments:

EXPORTing 'limits' the user to the least common denominator if other uses
are needed for the db. For example, an extremely useful report is to get a
snapshot of the entire network status for a given interval. If the user needs
to know what the entire network (from home) looks like with a maximum 
time lag of 15 minutes, the EXPORTs must be set for 15 minutes even though
utilization numbers are only needed hourly. That'll cost you.

MCC may want to consider breaking EXPORTs by partition if possible for future 
versions.

When either RECORDing or EXPORTing be aware of the default poll period. If
that interval is shorter than actually needed then take the effort to be
specific and define the poll period to the maximum needed. If you only need
utilization, hourly is quite sufficient (especially for long term trend
analysis). Occasionally there may be a need for shorter intervals but it is
typically for specific performance problems where the user should simply set
up a seperate RECORD and domain for handling unique analysis work.

When using RECORD, forgo the temptation of "partition=*" unless you actually
need all the partitions. This is especially true for node4 LINE because you 
only need characteristics once a day to calculate CIRCUIT statistics. Take 
the time to list seperate commands, better safe than sorry.

Be aware that wherever you create your domains from (directory), will be the 
repository for the RECORD historical files. If you're planning on using the
historical data services, try to use a seperate disk. If using RECORD services
then:

	$ set def <history_directory>
	$ mana/enter
	  create domain <history_1>
          create domain <history_2>
		.
		.
		.
	  create domain <history_1> member .<member_1>
          create domain <history_1> member .<member_2>
		.
		. etc.
	  exit
	$ 

One missing piece of data is the number of circuits or polls achievable
from a single process. We've currently got 20 circuits (hourly) and this looks
like it will go to at least 25 easily. Create seperate historical domains to
reduce the number of these processes if necessary, this is a MAJOR 
consideration not only for the disk, but for CPU and MEMORY also.

best regards,
brad...
T.RTitleUserPersonal
Name
DateLines