Conference smurf::buildhelp

2591.0. "re: Problem bci ing a large file (make that HUGE)" by AOSG::FILTER (Automatic Posting Software - mail to flume::puck) Tue Oct 15 1996 21:31

Date Of Receipt: 	26-SEP-1996 17:36:02.87
From: 	FLUME::jmf "Joshua M. Friedman Digital UNIX  26-Sep-1996 1734"
To: 	rll@unx.dec.com
CC: 	snow@DEC:.zko.flume, tea@DEC:.zko.flume, odehelp@DEC:.zko.flume,
	stave@zk3.dec.com, tomg@zk3.dec.com, aaron@zk3.dec.com,
	kdolan@DEC:.zko.flume, tresvik@DEC:.zko.flume
Subj: 	re: Problem bci'ing a large file  (make that HUGE)

Rich, please pass this message along to the developer who's trying to
do this check-in.  I've cc'd the QA teams responsible for the pools.

My feeling is that a single file of 145MB is unacceptable use of ODE.

The QA pools should not be accumlating large tar files, with lots of
regression results (for example); these regression results should each
be bci'd and submitted individually, both for best usefulness, as well
as most economic use of resources.

To submit a large tar file into a configuration management system
really makes no sense.  Say there are 1000 test items in that tar file,
and at the next milestone or release stream there are changes to say 2
of them.  If each test is submitted separately, then only 2 need to be
changed, and if they are each 100KB, then to save the changes might
take an additional 200KB, say.  The pool "snapshot" would then reflect
the 2 items that changed.  To put these two changes into the .tar.Z
file would mean that the new .tar.Z binary would be completely
different, and would require another 145MB or whatever to store, and
now the RCS image in the pool is 290MB.  In addition, there's no
configuration management record of what really changed.

We have had some images as large as 5MB in the pool (gemc), and after
10 to 20 check-ins when the underlying RCS file is nearing 100MB, people
have found the performance hit to be unacceptable (like over 20 minutes
for any ODE operation, even completely locally in zk3).

I suggest that the QA teams not allow this sort of "dumping" into the
pools; more thought needs to be given to what is being submitted and
how it should best be structured for how it will be used.

-josh


 ----------------------------------------------------------------------
 Joshua M. Friedman	 		Internet: jmf@zk3.dec.com	
 Mailstop: ZKO3-3/W20			DECnet:   flume::friedman
 Digital UNIX Engineering		603-881-1548, dtn 381-1548
 110 Spitbrook Road 			Fax 603-881-2257, dtn 381-2257
 Nashua, NH  03062 			Office: zko3-3/z20
 ----------------------------------------------------------------------

------- Forwarded Message

To: odehelp@zk3.dec.com
Subject: Problem bci'ing a large file
Date: Thu, 26 Sep 96 15:57:24 -0400
From: Rich Larsen <rll@unx.dec.com>
X-Mts: smtp


Someone here is trying to bci a large file ( around 145MB ) into the
ptbqa pool. Here is the error:

$ bci -m"Initial submit" Shells_Utili_Regression.tar.Z

[ ./test/shells_utilities/su_regression/Shells_Utili_Regression.tar.Z ]
[ using default log message ]
        Initial submit
bci: The following rcs command failed:
ode_client -t2 6 ci -q -f -sExp -u1.1.1 -m      Initial submit
 Shells_Utili_Regression.tar.Z 
./test/shells_utilities/su_regression/Shells_Utili_Regression.tar.Z,v

Has anyone else had a problem submitting a very large file ( > 100MB )?


Rich
================================================================
Rich Larsen, M/S: UNX			TCP/IP: rll@unx.dec.com
USSG/User Env. & Std. Group		DECnet: UNXA::LARSEN
Digital Equipment Corporation		FAX:	908-577-6003
200 Route 9 North			Voice:	908-577-6083	
Manalapan, New Jersey 07726		DTN:	462 


------- End of Forwarded Message