Conference 7.286::digital

    I disagree that "End-user" products don't need to be as 
    bug-free as lower level layers.  After all, the application 
    that a customer uses from day to day is what (s)he perceives
    as "us".  If the application that a user is working on blows
    up in his face, it really doesn't matter much whether that was
    caused by the application itself, or by a lower layer.  The perception
    will be that our software is not of good quality, and the reputation
    that we have all worked for starts to crumble...
    
    On the other hand, I do agree that the cycle time should be 
    shortened on smaller projects IF that can be done without any
    sacrifice of quality.
    
    And on another topic,
    
    > One example that really gets me going is a product called P/FM. 
    >			.
    >			.
    >			.
    > It's built on top of All-in-1, which                           
    > makes a nice "corporate strategy" but adds incredible overhead to
    > a product that doesn't need to interact with WPS, Mail, etc.
    
    Seems to me that a product that made it easy to produce analysis
    of phone usage costs, and then write memos around that information
    (WPS) and send it to the relevant managers (Mail) all from within
    one context (All-in-1) would be a fairly attractive to
    telecommunications managers...
    
    Kevin

    DEC STD. 28 is currently undergoing review. I suggest you go talk
    to your local CSSE person who should know someone through who you
    can input to that review.
    
    If need be, I'll be your conduit, but on the whole, I have a lot
    on my plate right now and would prefer you found someone more local.
    
    Andy

Re .0:

Frankly, I'm not convinced that in this specific instance the phase review
process is responsible for the problems encountered.  It may be getting
blamed for something that is the fault of poor program design (lousy ease
of use and evolvability, to use the official terms).

If this program has a requirement that its internal billing tables or
algorithms be updated much more frequently than a product distribution
cycle would allow, then it ought to have been designed to allow such
updates to be performed conveniently in the field. Ideally, users would be
able to do it themselves, but since I've no idea of the complexity of
telecommunications tariffs, I won't claim that is an attainable goal.

If the customer is incapable of reconfiguring their system, then Digital
should be pleased to provide a service which will do this for the customer
at a fair price.  Whether it means someone spending a week on-site
analyzing the impact of some billing change, or just shipping a new floppy
of some new tables every quarter to the customers, it shouldn't involve
making a whole new release of the software. We don't ship a new VMS release
every time a customer adds another VT100 to their system.  I don't see why
this program goes through a whole new product cycle every time the price of
a phone call between Peoria and Kalamazoo changes.


If you want to improve the phase review process, it sounds like you
might start by improving it so that in the future such a program would
never be allowed to passe phase 1 with such a fatal flaw, rather than
trying to speed up the process so you can ship the wrong thing even more
efficiently.  But since you've said that the process wasn't even used
during the initial development in the first place, you'll have to present
some evidence that the existing process wouldn't have caught this mistake
in the first place.

No doubt there are ways to improve the process.  However, focusing narrowly
on one inappropriate horror story doesn't make the point.

One improvement I can imagine is with the structure of the Component
Software Product Specification (functional spec).  It has always struck me
as rather unbalanced that 95% of the bulk of most functional specs is in
section 3 ("Software Capabilities"). This is because since section 3
contains the list of features.  It happens that specifying features and
functionality is much of what a functional spec is all about.  Now, I'm not
saying the other sections are unimportant. However, the existing division
of sections has a weird distribution.  This might be a symptom that
the structure is concealing some aspects of the product that should exposed.


As for the notion that end-user applications don't need the same
reliability that system software does . . .

It has been estimated in the past that it costs about $2000 to process a
(unique?) SPR *before* it is passed to the appropriate development group.
How many extra SPRs are you willing to generate over the lifetime of
a product in order to improve time-to-market?
				/AHM
P. S.  Does the fact that there hasn't been a note entered in the Software
Engineering conference (NANDI::SWENG, q.v.) for over a month say something
about The Way We Work At Digital?

    I was just on my way here to explain that Fred Goldstein had hidden
    the notes he had written and that the topic is being discussed by
    the moderators, but it seems the whole lot has been hidden.  I suppose
    since I said I didn't have time to help moderate I'll just butt
    out.
    
    - M

    I have requested that the participants in this topic consider if
    they can make their point(s) without getting product specific and
    thus upsetting fellow engineers.
    
    If, after reflection, they feel their reply should stand, it will
    be set /nohidden. Otherewise they can delete abd re-reply in a more
    general way.
    
    As .3 wasn't product specific, I've set that note (mine) /Nohidden.
    
    Full texts have been made available to the participants.
    
    Andy, one of the many Co-Moderators.

    I'm sure the same rationale was used to supress disucssion of the
    saftey factors concerned with O-ring seals in the Titan booster
    attached to the Challenger in cold weather.
    
    If we're so thin-skinned and defensive about products that failed
    in the marketplace,  then let's not upset fellow engineers.  Their
    feelings are more important than product quality and success in
    the marketplace.

I think before continuing it's important not to assume that just because Fred
(author of .0) feels that a project fails doesn't mean the rest of the DEC (or
the world) agrees with him.

However, there is a much higher level issue at hand and I think that is the
question of whether or not DEC should be in the applications business (I think
we should to a degree) and if so, do we need some other type of phase review
process to deliver product.

First of all I feel very strongly in the spirit of the process.  That is you
have no business building an application without knowing your market, goals,
etc (business plan and product requirements).  Similarly engineering shouldn't
forge ahead until they'ce clearly defined what they want to build (functional
spec) and how they want to build it (design specs).  The problem with 
applications as opposed to things like compilers, operating system, point 
products, etc is that they generally are filling a well understood need.

Applications (and I suppose what I really mean are applications in areas not
that well understood) tend to be aimed at a volatile market.  That is the
requirements tend to be in a constant state of flux.  From the time the product
requirements are agreed on by all, and the functional and design specs defined,
the requirements may have changed but the project may have built up too much
momentum to pull back resources and rethink it.  Furthermore, by now the market
window is getting closer as well and fear starts developing that if one goes
back and tries to redefine the requirements the window will slam shut.

I'm not saying is this is right or wrong, but I think that's the real issue
that Fred may have been trying to raise in his base note.

comments?

-mark

    Yes, Mark, that's exactly the point I was trying to raise!
    
    The product in question (hidden at my request because some of the
    engineers got very upset at having _their_ product mentioned) is
    aimed at such a volatile market, and our competitors in the space
    (including several ISVs listed in SOFTbase [tm]) use different,
    faster techiques.  I worked with one of them about 7 years ago,
    as a customer; he revised the program monthly to my whims!
    
    Perhaps we need some kind of review process like this:
    
    	a) Is there a market?
    	b) Are we the people to be there, or should oems or ISVs own
    		it?
    	c) What is the nature of the competition?  What do customers
    		expect?
    	d) If we go ahead, what process can best be used?
    
    If it's a volatile market, we may want to stick with Rapid Prototyping
    instead of Phase Review for version upgrades.  Or some other techique.
    Just not full phase review for every version.
          fred

Re .14:

Those questions you list look pretty much like the ones which must be
addressed in a Business Plan, at least the one in an old Software
Development Policies and Procedures manual I just borrowed. Did you check
to see whether the current Phase Review process has the features you desire
before making suggestions on what to add to it?
				/AHM/THX

    Also re .14 -- Those are precisely the questions which are meant
    to be asked at a Phase 0 review. Phase 0 exit criteria are supposed
    to be answers to those questions. The intent of Phase 0 is to answer
    the question, what is this product being proposed and should Digital
    build it?
    
    Richard.

I get a feeling that some people think that the Phase Review Process can 
impede rapid development of products.  Also that it makes making timely 
updates difficult.

I would suggest that the problem is the misapplication of the Phase Review 
Process.  The process is intended to assist in building products that meet 
their goals - and rapid development can validly be one of these goals.  If 
you really do think the process is causing a problem, then don't throw it 
out - figure out if it really is a problem and if so, at least start action 
to fix it!

jb

    The Phase Review Process can be used to speed up any project, even
    the shortest, most time-critical ones. The phase review process
    really only says three things:
    
    	o Write down carefully your plans
    	o Review them with the right people in a timely fashion
    	o Do the thinking process in the following phases and don't
    		slide back without repeating the writing/reviewing:
    		0. What is the problem?
    		1. How is it to be solved?
    		2. Solve it.
    		3. Verify that you solved it.
    		4. Ship it.
    		5. Stop shipping it.
    
    There a some specific levels of review required. But these tend
    to reflect the magnitude of financial resources required. For example,
    major investments require review at the Board of Directors to
    transition key phases. This is because they involve committing millions
    of dollars. But smaller, less impact investments can be reviewed
    at lower levels.

    re: .14
    
>    If it's a volatile market, we may want to stick with Rapid Prototyping
>    instead of Phase Review for version upgrades.  Or some other techique. 
 
    What is "Rapid Prototyping"?  
    
    I'm familiar with the Phase Review process and Project Life Cycle. What
    other techniques are used within DEC and where can I find out about
    them?
    
    -joet 

    Briefly, rapid prototyping works cyclically between the requirements,
    design, code, and test portions of the lifecycle.  (Depending on
    how this is done, it can range from a word to legitimize hacking
    to a well-controlled design technique.)  Question and potential
    rathole: how does this differ from "baselevel design"?
    
    I don't know of a conference dealing with this although the
    NANDI::SWENG conference is as good a place as any to ask for more
    details.  I'm going to move my reply there.

    							Ralph

    Perhaps I didn't make my earlier reply clear enough.
    
    The stages which are now part of Phase 0 are correct, but I was
    referring to BEFORE A PROJECT STARTS.  Where I don't agree with
    "always use Phase Review" is _for incremental upgrades_, like going
    from V3.3 to V3.4 (of an application, NOT of "system software").
    
    Right now, if someone wants to change a report so that it's only
    72 colums wide instead of 73 because some customers have a printer
    width problem, it usually takes a full phase review, and gets into
    the Phase 0 for the version which will typically be released in
    a year or two.  Each version contains many changes, each of which
    was fully scrutinized, engineered, etc.  Sometimes that's too long.

    [The application whose mention led to the hiding of earlier notes is one
    that must deal with changes that we have no control over, outside
    world things.  Since these changes occur rapidly and with little
    warning, and are unpredictable, they're hard to engineer for using
    normal processes.  Many customer applications work the same way
    -- what happens to an accounting system when a tax ruling screws
    up the way you've been keeping the books?  Pick your favorite example.]
    
    I'm suggesting that some changes should be made more quickly.  Not
    whimsically or sloppily, but independent of the full process.
    Sometimes time to market is too important to go the usual route.
          fred

    I think the question raised in .17 is the key to this whole topic
    and should not be passed over lightly.  I don't really think the
    problem here is choosing a specific development model.  The ability
    to accomodate rapid change is a functional requirement of many systems,
    especially those that must deal with externally-defined rules.
    
    If you leave this "function" out of your design, it doesn't matter
    if you use phase review, rapid protoyping or creative guesswork,
    you're not going to have a successful system.

    re .21:
    
    I believe that even point releases should use the phase review process.
    Consider, for example, the case of a payroll program. Last October,
    the US Congress changed the withholding algorithm, to be effective
    Jan 1. A proper use of the Phase Review process might have been:
    
    	Phase 0 -- "We will update the withholding calculation program
    to introduce a third tax table, and we will revise all the tables.
    This will be shipped to be in customers hands prior to Dec 15 with
    instructions to apply the change before payroll is run the first
    paycheck in January." Quick meeting between engineering, documentation,
    SDC planner, and product management.
    
    	Phase 1 -- "To achieve this, we must have the new code complete
    and integrated by Nov 15; QA test done by Nov 25; SDC commits to
    a quick turnaround so shipments are in the mail by Dec 7." Detail
    meeting to make sure all groups can implement this plan.

    	Phase 2 -- On Nov 15, "Here is the kit to test."
    
    	Phase 3 -- On Nov 25, "Tests succeeded; release the kit."
    
    This example was essentially a true story.

    I've seen some potentially good products go down in flames because
    they did not follow the process or skipped a step.  I haven't
    personally seen it, but I would bet that one or two suffered from
    blindly following the process without thinking about what they really
    needed to do.  It is the right thing to do for our "mainline" revenue
    products.
    
    Now, that having beem said, doesn't Software Services play a role
    in helping us address the one-time or limited audience or "needed
    yesterday" projects?
    
    						Mike

    I've also seen groups that are good at meeting the formal 
    requirements of the phase review process because they've spent
    money on product management, not engineering.
    
    The result?  Vaporware.  
    
    I've seen more than one phase review meeting BEGIN with the 
    product manager saying "I expect there will be no issues raised 
    here that will prevent me from closing phase x," when there are
    tons of obvious problems that everyone in attendence knows 
    about that weren't resolved during that phase.  To complicate things, 
    all the parties are ALREADY being measured on how fast they 
    deliver the product, so they're hardly motivated to suggest 
    changes in strategy, question the validity of ROI estimates, 
    or challenge performance projections.  Even in the worst cases, 
    these meetings end with "Phase xx is closed with the proviso
    that ....", leaving some part undone.  
    
    For some, the process has become a hollow exercise, which 
    legitimizes a project in the eyes of DEC, and for that reason
    alone should be followed.  In these hands the phase review
    process becomes political, fraudulent, and costly to DEC.
    
    
    MATT

re:.23
    
>    Phase 1 -- "To achieve this, we must have the new code complete
>    and integrated by Nov 15; QA test done by Nov 25; SDC commits to
>    a quick turnaround so shipments are in the mail by Dec 7." Detail
>    meeting to make sure all groups can implement this plan.

	That's well and good, but it doesn't fit in too well with the
    "standard" cycle.  Most products go through a version every _n_
    months.  So if it's, say, January, we may be doing Phase 1 for version
    2.3, Phase 2 for version 2.2 and shipping Version 2.1.  V2.3 is
    not due to ship until, say, NEXT March.  How do you fit in a point
    change?  You're getting out of sequence.
    
    	I'm sure it's possible, but the standard methodology doesn't
    document how or when.  It pretty much requires an emergency that
    would grind the whole program to a halt if not met.  So "ordinary"
    changes that customers want, but which aren't critical for everyone,
    must wait.  This is what gives our more flexible competitors a huge
    advantage. The fact that quick turn around is occasionally possible does
    not mean that our methodology is suited to markets that expect it
    all the time.  Our products may be "better" but the customers won't
    care!

    The procedures by which products are built and enhanced depends
    much more on the policies of the organization that is doing the
    work than on the phase review process.  
    
    In the case you site, the organization could plan time for adding
    important features that aren't known until the last minute.  There
    would be time in the schedule for reviewing suggestions for last-minute
    features, implementing the most important of them, testing them, and
    documenting them in the release notes (since manual changes need a long
    lead time). 
    
    If this activity is planned for and scheduled in phase 1 it does not
    require an emergency to get something important into the product at the
    last minute.
        John Sauter 

    
    Anyone who believes they can manufacture software without using
    a quality assurance process as an aid to maintaining a high level
    of quality output is either STUPID or UNPROFFESIONAL (or both).
    
    It seems to me that this discussion would be more productive if
    it tried to address the problems rather than trying to blame the
    process.                   
    
    i.e.                       
    
    If people are abusing the process how can it be prevented ?
    
    If the process isn't flexible enough, how might it be enhanced ?
    
    If quick enhancements are going to be required, how can we plan
    for them ?
                                                              
    Kim

    re:-.1
>    If quick enhancements are going to be required, how can we plan
>    for them ?

    That's the crux of the matter.  Phase 1 closes out many moons before
    the SDC ships the product.  How can changes be made that were simply
    not anticipated, or possible to anticipate, quicker than that?
    
    Somewhere there should be a provision for what I think used to be
    done on RT-11 and other early OSs.  There was version 4.0a, 4.0b,
    etc., each of which represented a patch level.  The Autopatch kit
    brought your SDC kit up to rev.  I haven't seen this done in any
    VMS products.  Might that type of procedure help?

    The key is good MANAGEMENT. Bad management can allow the illusion
    of control and planning without the reality. Bad management can
    cause everyone to hide the obvious. Bad management can cause quality
    or market-responsiveness to be sacrificed on the altar of schedule.
    Or schedule to be sacrificed on the altar of functionality.
    NO PROCESS CAN PREVENT MANAGEMENT FAILURE.
    
    A good process can be a valuable tool to aid in good management.
    In my experience, the Phased Development Process is a good process.
    Phase Reviews are a useful tool to measure progress under the phased
    development process.
    
    Note that each and every professional is a manager--of at least
    his own efforts.
    
    The version numbering standard implies clearly that point (minor)
    releases should not be labelled by the planning process. Rather,
    the numbers are assigned sequentially at time of release. If you
    need to introduce an extra release, do so and give it the next number
    in sequence. Failing to do this on the excuse of no numbers available
    is simply bad management--copping out of a decision.
    
    Patches are no faster than resubmitting to the SDC. Every mechanism
    used to distribute patches takes about the same length of time as
    a well-managed SDC release cycle. Besides, you can always transmit
    the changes in parallel with the SDC. But submitting ensures that
    future customers will get the latest version without fail.
    
    Printed documentation can be arranged to coincide with the media
    in the SDC. I can get a small manual printed in two days. It usually
    takes longer to copy the media.
    
    Most of the SDC horror stories can be traced to sloppy submissions.
    There have been times that the SDC got behind. But even then, there
    were controlled mechanisms that got fast turnaround reliably. These
    times of SDC failure have been far outweighed by the frequency of
    sloppy submission, i.e., bad engineering management.

Now that I've set .4 unhidden, because I still believe in the principles
it espouses, I should note some contingencies:

I have never used or previously heard of the specific product that was
under discussion.  And I can't think of any motive I might have for bashing
it.  While I can't rule out the effects of my imagination, most specifics
of .4 are predicated on claims in preceding notes, particularly .0.  So,
while readers should feel free to correct my grasp of the facts, they
should be aware of how I came to hold them.  In any case, .4 may be
read as a general statement about software engineering, regardless of the
facts in this specific case.
				/AHM/THX