[Search for users] [Overall Top Noters] [List of all Conferences] [Download this site]

Conference 7.286::digital

Title:The Digital way of working
Moderator:QUARK::LIONELON
Created:Fri Feb 14 1986
Last Modified:Fri Jun 06 1997
Last Successful Update:Fri Jun 06 1997
Number of topics:5321
Total number of notes:139771

1064.0. "hazardous working conditions" by SAUTER::SAUTER (John Sauter) Wed Mar 28 1990 11:52

    With reference to the side issue in 1057.24, which referred to working
    on a machine with power applied, for God's sake if you must work under
    such conditions have a companion you _trust_.  It doesn't much matter
    if your friend is a co-worker, customer, bystander, spouse, descendent,
    ancestor, or whatever.  If you can't have someone you trust to rescue
    you if you goof, then refuse the job.  The worst that can happen is
    you'll get dismissed.
        John Sauter
T.RTitleUserPersonal
Name
DateLines
1064.1ULTRA::PRIBORSKYAll things considered, I'd rather be rafting.Wed Mar 28 1990 15:3912
    Time out.
    
    I haven't seen the memo referred to, buy something is wrong here.
    Until I do, I'm operating on only half the story.
    
    If we (DEC) are building a machine in which we knowingly send even
    trained service personnel to work on a machine with power applied, 
    then the implementation of "hot" swap is taking on new meaning.  I'd
    suggest that someone from the health and safety group be involved.
    
    The only job in the world I know of where you MUST knowingly put your
    life on the line is the military during time of war.
1064.2Power utility journeymen work "hot" ...YUPPIE::COLEWish? Did somebody say "Wish"?Wed Mar 28 1990 16:346
	routinely, and at voltages a lot higher than 280!  With the right pro-
tection, it would be possible.  Whether or not it makes SENSE is yet another
question!

	I don't know squat about the 9000, but I get the feeling 1057.24 is
talking about an "uninteruptible power center(?) (UPC)"?
1064.3HOW ABOUT 80000 WATTS!!!!!!!MAMTS2::JFARLEYThu Mar 29 1990 01:3224
    HAZARDOUS WORKING CONDITIONS -Are kidding me? I really have mixed
    emotions about DEC taking on UPS's (uneruptible power source). I have
    been trained on both models HA1000 and HA3000,but I feel not enough 
    precautions are being taken by DEC. I do not want to rely on MR or
    MRS. customer to come my aid if the unforseen should happen. You have
    to work on these while they are HOT,full power,the juice turned on,I
    hope ypu get the message. They will kill you Period!! Even though
    DEC has provided good training and safety tools are available,there
    is something about making a honest boo boo and some engineer may lose
    his life over it. I said to my classmates while we were being trained
    on UPS's it is all well and good now but the first time one of us
    gets fried someone in the ivory tower will rethink the policy of
    DEC getting into livr high voltage. They also tell you that it is
    ok for Mr customer to be there while you work on these guys.They don't
    tell you if something happens and CPR is not started within 4mins
    YOU ARE DEAD FRED. Do you think for one moment I am going to trust
    Mr customer with my life,NO WAY JOSE.Somehow, someway someone is 
    going to get seriously hurt and then the _-_-_ will hit the fan. I
    don't portend to be DOOM and GLOOM but I still believe there are not
    enough safety procedures in place yet for a Limited Few to lose their
    lives over.
    procedures
    
    
1064.4Lots to be thought through...LIOVAX::CRAPAROTTAPhysical T5-Virtual T7Thu Mar 29 1990 12:1332
    I agree with .3 100%... What we (Digital) have to sit back and think
    about is, WHY should the CUSTOMER supply someone? Also I don't want
    to go thru the training of a person every time I go on site and
    expect them to cover my BUTT. I DON'T want to be looking over my
    shoulder to make sure they're there when I'm troubleshooting the
    UPC. Since this thing is so well engineered, we will for the most
    part be going in late at night ( we don't expect them to bring down
    a $2mill system during the day do we??) and it will for the most
    part have more then 1 problem.... We had a bug in lab, where it
    DID take TWO ENGINEERS to EFFECTIVELY troubleshoot and REPAIR. This
    is NOT something out of the ordinary for the UPC.. 
    
    On the other note of CPR.. The memo STATED, that if we didn't get
    CPR within 4-6 minutes, you or your friend are gone. PERIOD... I
    wonder if the person who wrote the MEMO ever heard of BIG CITIES
    and multiple floor buildings.. It will take the EMS people at least
    10-15 minutes to get to the site, let alone get up the elevator
    to the 40th floor!!! I have taken on the initiative to get CPR trained
    and PRAY that I will never have use it.... 
    
    If DIGITAL does not want to spend the $15-20 and hr. to send another
    trained tech out, then that $2 million computer will be down until
    they do....  Or until they get someone else who will work on it alone..
    
    Last but not least YES, electricians do work on higher voltages.
    That is also ALL they work on. I never professed to work on  HIGH
    VOLTAGE Power conditioners for a living. I DO work on computers
    though....
    
    Still Alive and Kicking..
    
    Joe
1064.5MOVIES::LESLIEAm I an _Engineer_ yet?Sat Mar 31 1990 06:2015
    This sounds very strange and presents a picture that I cannot believe is
    in any way complete.
    
    DIGITAL is a very concientous employer, groups such as CSSE (for whom I
    work) have amongst their aims the preservation of safety at all times.
    
    The above doesn't even begin to take federal/applicable safety
    regulations and laws worldwide that Digital is cognisant of and obeys
    with great happiness.
    
    Bottom Line: sounds like you have the wrong picture. I suggest you
    return to whomsoever painted this picture and ask them to complete it.
    
    Andy Leslie
    CSSE/VMS
1064.6The Picture was Painted .. and is COMPLETE..LIOVAX::CRAPAROTTAPhysical T5-Virtual T7Sat Mar 31 1990 20:3019
    Andy,
    
    I'm sorry you don't like the picture. I can only tell you that HAVING
    GOTTEN THE TRAINING, these are the FACTS!! I might've forgotten
    some of the things mentioned in class, but I did not by ANY means
    forget the saftey issue. I was the one who brought it up and I am
    the one who WILL NOT work on it.... UNLESS i have another DEC engineer
    with me. Preferably one who like ME, who WILL be CPR trained. If you
    CAN do anything about changing someones mind, let me know and I'll
    be glad to call you and continue this. My District manager is involved
    and I will be drafting a MEMO to Employee Health and Safety also...
    
    Joe
    
                                             
    ps: I NEVER said that Digital is not a concientous employer!! I
    LIKE working for DIGITAL,and just think that someone made a real
    STUPID decision. Which for the most part happens at times.. It's
    just that this one CAN cost me MY LIFE..... 
1064.7ESCROW::KILGOREWild BillSun Apr 01 1990 01:224
    For those of us who haven't touched much hardware lately, and have never
    worked with uninterruptible power (UPC? UPS?), could you provide a little
    detail t the technical problem? What are you working on that's hot, and
    why does it have to be hot?
1064.8needs more attention than it can get hereCVG::THOMPSONMy friends call me AlfredMon Apr 02 1990 00:267
    If I had a paper copy memo listing this kind of risk or had first
    hand training that indicated this risk I'd send a memo. I'd copy
    my VP, the VP responsible for the product, corporate Legal and KO.
    I'd also copy my boss and most of the people up the line. 
    
    		Alfred
    
1064.9SSDEVO::EGGERSAnybody can fly with an engine.Mon Apr 02 1990 18:433
    Re: .8
    
    ... and mail them, US Post Office, return receipt requested.
1064.10what they are/doBTOVT::CACCIA_Sthe REAL steveTue Apr 03 1990 14:1770

    I will try To answer Bill in .7 about what some of these power 
    products are what they do and why they have to be worked on "hot".
    Please don't expect major detail this is not the forum nor is there
    time/space to do it.

    PCS/PDS - Power Conditioning System/Power Distribution System

    Basically they are just large transformers with circuit breakers on the
    out put side of the main transformer. They take the input
    (city/utility) power and clean it up a little then pass it on to the
    parts of the installation that should be kept on dedicated power lines.
    Some of these PCS/PDS units have monitors built in to keep track of
    temperature, moisture, and security. These units in "Maintenance
    by-pass" have no power to the monitor electronics. but still will pass
    power thru to the load. When the power distribution part of the unit is
    to be worked on they are typically take off line.

    UPS - Uninteruptible Power Supply - Just what it sounds like.!!!!

    These units are designed to provide Uninteruptible power to a load.
    they all have some form of either battery back up or capacitive
    discharge network that will provide continuous power during a utility
    failure. This utility failure can be anything from a plain old black
    out to a drop in line voltage below a certain level ( the level varies
    with the load, input, and system type) The idea is to keep the system
    operational at least until an orderly shut down can be accomplished.
    The bigger units will keep things going until auxiliary power can be
    brought on line, usually a motor generator in the building somewhere.

    Why work on it "hot"? What is the sense of having several million bucks
    worth of computer doing billions of bucks worth of bank transfers if
    the first time NY Edison sees rain the entire works stops, cold, with
    disk head crashes, and tapes fouling and data corruption and loss?

    Again the high voltage input is usually separate from the DC 
    electronics.

    UPC - Utility Port Conditioner - A massive switching DC power supply.

    This takes the utility power and turns it into a very clean very stable
    high current DC output. This is a unit where AC line is mixed with DC
    logic and even if the input breaker is tripped power is still coming
    in. If the thing is unplugged DC power is still provided thru
    Capacitive discharge (ride thru) or battery back up. The unit must be
    worked on with power applied for most of the trouble shooting and
    fault isolation. 

    None of these machines are inherently unsafe, BUT, the conditions they
    must be worked on does add a significant level of risk to the engineer. 
    We have tried to reduce this risk by supplying training, and special
    tools. I do realize that most of the engineers currently in the field
    have a mind set of ones and zeros. 5 volts, 12 volts, 24 volts. Little
    spark but big deal nothing happens. Fix the power supply by swapping it
    out. That is where the real risk comes in. Now he is working with full
    AC line at upto 440 volts in the US and upto 100 amps input power. He
    is not used to dealing with this and is either afraid of it, or unaware
    of the differences. Joe in 1057.24 is one of the former but in his case
    he recognizes it and is trying to do something positive. when he went
    through t training he was told he could go to his manager and ask not
    to be sent out on this product alone, and he was given phone numbers of
    people who would give him support in that direction. Maybe with a
    little experience Joe will gain confidence, but even if he doesn't, I
    respect his views on safety and will never suggest that a person work on
    any power product totally alone. It may sound dumb but even if it is
    only a janitor that pulls a plug and dials a phone that does meet
    minimum (just barely) requirements. 

      
1064.11policy should require certain skills SAUTER::SAUTERJohn SauterTue Apr 03 1990 20:4716
    re: .10---``It may sound dumb but even if it is
    only a janitor that pulls a plug and dials a phone that does meet
    minimum (just barely) requirements.''
    
    I have known janitors that I would trust to pull the plug and dial
    the phone, and I have known janitors that I would not trust.  I don't
    think the plug-puller's job is what's important, I think it's his
    ability to do what is necessary in an emergency.  In other words, 
    I would feel better about a policy that required a certain amount
    of training in the second person, rather than a specific job title
    (unless the job title implies the training, as Janitor doesn't.)
    
    How about this: require that a person who is assigned to work on
    such equipment be accompanied by someone who is certified by the
    Red Cross (or equivalent organization outside the USA) in CPR.
        John Sauter
1064.12slowly things changeBTOVT::CACCIA_Sthe REAL steveWed Apr 04 1990 20:2721
    RE-.11

    John, The original intention on the part of the Customer Services
    Product Safety Engineer who has been working this issue for the last
    2 1/2 or 3 years was to start a whole new FS job code. The new job
    would have REQUIRED minimum background experience as a licensed
    electrician and also would have required 2, count them, 2 product
    specific as well as safety trained and CPR/first aid certified persons
    on site at any given time. This new job was envisioned as being a power
    products specialist who would deal exclusively with site power and
    ground and the PCS/UPS/UPC/motor generator type products.

    I don't like it but I had to settle for the best I could get after
    being laughed at and called several shades of spendthrift fool. At
    least we do now require product specific training and a second person
    on site. Electrical safety and lockout/tagout are now prerequisite for 
    all power courses. Managers are being made to take the training and so
    are several job codes in manufacturing, logistics and training. There 
    are rumors afoot that CPR may become mandatory for certain job codes in
    the not too distant future. It takes time but things will come around. 
1064.13Right Person for the Right Job...KYOA::CRAPAROTTAPhysical T5-Virtual T7Wed Apr 04 1990 21:3218
    I think, having a CPR person working with you SHOULD be a requirement!
    I would hate  to have the janitor pull the plug and get me off the UPC 
    and then wait for EMS people to get to me. At that point the person is
    long gone... As I stated before I am going to take CPR for this reason
    and won't mind being the SECOND person ..:-(...  I just don't want to
    have my hand in this box and have to be looking over my shoulder to see
    if my backup (ie- the CUSTOMER) is still behind me.. Am I scared to
    work on this box?? YOU BET!!! NOT because I don't feel that my training
    was adequate, on the contrary, the instructor was excellent!! It's
    probably because I don't feel comfortable working on it due to my
    SKILLS being in computers and not power. I think the WE (DIGITAL)
    should take a long and hard look at this procedure. It just smells like
    a cop out to me. Whatever happened to the right people for the right
    job??
    
    
    Joe
     
1064.14NSSG::ROSENBAUMWed Apr 04 1990 23:275
    
    
      Of course with a UPS, even pulling the plug doesn't buy you much...
    
    
1064.15looks hopefulSAUTER::SAUTERJohn SauterThu Apr 05 1990 12:5418
    re: .12, .13
    
    It sounds like the people directly involved understand the issue
    and are doing the right thing.  The author of .12 appears to be
    under pressure by his management to avoid spending money on additional
    safety, and the author of .13 is refusing to work under what he
    perceives as unsafe working conditions.  If all of the workers
    follow the example of .13, the author of .12 will have the ammunition
    he needs to convince his managment chain to spend the additional money.
    
    The problem is really a political one.  The technical issues are
    understood by all parties.  The solution, therefore, must be political:
    refuse to work under unsafe conditions, support your direct reports
    who refuse to work under unsafe conditions, and stick to your
    convictions until the guardians of the purse strings correct the
    lack of safety.  As .12 said, it takes time but these things will
    come around.  I would like to add that it also takes persistence.
        John Sauter 
1064.16enter joe's header hereBTOVT::CACCIA_Sthe REAL steveThu Apr 05 1990 12:5915

    RE-.13 Joe's header -< the right person for the job>-

    A printer specialist is not a system tech.
    A gas pump attendant is not a diesel mechanic.
    A digital technician (bit chaser) is not a utility line man.

    This says that Ed Services presents the best information we have in the
    best manner possible in the time allowed. Then pray that what has been
    presented is helpful to those who need it. We also pass on all the
    inputs from the end of course critique on to our managers, and through
    them or even with them, on to any other groups as required. Sometimes
    we are even listened to.

1064.17are customers aware of the risk??ODIXIE::SILVERSGun Control: Hitting what you aim forThu Apr 05 1990 16:578
    I don't know about y'all, but having been in the field and on customer
    sites for quite awhile (I'm a software resident), I know of many
    customers who would refuse to purchase 9000's or worse yet, send them
    back if they were aware that we require a customer body to be onsite
    to pull our engineers 'out of the UPC, and bring them back to life'.
    
    Do we inform customers that this is the way these machines are
    serviced?
1064.18Help keep your boss out of prison.STRATA::RUDMANAlways the Black Knight.Fri Apr 06 1990 17:2619
                  "Safety is Everyone's Responsibility"
                  
    I'm sure the boss doesn't expect you to work under conditions you
    aren't qualified for which may result in your injury (or worse).
    After all, that would mean the loss of an FSE, strained customer
    relations, time lost, DEC money lost due to the OSHA fine (and
    they *will* find cause to fine DEC, make no mistake about that)
    and a possible court settlement, unwanted media coverage (contrary
    to Douglas Kenney, co-author of ANIMAL HOUSE, who said, "The 
    only bad publicity is your obituary.") and, last but not least, if 
    the accident report reveals your boss was aware of the danger and 
    sent you in anyway, *he* can be liable for a large fine, 
    imprisonment, or both.
    
    So keep pushing until your Safety group takes a position on the
    issue.  *Before* someone is hurt.

     						Don
                          
1064.19BOLT::MINOWGregor Samsa, please wake upSat Apr 07 1990 14:1917
A few observations:

-- Somewhere in the P&P, I'm pretty sure Digital says that it will not
   ask you to do any work you consider dangerous.

-- In many jurisdictions, only "licensed electricians" (i.e. licensed
   by the municipality/state) are permitted to work on "live" circuits.
   In some jurisdictions, specifically Sweden, an electrician is not
   permitted to work on powered circuits alone.  I would doubt that
   a "janitor" or other person who is not trained on power circuits
   would be acceptable in that circumstance.

-- Although I have been trained in CPR, I would not consider myself
   competent to assist someone working on a live powered circuit:
   I am not trained in all of the other safety/rescue issues.

Martin.
1064.20COMET::MONTGOMERYThe California RAIDERSMon Apr 09 1990 18:4110
The levels of current on some of the systems is fairly high... You won't
have to worry about someone pulling you off the live line, a shovel will
work just fine with a small bag.....


Monty who is an EMT and as seen first hand what high current does to someone!!




1064.21appoloy/clarificationBTOVT::CACCIA_Sthe REAL steveTue Apr 10 1990 13:1754
    
    
It seems that I may have been somewhat misleading in my response to previous 
notes. Please allow me to clarify if I may, and to apologize to anyone that 
was caused distress. 

The Uninteruptible Power Supplies are intended to provide power to a load 
either long enough to perform an orderly system shut down or, until a source of 
primary power is returned. There are maintenance by-pass modes in the UPS 
which does allow for the system to be powered on but areas of the UPS being 
serviced are unpowered. The "hot" portions of the UPS are isolated from the 
service person by screens, covers, and labels. The batteries in a UPS are 
isolated by being placed in separate Cabinets or in enclosed sections of the 
UPS cabinet itself. The Battery back up circuits and the batteries can be 
isolated by circuit breakers and disconnecting the cables.

A Utility Port Conditioner is a massive switching DC power supply with 
capacitive discharge ride thru. This takes the utility power and turns it into 
a very clean, very stable high current DC output. In the event of utility 
power problems where AC line actually goes away or falls below a certain level 
a discharge path is created that will allow for orderly system shut down. 
After having read over the note, and having had it pointed out to me, I 
realize now that it was misleading. AC line is not mixed with DC logic. on the 
one power path, like any other power supply it is used after a transformer to 
create 24 volts DC andor in this case, after attenuation, certain phase monitor 
voltages typically between 50 and 70 volts AC. The other power power path is 
through the input circuit breaker to the pulse width modulators to create the 
+/- DC for the load. If the input breaker is tripped, power is still coming in,
but only to the 24 volt power supply, line monitor circuit, and to the line 
side of the circuit breaker. Once the line cord unplugged there is no power in 
the unit. The LEDs will give indication of the general location of many of the 
faults that may occur, how ever, like any other computer product, the unit must
be worked on with power applied for much of the trouble shooting and fault 
isolation. Any part changes, be they fuses, modules, or FRU are made with 
the unit powered down, unplugged, and the facility circuit breaker locked out 
and tagged out. 

NEVER IS THERE ANY HOT SWAP AUTHORIZED, ADVOCATED, OR SUGGESTED. 

All engineers who are selected to work on any power products are now required 
to take and pass the basic electrical safety and the lockout/tagout course. 
These courses were not developed specifically for the UPC, but are used for all 
power products. The emphasis on safety is intended not to frighten people, 
but to make them aware of the difference between a typical computer product 
and the new generation of high power, high efficiency, multiply redundant 
products that they will be facing. This is a new mind set for most field 
engineers who are used to the idea that one plug or one circuit breaker shuts 
down the entire system.

None of these power products are inherently unsafe. With proper training, use 
of the power products insulated tool kit, and appropriate safety precautions, 
they can all be worked on with a minimum of risk. 

    
1064.22It's still no enough..KYOA::CRAPAROTTAPhysical T5-Virtual T7Mon Apr 16 1990 03:025
    Diving is a great also, but I wouldn't want to do it with someone that is
    not familiar with all the saftey requirements or who isn't trained..
    
    Joe