Conference decwet::advfs_support

1009.0. "What good is verify, issue #1" by NETRIX::"trewitt@pa.dec.com" (Glenn Trewitt) Sat Mar 01 1997 21:38

We had an AdvFS panic last night:
  panic (cpu 3): ADVFS INTERNAL ERROR: dealloc_bits_page: can't clear a bit
twice!
No other errors were logged by uerf.  Syslog reports:
  Feb 28 23:17:34 nsl-too vmunix: ADVFS INTERNAL ERROR: dealloc_bits_page:
can't clear a bit twice!
  Feb 28 23:17:35 nsl-too vmunix: ADVFS: dmnId = 32320fc4.0001c80d, vd = 1, pg
= 277, wd = 426, mask = 00000000
  Feb 28 23:17:35 nsl-too vmunix: ADVFS EXCEPTION
  Feb 28 23:17:35 nsl-too vmunix: Module = bs_sbm.c, Line = 292
  Feb 28 23:17:35 nsl-too vmunix: ADVFS INTERNAL ERROR: dealloc_bits_page:
can't clear a bit twice!
  Feb 28 23:17:35 nsl-too vmunix: 
  Feb 28 23:17:35 nsl-too vmunix: panic (cpu 3): ADVFS INTERNAL ERROR:
dealloc_bits_page: can't clear a bit twice!

The configuration is:
	4-CPU A/S 4100
	File Domains are all single-volume; the ones affected are HSZ40 RAIDsets.
	Digital UNIX 4.0B
	(Although the affected file domains were created under UNIX 3.2C.)

When I try to mount the fileset, I get:
	# mount -t advfs old_nsl_proj#nsl /mnt
	ADVFS : Domain old_nsl_proj not activated - inconsistency detected
	old_nsl_proj#nsl on /mnt: Device does not contain a valid ADVFS file system

When I try to run AdvFS verify, I get:
	# /sbin/advfs/verify old_nsl_proj
	ADVFS : Domain old_nsl_proj not activated - inconsistency detected
	verify: can't get set info for domain 'old_nsl_proj'
	verify: error = E_DOMAIN_NOT_ACTIVATED (-1067)
	+++ Domain verification +++

	ADVFS : Domain old_nsl_proj not activated - inconsistency detected
	main: unable to get info for domain 'old_nsl_proj'
	    error: -1067, E_DOMAIN_NOT_ACTIVATED (-1067)

Now, the manual page says that I can use "verify -F", which will force the
mount, without "running recovery" on the file domain.  It also says:
    Use this option with extreme caution and only as a
    last resort when you cannot mount a fileset.  The mount -d option mounts
an
    AdvFS fileset without running recovery on the file domain.  Mounting
    without running recovery WILL CAUSE your file domain to be inconsistent.

Now, I don't know about you, but it sounds to me like the cure is worse than
the disease!  Aren't there any tools to do the sort of low-level bit surgery
that appears to be called for in this case?

Side note: Based upon my experiences with verify, it seems to be quite lame
for dealing with *actual* AdvFS problems, so I'm not inclined to experiment
by forcing the mount.  My next note discusses the problems that I have
encountered using verify to deal with less-severe problems with a different
file domain.

Current status:
	Restoring from backups onto a separate partition (I love RZ29's!)
	The offending domain/partition is still "intact."

I would like some help here - this is the second time I've lost this 14GB file
domain due to AdvFS errors.

	Glenn Trewitt
	Network Systems Laboratory
	Palo Alto, California

[Posted by WWW Notes gateway]

    Based on the information you provided and note 982.1, it appears that
    you may have a problem other than an SBM corruption. Have you filed a
    QAR yet? Having a crash dump file to look at would help us, too. Have
    you looked in the error log to see if you had any I/O errors on the
    volume?
    
    >Aren't there any tools to do the sort of low-level bit surgery
    >that appears to be called for in this case?
    
    It's not clear that your problem can be fixed with some bit surgery and
    we don't have any tools yet that can do that kind of operation.
    
    >Side note: Based upon my experiences with verify, it seems to be quite 
    >lame for dealing with *actual* AdvFS problems,
    
    Although it may seem like it would be easy to fix some of the problems
    that people run into, it usually isn't so easy. Writing a program to
    diagnose problems and then fix them is even harder. As we gain more
    knowledge in these problems, we can upgrade verify to handle them. Of
    course, work on verify has to be prioritized with all the other AdvFS
    tasks, but it is getting some cycles.
    
    >Current status:
    >    Restoring from backups onto a separate partition (I love RZ29's!)
    >    The offending domain/partition is still "intact."

    >I would like some help here - this is the second time I've lost this 
    >14GB file domain due to AdvFS errors.
    
    I'll contact you offline. We could use some help, too, and having a
    domain to look at would be very useful.
    
    						Jan