Conference decwet::advfs_support

>            To create so many file which we don,t forecast
>              (for example news server) 
>            
>            1.What option we use if we want to create unlimited file number
>              under advfs.
    
    At the current time there is no 100% way to insure that the BMT will
    not become completely fragmented.  There are plans being made for a
    future release to avoid BMT fragmentation.
    
    In the mean time, you could do several things to minimize the problem. 
    
    If you have an existing domain which has usage which is typical of the
    files that will be placed on a new domain, then get the size of the BMT
    and the number of files for the existing domain and do a curde
    calculation of how much BMT space is used per file.  Then when creating
    the new domain, figure out how many files you expect and create the
    domain with the approprate sized -p value.  This gets a little more
    complicated when there are multiple volumes in the domain, since there
    is a BMT per volume and you would need to divide up the BMT allocation
    so it was spread out over all the volumes of the domain.
    
    But having multiple volumes in the domain has its good points as well. 
    For one thing, you can add in additional volumes as your needs increase
    and each new BMT can have its own -p initialization that you could
    sized based on looking at the size of the BMT on the other volumes of
    the domain.
    
    Also with multiple volumes, you can remove volumes with fragmented BMTs
    and then add them back in with a -p value that specifies their prior
    size as a starting point.  It is a way to defragment the BMT (curde,
    but effective).
    
    Defragment your domains on a regular basis.  This will help insure that
    the next time the BMT needs to be expanded, it will be able to find
    large contiguous extents, instead of being forced to get 2048 pages in
    1 page extends.  If you look at your extent map, you will see lots of
    extents that are only 16 (512 byte blocks) long.  This is because the
    domain was so fragmented, that the BMT was unable to get contiguous
    space to grow itself with.
    
>            2.Let me know  the option -x ,-p and
>              the relation of  number of file ,BMT Extent size, BMT Size.
    
    The amount of BMT space used per file is a variable depending on the
    file.  A large highly fragmented file will require more BMT space.  A
    small file less.
    
    The best guide is to take an existing domain which typical file usage
    that you wish to model.  Then divide the number of files into the size
    of the BMT to get an average bytes of BMT/file value.  If there are
    multiple volumes in the domain, then you need to get the sizes of each
    BMT on each volume.
    
    To find the size of each volume's BMT, you need to look at .tags/M-6,
    M-12, M-18, M-24, etc...  Use the showfdmn command to get the lists of
    volumes and their respective volume numbers.  Multiple the volume
    number by 6 to get the M-xx value.
    
    Also remember that the -x and -p options are in 8192 byte pages.
    
    					Bob Harris

    
    Thanks for reply
    
    I don,t understand exactly. so i have some questions
    
    According to  note 1069.1, 
    Get the size of the BMT and the number of files for the existing domain
    and do a curde calculation of how much BMT space is used per file.
    
    Then when creating the new domain, figure out how many files you expect
    and  create the domain with the appropriate sized -p value.
    
    (The number of file is 819000)
    
    1. How to find BMT size? 
    2. How to calculation  how much BMT space is used per file?
    3. How to create new domain if the number of file is about 819000000?
    
    4.Can you explain above statement using extentmap of note 1069.0
    
    As following  man page of mkfdmn, if we use -x 2048, the BMT size is
    57,600.
     
    I think if a BMT overflow the value of BMT size(57,600), we don,t
    create any file.  
    Because pageCnt reached the BMT size(57600)
    (47107 * 8192=385900544)<---- extent size
    (57600 * 8192=471859200)<---- BMT size 
    5.What is the maximum value of BMT size ?  can we increase value of BMT,
      BMT extent size over 2048,57600 ?
     
    6.If advfs  has restricted, how about change to UFS filesystem? 
           
    

>    I don,t understand exactly. so i have some questions
>    
>    According to  note 1069.1, 
>>    Get the size of the BMT and the number of files for the existing domain
>>    and do a curde calculation of how much BMT space is used per file.
>>    
>>    Then when creating the new domain, figure out how many files you expect
>>    and  create the domain with the appropriate sized -p value.
>    
>    (The number of file is 819000)
>    
>    1. How to find BMT size? 
    
    # cd /fileset_mount_point/.tags
    #  showfdmn domain_name
    # ls -l M-6		# BMT for vol 1
    # ls -l M-12	# BMT for vol 2
    # ls -l M-18	# BMT for vol 3
    etc... (vol # times 6 to get M-xx value)
    
    add up all the BMT sizes
    
>    2. How to calculation  how much BMT space is used per file?
    
    # showfsets domain_name | grep "Files"
    
    add up all the "Files  : xx" values from the showfsets display for each
    fileset"
    
    Divide the calculated size of all the BMTs by the total number of files
    in the domain from the showfsets command.  The resulting value is the
    avarage number of bytes of BMT used per file.
    
>    3. How to create new domain if the number of file is about 819000000?

    BMT_needed = number_of_desired_files * BMT_bytes_used_per_file
    
    BMT_pages_needed = BMT_needed / 8192
    
>    4.Can you explain above statement using extentmap of note 1069.0

    While an extent map can be used to calculate the size of the BMT, it is
    much easier to just use the 'ls' command as shown above.  Besides the
    example in 1069.0 was not a full extent map so I would not be able to
    calculate the BMT file's size.  Also I do not know if there are
    multiple volumes in the domain, so I don't know if there are other BMT
    files to look at.
    
>    As following  man page of mkfdmn, if we use -x 2048, the BMT size is
>    57,600.
    
    -x specifies how many pages the BMT should be expanded by each time it
    runs out of space.  If -x number of contiguous pages are available,
    then only 1 extent will be allocated for the BMT to map the -x pages. 
    If free space if highly fragmented, then as many extends as are needed
    to map the -x pages will be used.  Running the defragment utility
    frequently will help, since it will allow for a greater possibility
    that a -x sized free extent will be found.
    
    The -p option will pre-allocate the BMT to whatever size you desire. 
    -x will just be used to extend the BMT if the initial allocation runs
    out.
    
    Now if you over allocate the BMT, then you could be wasting space.  So
    a good first guess of the -p value is one that is slightly smaller then
    the ultimate size and allow the BMT to be extended a few times.  The
    goal is to avoid extending the BMT so that it runs out of extent map
    entries, that is all.
  
>    I think if a BMT overflow the value of BMT size(57,600), we don,t
>    create any file.  
>    Because pageCnt reached the BMT size(57600)
>    (47107 * 8192=385900544)<---- extent size
>    (57600 * 8192=471859200)<---- BMT size 
>    5.What is the maximum value of BMT size ?  can we increase value of BMT,
>      BMT extent size over 2048,57600 ?

    There is no maximum size for the BMT.  The limit is how many extents it
    is allowed to have.  Only the BMT is limited in the number of extents
    it is allowed to have.  All other files can have as many extends as it
    takes to map the file.  The BMT is special in that all the Mapping
    pointers have to fit in the very first page of the BMT (called BMT0)
    and when the first page is full, the BMT can not be extended any
    further.
    
>    6.If advfs  has restricted, how about change to UFS filesystem?    

    Your choice of a file system is your decision.
    
    Again the restriction on AdvFS is that the BMT can become too
    fragmented such that it can no longer be extended.
    
    There are work arounds for this problem.  In addition, the AdvFS
    development team is working designs that would help avoid fragmenting
    the BMT for a future release of Digital UNIX.
    
    One of the work arounds is to pre-allocate BMT space via -p.
    
    Another work around is to run the defragment utility frequently and
    have a large -x value so that when the BMT needs to be extended there
    is a good chance that it will get a large contiguous allocation of free
    space.
    
    Another work around is to use addvol and rmvol to migrate volumes into
    and out of the domain.  As each volume is added, approprate -p and -x
    values can be specified so that as files are migrated to the new volume
    its BMT will not be fragmented.  And an old volume with a highly
    fragmented BMT that is removed can then be added back in with
    approprate -p and -x values.  Thus, it is possible to clean up the BMT
    fragmentation without shutting down the domain or stopping operations.
    
    Finally, vdump and vrestore can be used to defragment the BMT. 
    Although I much prefer the above addvol/rmvol approach.
    
    ----
    
    The next reply is a quick and dirty shell script that makes an estimate
    of how large the -p values should be, given an example domain that has
    files typical of what the target domain will contain.  It is very
    crude and can easily over estimate the size of -p
    
    					Bob Harris

#!/bin/sh
# estimate_BMT_size.sh - this script attempts to use an example domain to
#	estimate the number of BMT pages to pre-allocate so that a domain
#	will be able to support a specified nubmer of files.  The script
#	assumes that the example domain is a good representation of files
#	and file usage on the target domain.  It also assumes that most of
#	the existing BMT space is being used and that there is not a lot of
#	free space in the BMT (all free space is assumed to be used by files
#	so lots of free space will cause the estimate to be higher than
#	necessary).
#
usage() {
    echo 1>&2 "Usage:  estimate_BMT_size  example_domain  [desired_number_files]"
    exit 1
}
#
#  Fetch a list of volume numbers in the domain.
#
get_vol_list() {
    /sbin/showfdmn $1 | /usr/bin/awk '
	NF == 8 {
	    if ( length($1) <= 3 ) {
		sub("L"," ",$1)
		if ( $1 < 256 )
		    print $1
	    }
	}
    '
    return 0
}
#
#  Fetch to total number of files in the domain.
#
get_file_count() {
    /sbin/showfsets $1 | awk '
	BEGIN { count=0 }
	/Files[ ]*:.*SLim=.*HLim=/ {
	    sub(",","",$3);
	    count += $3;
	}
	END { print count }
    '
    return 0
}
#
#  Fetch the total number of extents from the file.
#
fragments() {
    /usr/sbin/showfile -x $1 | awk '
	/extentCnt:/ { print $2 }
    '
}
#
#  Check the input arguments.
#
case $# in
    1) dmn=$1; est_files=0;;
    2) dmn=$1; est_files=$2;;
    *) usage;;
esac
#
#
# Find at least 1 fileset in this domain which is currently mounted.
#
mount_point=`/sbin/mount | /usr/bin/awk -v domain="${1}#" '
    index($1,domain) == 1 { print $3; exit }
'`
if [ ! -d $mount_point/.tags ]
then
    echo 1>&2 "Domain $domain does not appear to have any file sets mounted"
    usage
fi
cd $mount_point/.tags
#
#  Calculate the number of bytes used by all the BMTs in the domain (each
#  volume in the domain has its own BMT).
#
BMT_size=0
vols=`get_vol_list $dmn`
for j in $vols
do
    n=`expr $j * 6`
    blocks=`/usr/bin/du M-$n`
    set -- $blocks
    size=`expr $1 \* 512`
    BMT_size=`expr $BMT_size + \( $1 \* 512 \)`
    frags=`fragments M-$n`
    echo "Vol $j BMT size in bytes:        $size  BMT extents: $frags"
done
#
#  Display the current BMT stats.
#
files=`get_file_count $dmn`
BMT_bytes_per_file=`expr $BMT_size / $files`
#
echo "                                `echo $BMT_size|sed 's/./-/g'`"
echo "BMT size in bytes:              $BMT_size"
echo "Files in this domain:           $files"
echo "Approximate BMT bytes per file: $BMT_bytes_per_file\c"
if [ $BMT_bytes_per_file -lt 2000 ]
then
    echo ""
else
    echo "  WARNING: This seems a bit too high."
fi
cat <<EOD

    This is approximate for several reasons.
    
        1)  This approximation does not take into account any currently
            unused space in the BMT.  For example if a new domain were
            created (or created with a larger initial -p) and only a few
            files existed on that domain, then most of the BMT would be
            empty.  However this script treats all of the BMT as if it were
            being used to map files.  This would result in a much larger
            bytes of BMT being used per file.  This would also be true if an
	    old domain had a lot of files deleted, which will free up BMT
	    space.

        2)  Not all files use the same amount of BMT space.  A large file
            which is highly fragmented will use more of the BMT than a small
            file.  Also files which have ACLs and other extra attribute
	    information associated with them will use more BMT space.  If
	    the usage of this domain is different than the target domain,
	    then the estimate may not be approprate.

EOD
#
#  The user would like an estimate of how large a BMT pre-allocation would
#  be needed to support a user specified number of files.
#
if [ $est_files -ne 0 ]
then
    set -- $vols
    n=$#

    new_BMT_size=`expr $BMT_bytes_per_file \* $est_files`
    new_BMT_pages=`expr \( $new_BMT_size + 8191 \) / 8192`

    echo "Number of BMT pages to pre-allocate (8192 bytes per page): $new_BMT_pages"
    echo ""
    echo "    The Number of BMT pages to pre-allocate should be"
    echo "    divided across all the volumes of the domain."
    if [ $n -gt 1 ]
    then
	per_vol=`expr \( $new_BMT_pages + $n - 1 \) / $n`
	echo "    This domain has $n volumes, so if this is the target domain,"
	echo "    then the value should be -p $per_vol, assuming all the"
	echo "    volumes in the domain are of the same size."
    fi
    echo ""
    echo "    Remember this is just an estimation based on an example domain."
    echo "    If the example domain is a poor example, the quality of the"
    echo "    estimate will be poor as well."
fi