Hierarchical Project Resource and Disk Reporting

Last Update 2000-05-07 by kcarlson

ARSC Hierarchical Project Resource and Disk Reporting

Overview (resources, disk)

Two general utilities exist for resource reporting by project hierarchy by month(s) or by year:

uacrep - CPU and System Resource Summary
uadrep - Disk Resource Summary

These commands use a pre-defined 'project tree' hierarchy. At times there may be special reporting requirements to re-group project reporting by some other hierarchical method or to utilized a historical tree, both uadrep and uacrep default to the tree file being in the current directory, this may be set with "-tree filename".

Non-hierarchical 'end-user ad hoc reports' can be generated with other commands.

Resource Reporting with `uacrep` (overview, disk)

The uacrep reports system resource utilization, examples:

Command summary uacrep
Example of project report
Example of user report
Example of a summary report
Example of multiple resource reporting

The resource fields, "-show" are the same for both chilkoot and yukon, however, the metrics are different (e.g., "mpp" is meaningless on a J90) so these systems should always be reported with separate commands.

_______ (start uacrep)
1. Command summary, see also 'man -M /usr/local/adm/man uacrep':

yukon: cd /allsys/support/acct/summary/yukon # where data is stored

yukon: /usr/local/adm/bin/uacrep -? # command summary:
uacrep   v0.6           Report consolidated csacrep
Unicos           (must first format input with
|U/mk             csacrep_uacrep and uacrepmrg)

  -file        String(2) file(s) to report from (-1 to request)   Default:
  -tree        String(2) project tree file                        Default:tree

  -show        String(2) fields to show, choices are:             Default:cpu,mpp,res,blk,lio,job
                              cpu, mpp, phr, res, iom, iow,
                              blk, lio, pio, con, job, pro
                              (use 'all' for everything)
  -pct           Flag(2) show percentages instead of totals       Default:False
  -users         Flag(2) show users as well as projects           Default:False
  -hierarchy     Flag(2) show hierarchy totals only               Default:False
  -level          Int(2) hierarchy level to display               Default:2
  -mask        String(2) hierarchy to mask (display)              Default:

  -output      String(2) report output file specification         Default:
  -verbose       Flag(2) verbose option                           Default:False

_______ (start uacrep)
2. The following example prompts for input files and masks a level=4 project report:

yukon: uacrep -show mpp -mask DOD/NAVY/NRL -h -l4 
Input File(s)? 200001.Sum,200002.Sum,200003.Sum,mtd.cacct 

#Host:  yukon           #Tags:  200001 200002 200003 0409td
#
User   MPP
Count  Hours        200001 200002 200003 0409td
----- ------        ------ ------ ------ ------
   55 467859  82.9% 133308 141294 153628  39629 DOD
    8  30768   5.5% 2945.3  14833  11099 1891.1 DOD/NAVY/NRL
    2  24711   4.4%         12185  10635 1891.1 DOD/NAVY/NRL/NRLDC213
    1  908.5    .2%  160.3  748.2               DOD/NAVY/NRL/NRLDC282
    1 5148.4    .9% 2785.0 1899.6  463.8        DOD/NAVY/NRL/NRLDC367
    4                                           DOD/NAVY/NRL/NRLSS018

   25                                           ROOT

   59  96371  17.1%  38149  25176  27646 5399.0 UA
----- ------        ------ ------ ------ ------
  139 564229        171457 166470 181274  45028 Totals

_______ (start uacrep)
3. The following example specifies input files for fiscal year-to-date and displays use by user in masked projects. Specifying level=0 eliminates the project report:

yukon: F="199910.Sum,199911.Sum,199912.Sum,200001.Sum,200002.Sum,200003.Sum,mtd.cacct" 
yukon: uacrep -f $F -show mpp -mask DOD/NAVY/NRL/NRLD -users -l0 
#Host:  yukon           #Tags:  199910 199911 199912 200001 200002 200003 0409td
#
User   MPP
Count  Hours        199910 199911 199912 200001 200002 200003 0409td
----- ------        ------ ------ ------ ------ ------ ------ ------
    2  24735   2.4%                 23.6         12185  10635 1891.1 DOD/NAVY/NRL/NRLDC213
       24711   2.4%                              12185  10635 1891.1   :dev07d
        23.6                        23.6                               :kar17a
                                                                       :root
    1  908.5                              160.3  748.2               DOD/NAVY/NRL/NRLDC282
       908.5                              160.3  748.2                 :erw10
                                                                       :root
    1 5357.7    .5%          65.1  144.3 2785.0 1899.6  463.8        DOD/NAVY/NRL/NRLDC367
      5357.7    .5%          65.1  144.3 2785.0 1899.6  463.8          :hz15
                                                                       :root
----- ------        ------ ------ ------ ------ ------ ------ ------
  177  1011k        169354 117927 159321 171457 166470 181274  45028 Totals

_______ (start uacrep)
4. The following example uses an imbedded ls to specify all calendar year 2000 files and produces a level=3 summary:

yukon: uacrep -show mpp -h -l3 -f "`ls 2000*.Sum`" 
#Host:  yukon           #Tags:  200001 200002 200003
#
User   MPP
Count  Hours        200001 200002 200003
----- ------        ------ ------ ------
   54 428230  82.5% 133308 141294 153628 DOD

    2  42160   8.1%  13218  14568  14374 DOD/ARMY
    2  42160   8.1%  13218  14568  14374 DOD/ARMY/ACOMMKK1

   25 273734  52.7%  86461  92594  94679 DOD/CHALLENG
    3   48.8          46.4           2.4 DOD/CHALLENG/DTRAAC48
    6 177359  34.2%  60118  57560  59681 DOD/CHALLENG/NPSCAC43
    5  41886   8.1%  11813  17542  12531 DOD/CHALLENG/NSWCCC28
    1                                    DOD/CHALLENG/ONRC28B
    2  22408   4.3%    2.7  11396  11009 DOD/CHALLENG/ONRDCC28
    8  32032   6.2%  14482 6095.3  11455 DOD/CHALLENG/ONRDCC29

   27 112336  21.6%  33628  34132  44575 DOD/NAVY
    1 4719.0    .9% 3885.8         833.2 DOD/NAVY/NAWC
    1                                    DOD/NAVY/NPS
    8  28877   5.6% 2945.3  14833  11099 DOD/NAVY/NRL
    3  20175   3.9% 9194.7 2457.7 8522.5 DOD/NAVY/NSWC
   14  58565  11.3%  17603  16841  24121 DOD/NAVY/ONR

   25                                    ROOT

   58  90972  17.5%  38149  25176  27646 UA

   30  89591  17.3%  38083  24623  26885 UA/ACADEMIC
    1                                    UA/ACADEMIC/COLORADO
    1    0.2           0.2               UA/ACADEMIC/LSU
    1                                    UA/ACADEMIC/MCGILL
    1  11483   2.2% 2151.9 4676.9 4654.6 UA/ACADEMIC/RUTGERS
   17 2888.1    .6%   10.6  694.0 2183.6 UA/ACADEMIC/UAF
    9  75220  14.5%  35921  19252  20047 UA/ACADEMIC/UWASH

   25  448.0           4.6   34.7  408.7 UA/ARSC
    3    2.8                         2.8 UA/ARSC/CRAY
    2                                    UA/ARSC/NEEDPROJ
   18  445.2           4.6   34.7  405.9 UA/ARSC/STAFF
    1                                    UA/ARSC/TSTACCTS
    1                                    UA/ARSC/WEBADM

    1                                    UA/FEDERAL
    1                                    UA/FEDERAL/NOAAERL

    2  932.2    .2%   61.3  518.7  352.2 UA/IARC
    2  932.2    .2%   61.3  518.7  352.2 UA/IARC/NCARCCM
----- ------        ------ ------ ------
  137 519202        171457 166470 181274 Totals

_______ (start uacrep)
5. The following example shows several system resources for a level=2 summary:

yukon: uacrep -h -l2 -f mtd.cacct -show cpu,mpp,phr,job,iow 
#Host:  yukon           #Tags:  0409td
#
User    CPU          MPP   MPP-PE Total  iowait
Count  Hours         Hours  Hours Jobs   Hours
----- ------        ------ ------ ------ ------
   24  38109  87.5%  39629  746.0   2736  628.9 DOD

    1 3029.0   7.0% 3039.3   31.7     30    8.8 DOD/ARMY
   16  26950  61.8%  28255  398.4   2328  358.9 DOD/CHALLENG
    7 8129.8  18.7% 8334.5  315.9    378  261.2 DOD/NAVY

    3   42.5                        4299  132.0 ROOT

   30 5424.4  12.4% 5399.0  161.3  24940 1092.1 UA

   13 5390.8  12.4% 5376.8  160.6    496  466.7 UA/ACADEMIC
   16   10.3                       24416  560.6 UA/ARSC
    1   23.3          22.1    0.7     28   64.8 UA/IARC
----- ------        ------ ------ ------ ------
   57  43576         45028    907  31975   1853 Totals

Disk Use Reporting with `uadrep` (overview, resources)

The uadrep reports disk utilization, examples:

Command summary uadrep
Example of user report

Since the command syntax and options are similar to uacrep, fewer examples are shown for uadrep. Disk utilization is reported in megabyte-days. Since the file format is identical both chilkoot and yukon can be reported with the same command.

_______ (start uadrep)
1. Command summary, see also 'man -M /usr/local/adm/man uadrep':

yukon: cd /allsys/support/acct/summary/yukon # where data is stored

$HOST: /usr/local/adm/bin/uadrep -? # command summary:
uadrep   v0.6           Report consolidated diskusg
Unicos           (must first format input with
|U/mk             disku.nawk and uadrepmrg)

  -file        String(2) file(s) to report from (-1 to request)   Default:
  -tree        String(2) project tree file                        Default:tree

  -show        String(2) fields to show, choices are:             Default:monl
                              ftot,fonl,foff,mtot,monl,moff
                              (use 'all' for everything)
  -device      String(2) Storage types, 'all' for everything      Default:all
                              (can have multiple -s OR -d, not both)

  -pct           Flag(2) show percentages instead of totals       Default:False
  -users         Flag(2) show users as well as projects           Default:False
  -hierarchy     Flag(2) show hierarchy totals only               Default:False
  -level          Int(2) hierarchy level to display               Default:2
  -mask        String(2) hierarchy to mask (display)              Default:

  -output      String(2) report output file specification         Default:
  -verbose       Flag(2) verbose option                           Default:False

_______ (start uadrep)
2. The following example prompts for input files to report megabyte-days for all users of a masked project:

$HOST: uadrep -l0 -mask DOD/CHALLENG/NPSCAC43 -users 
Input File(s)? chilkoot/mtd.dacct yukon/mtd.dacct 

#Host:  chilkoot        #Tag:   0409td  Days:   8.0
#Host:  yukon           #Tag:   0409td  Days:   8.0
#
User  chiAll chiHom chiTmp chiViz yukHom yukTmp
Count MByOnl MByOnl MByOnl MByOnl MByOnl MByOnl
----- ------ ------ ------ ------ ------ ------
   11  132.8  117.4               150843  68110 DOD/CHALLENG/NPSCAC43
       131.2    0.8               1312.0    0.2   :bra04x 
                                   31577          :dst08e
                0.8                  0.8          :mal21x 
                0.6               5439.1  11484   :mar22f
         1.6   13.6               112010  56627   :mas26x  
                                   501.6          :mcc28x 
                0.8                  0.8          :pia36x 
                0.8                  0.8          :pos37
                0.8                  0.8          :pre38x 
                0.8                  0.8          :tok42x  
               98.4                               :yzh50z
----- ------ ------ ------ ------ ------ ------
  414 438086 295052 120571   7804 851937 363876 Totals: MByOnl Days:8.0 chilkoot

ARSC Hierarchical Project Resource and Disk Reporting

Overview (resources, disk)

Resource Reporting with uacrep (overview, disk)

Disk Use Reporting with uadrep (overview, resources)

Resource Reporting with `uacrep` (overview, disk)

Disk Use Reporting with `uadrep` (overview, resources)