[Search for users] [Overall Top Noters] [List of all Conferences] [Download this site]

Conference eps::benchmark

Title:Performance Benchmarking
Notice:Please read 1.* before writing
Moderator:MRKTNG::SLATER
Created:Tue Jan 22 1991
Last Modified:Thu Jun 05 1997
Last Successful Update:Fri Jun 06 1997
Number of topics:257
Total number of notes:561

253.0. "CPU Utilization??" by NYOSS1::MONASCH (I wrote the DECmate games) Mon Mar 17 1997 17:26

    While I was at Wizards training (about 2 weeks ago) I kept hearing the
    comment that on Windows NT application benchmarks, Alpha does NOT run
    out of CPU.  This might indicate that the Alpha box has cycles to spare
    to run other applications.  I guess what this comes down to is CPU
    Utilization.  So my questions are:
    
    1. Is this true?
    
    2. Can a case be made that we have CPU to spare to run other
    applications when an Pentium Pro does not?
    
    3. If so, can we start to quantify this by reporting CPU utilization
    for a given benchmark? On both Alpha and Prioris?
    
    4. Also assuming its true, could we develop a "mixed, concurrent
    applications benchmark" that would prove this?
    
    As you can tell, I looking for a competitive advantage for Alpha when
    the customer is choosing Compaq or Alpha (not prioris).
    
    Jeff
    
    
    
    
    
T.RTitleUserPersonal
Name
DateLines
253.1EPS::SLATERMarc, DTN 381-2445Tue Mar 18 1997 10:4172
Jeff,

|    While I was at Wizards training (about 2 weeks ago) I kept hearing the
|    comment that on Windows NT application benchmarks, Alpha does NOT run
|    out of CPU.  This might indicate that the Alpha box has cycles to spare
|    to run other applications.  I guess what this comes down to is CPU
|    Utilization.  So my questions are:
|    
|    1. Is this true?
|
 Depends on the application(s) being tested.  With MS Exchange, the CPU is
not saturated before response time and sendqueue limits are exceeded.  Lotus
Notes can be tested to CPU saturation.  PeopleSoft with SQL Server as well.

|    
|    2. Can a case be made that we have CPU to spare to run other
|    applications when an Pentium Pro does not?
|    
We are trying to build this case, see below. But think in terms of system
capacity rather than raw performance.  Include memory, system bus, I/O,
and network capacity.

|    3. If so, can we start to quantify this by reporting CPU utilization
|    for a given benchmark? On both Alpha and Prioris?
|    
If the CPU is not the limiting resource in the benchmark, then what is? It
should not be memory, disks, or network, unless a strong price/performance
case exists to justify crippling the CPU with inadequate peripheral 
resources.

|    4. Also assuming its true, could we develop a "mixed, concurrent
|    applications benchmark" that would prove this?
|    
Just such a workload has been built and is being characterized now. The
workload is made up of MS Exchange, file and print services, and Great
Plains Financials on SQL Server.  The mix is something like:

	15 Great Plains clients (no think time or intertransaction delay)
	16 file and print clients (no think time or intertransaction delay)
	600 Exchange users (medium workload)

|    As you can tell, I looking for a competitive advantage for Alpha when
|    the customer is choosing Compaq or Alpha (not prioris).
  
This is the case we are trying to build and have encountered some issues that
are being investigated.  They revolve around load balancing (we are getting
too much work done on the Alpha in some cases, but not in others, and can't
explain it) and where the CPU time is being spent (too much interrupt stack
time on the Alpha).  We are doing profiling on both the Alpha and Intel 
platforms and trying to compare across them.

Bear in mind that customers won't neccessarily buy the argument of "more
headroom" as a decision factor.  Price/performance is the issue.  The 
argument would be that Alpha allows the customer to support the same amount
of work on fewer servers (lower admin cost, smaller foot prints, less power,
less HVAC).

Look at the Lotus Notes results.  By tuning the Alpha code (SPE has Engineers
on-site at Iris) we are able to achieve record breaking numbers of users on
Alpha (before anyone flames let me state that this code is part of the standard
code pool, and is not a special build for Alpha). 

The same is true with RSA's RC4 block-encryption algorithms.  By tuning the
code and the build, Alpha can achieve greater than 100% better performance
than any Intel.

The same is true with QAD MFG/Pro, where a dual AlphaServer 4100 5/466 is
comparable to a quad 200 Mhz Pentium Pro.

Regards,

Marc