| T.R | Title | User | Personal
Name | Date | Lines |
|---|
| 3410.1 | PC Magazine | SIERAS::PARK | | Fri Sep 23 1994 23:33 | 1 |
| It is in PC Magazine, not in PC Week.
|
| 3410.2 | | QUARK::LIONEL | Free advice is worth every cent | Sat Sep 24 1994 00:43 | 3 |
| I'd suggest taking this to VAXAXP::ALPHANOTES.
Steve
|
| 3410.3 | Call the ASE group, Merrimack | RDGENG::WILLIAMS_A | | Mon Sep 26 1994 08:39 | 15 |
| Contact the ASE group, who do a fair amount of comparative
characterisation (ie, concentrate on much more than the MHz of the chip
in the box). Try Steve Davis or Mark Slater as first contacts.
My take (FWIW)... If the test suites aren't processor intensive, then
we shouldn't be too surprised if we don't have the 'expected'
performance advantage.
ASE can probably give you a whole bunch of other issues that need to be
considered, to determine exactly how 'like-for-like' this test might
have been.
Hope this is of some help,
Adrian.
|
| 3410.4 | Sybase has problems? | MIMS::SANDERS_J | | Mon Sep 26 1994 13:17 | 7 |
| Holiday Inn Worldwide just dumped Sybase in favor of Informix, claiming
that Sybase could not cut it when it came to performance.
Maybe there are other problems here.
Informationweek, 9/5/94, p. 15.
|
| 3410.5 | Sybase's problem is not a AXP problem... | CSC32::C_BENNETT | | Mon Sep 26 1994 14:29 | 13 |
| Any benchmark if not done fairly can be manipulated to a vendors
advantage. Is Sybase's version which runs on the AXP a native
version or was it simply pulled over to the AXP from the VAX?
What disk's were used?
Assuming the oranges and apples match up (TPS standards...) how does
the Sybase database product match up to the Rdb TPS results?
Maybe this is a pure case that Sybase doesn't do AXP? Maybe the
leason to be learned is that for Sybase customers they should look at
other database vendors (such as Rdb or Oracle) if they really want
a screaming DB on the AXP platform.
|
| 3410.6 | | NOVA::FISHER | Tay-unned, rey-usted, rey-ady | Tue Sep 27 1994 08:50 | 7 |
| About 8 months ago, I was discussing database software products
on AXP with one of our engineers and was told that Sybase had
done nothing to improve their performance on AXP, especIally
aligning data structures. I don't know whether that is still
true. [But it does appear to be!]
ed
|
| 3410.7 | | CSC32::C_BENNETT | | Tue Sep 27 1994 11:56 | 9 |
| .6 - seems to me that the fact that Sybase does care about a simple
(but critical on AXP) matter like data alignment on the AXP would
be the rebuttal...
Maybe Digital (or Oracle) from the database software side could rebut
this with an add comparing Rdb to Sybase on an AXP... From the
hardware side - I dislike the fact that the AXP was 5th out of 5 but
really when the facts come out and a tuned, aligned database is invoked
- Rdb on the AXP is the fastest relations Db around!
|
| 3410.8 | fwiw | KLUSTR::SOUTHY::Gardner | Southie Mudshark | Tue Sep 27 1994 12:37 | 30 |
| I have read this "test" a number of times...it is crude at best
and extremely uninformational at worst......for those who have not
seen it, the write up is a 3/4 page insert in the middle of a
much larger (and otherwise extremely thorough) comparo of db products
on *Intel* platforms..........
problem 1: no pricing is supplied for the RISC configurations
compared....just glancing at it, the systems would appear to
have an extreme variance in price
problem 2: no configurations specifics (disk, mem, or even o/s)
are noted......our single CPU sys probably went in with 1 disk
32M mem while you can bet both the Compaq and HP systems had at
least hardware RAID and lotsa mem....
in summary, this "test" is crap, and not worth the ink used to
print it...certainly with the information supplied, I find it
impossible to subscribe to their conclusion that "Intel SMP
hardware is clearly in the same league as the RISC based
heavyweights"....in fact, I would say that the entire "test"
appeared hand-crafted to support the conclusion (another
classic case of Ziff/Davis putting publishing before science)....
the only thing I'm curious about is from where they got the
Alpha...did we supply it (in which case we were either misled
about the "parameters" of the test or made a grave mistake in
not supplying a 4 processor Sable) or did PCMag just grab one
out of another ZD lab?
_kelley
|
| 3410.9 | | PASTIS::MONAHAN | humanity is a trojan horse | Tue Sep 27 1994 13:01 | 2 |
| Could we persuade Oracle to put out advertising saying "If you are
thinking of moving to 64-bit computing, don't use Sybase"?
|
| 3410.10 | They're Wrong So It Doesn't Matter? | LJSRV2::FEHSKENS | len - reformed architect | Tue Sep 27 1994 13:22 | 13 |
|
I wish I could just dismiss this test out of hand as flawed,
irrelevant, whatever, but the fact is that, for whatever reasons,
the performance of an important database product on Alpha was bad.
That is what people who read this article will remember, not some
litany of excuses (which will probably never get outside Digital)
As long as we continue to believe that having the fastest
microprocessor chip in the world is all that counts, we deserve
our "success".
len.
|
| 3410.11 | apples vs. pita bread | KLUSTR::GARDNER | The secret word is Mudshark. | Tue Sep 27 1994 13:44 | 23 |
| re: .10
my point is, although it may be true that Sybase System 10 on
Alpha (OpenVMS? DEC OSF/1?) may need performance work, it is
impossible to conclude from this "test" that this in fact is
the reason we came in "last"...instead, I prefer to think
that there was simply a huge skew in the hardware that was
"compared"....its possible to imagine, for example, that the
Alpha represented the "entry level" system in this comparison,
in which case it didn't do so bad, did it? of course we may
never know......
oh, and btw, the "testers" really had no intention of determining
the actual performance of Sybase on Alpha, which would have
required much more extensive testing/tuning than was appearently done...
they simply needed to find a convenient way to conclude that a
two processor Pentium box is "in the same league" as the "RISC
heavyweights", a generalization applied to *all* the non-Intel boxes
"tested" which, if you think about it, doesn't put us in such
a bad light afterall......
$.02
_k
|
| 3410.12 | it's happened before | WHOS01::ELKIND | Steve Elkind, Digital Consulting @WHO | Tue Sep 27 1994 14:09 | 19 |
| Several months ago, a DEC 300 mod 800 (running OSF/1) was in a similar
test written up by, you guessed it, ZD labs. It also lost out to
everything but an AT&T dual-Pentium box. It had the smallest number of
spindles, only a single SCSI bus, ... (although all systems had the
same RAM). I think it was also Sybase, again - and system params were
set up as recommended by Sybase (I've had some negative experience with
this approach on a VAX).
If I remember correctly, it had been obtained from Digital, and
somebody here configured it based on some description (supposedly) of
the test. It appeared to me to have been configured for minimum cost
(all 2GB disks to meet the storage space requirements, no expander box
for storage...), rather than for performance. The AIX machine had 9
spindles over at least 2 busses (they didn't say how many),...
It was the cheapest box in the tests, but SO WHAT? The primary
impression one walked away with was that it was the close to being the
least effective performer. Most readers would not have spotted the
more glaring shortcomings of this "benchmarking".
|
| 3410.13 | oops | WHOS01::ELKIND | Steve Elkind, Digital Consulting @WHO | Tue Sep 27 1994 14:10 | 1 |
| oops, DEC 3000, not DEC 300
|
| 3410.14 | sybase solution in the works...? | SMURF::KHALL | | Tue Sep 27 1994 15:24 | 8 |
| Six or eight weeks ago, I heard thru the grapevine that one of the
Rdb developers who had contributed greatly to Rdb's performance
improvements was leaving Digital to work for Sybase. If true,
perhaps we can expect the Alpha/Sybase performance problem to be
remedied in the not too distant future.
\ken
|
| 3410.15 | | MSBCS::BROWN_L | | Tue Sep 27 1994 15:30 | 8 |
| re .12
I'm pretty sure they just used the same numbers over again from their
earlier May review. This latest Oct 11 article is just a rehash of
those results. I believe the problem was that the 3000/800S only
had 64Mb, so the box was memory constrained. While we'd like to
compare OSF/Sybase systems with 128Mb or more, we must acknowledge
that 64bit OSF needs more memory than anybody else. A good case for
a 32bit "OSF-lite". .02 kb
|
| 3410.16 | Some facts about Digital/Sybase and AXP | SX4GTO::MACKBEE | | Wed Sep 28 1994 08:07 | 72 |
| I too grabbed a copy of PC Magazine and read this article. And I must
agree with .8 and .12 that this, at best is a very ambigous performance
test (if I could be so generous to call it that).
As the Digital on-site Project Manager at Sybase, I do have my own
prejudice regarding Sybase on AXP, but that aside, here are some facts:
o The current shipping version of the Sybase SQL Server on AXP OpenVMS
is v1.5 and v1.3 on OSF/1. I am positive that these tests were
performed using one of these (I suspect they used OSF/1) or possibly
an even older version.
o We are currently finishing up QA testing on our OSF/1 V3.0 SMP
version which was built on the SABLE (2100 Server).
o This product will be released based on the latest and greatest version
of the Sybase SQL Server (v10.0.2 planned to be released in Q4).
o We are working "shoulder to shoulder" with the Sybase Performance
Engineering Group, Sybase Platform Development and our own OSF/1
Engineering Organization to address performance issues.
o We are performing TPC-C testing and analysis on-site (Sybase central
engineering in Emeryville CA) as well as back east. This activity
will result in "Audited TPC-C" numbers on AXP.
o We are the first system vendor (HP, IBM, Sun, etc) to perform and
complete a joint Product Certification on-site with Sybase
(OSF/1 V2.0 and V2.0B). The Certification was performed using
a UNI 2100 (SABLE).
o We have a full team of engineers on-site at Sybase working on 11
projects that span Product Certification, Product Porting,
QA/Regression Testing, Performance Analysis and Characterization,
Technical Support and Advanced Development activities.
o The Digital on-site team, consists of engineers that "live" on-site
at Sybase - "8x5X40"!
o We have "weekly" joint technical meetings with Sybase engineering
to discuss project status, resolve outstanding problems/issues and
coordinate plans for additional joint engineering/development activity.
Jim Bolton (The Sybase Global Account Manager) and I will be working
together to address this PC Magazine article. Personally, I'd like to
see this test performed on our new Sybase SQL Server running OSF/1 v3.0
on a MP SABLE. (As stated above, this product is currently in
QA - once released, it's availability will be published in the Sybase
futurs::sybase notes conference)
I'm positive - all things being equal the results would be quite
different. We'll coordinate with the appropriate corporate and
engineering organizations to develop the best rebuttal to this test and
push for another test which is more indicative of the capability of
Sybase SQL Server on AXP.
Over the past 4 months we've made great progress, but there is much
more that we wish to accomplish. We will continue to strive to make
Digital the Sybase SQL Server price/performance leader.
If you have any questions, comments or suggestions, contact me at,
sx4gto::mackbee or
mackbee@sybase.com (my Sybase e-mail address)
regards,
Douglass Mackbee
|
| 3410.17 | | SPECXN::WITHERS | Bob Withers | Wed Sep 28 1994 14:33 | 20 |
| PC Magazine tests are based on two things:
Are the products shipping at the time the review is done?
Using a *vendor-supplied* configuration.
Two months ago, we had Sables, but the bulk of Alpha PCs were Jensens. Sybase
V10.0.2 will be available *next* quarter. So, they tested
state-of-the-then-art and we came out last. Wouldna, couldna, shouldna, wait
till next quarter won't cut it.
Some marketiod probably decided that they wanted to shoot for the best-bang
corner and low-balled the configuration. Unfortunately, other things on the
market were faster, smarter, cheaper. Remember that PC Mag won't test (as a
rule) Beta Test products and our configs were not on the dime.
In other words, we lost and, while the tests may be flawed, we largely did it
to ourselves.
My tuppence,
BobW
|
| 3410.18 | progress! | MBALDY::LANGSTON | our middle name is 'Equipment' | Wed Sep 28 1994 23:14 | 9 |
| 3410.19 | | MRKTNG::SLATER | Marc, ASE Performance Group | Wed Sep 28 1994 23:49 | 13 |
| We have tested the performance of only one application that layers upon Sybase
on OSF/1: The Dodge Group's OpenSeries General Ledger. After overcoming some
shoddy memory management and applying standard Sybase tuning techniques, the
application performed rather well. As far as I know, OpenSeries does not
run on any Intel platforms (yet), only HP and IBM and ours.
At any rate, thanks to Doug for repsonding here. I know some of the history
behind his efforts to get Sybase to pay attention to performance on our
platforms, and am glad that he has more patience and persistance than I.
Regards,
Marc
|
| 3410.20 | Response to PC Magazine Article | MSBCS::ANDERSON | | Fri Sep 30 1994 21:41 | 30 |
| This note is for Digital Sales people who may be asked about a recent article
in PC Magazine.
The October 11th PC Magazine (p268) contains an article showing poor Alpha OSF
RDBMS performance compared to other UNIX platforms and to a Compaq ProLiant PC.
The article describes a test where the ProLiant PC performed as well or better
than four popular UNIX platforms. This conclusion, however, is based on an
unusually vague test description that is difficult to either support or
disclaim.
The test is described as Sybase System 10 running five platforms against a
named test suite but beyond that there is:
1) No defined basis of comparison other than UNIX vs PC.
2) No platform description other than model number and number of processors.
3) No description of the test suite.
Further, all the competitor's platforms in the test were SMP while the Digital
platform was single processor.
Though not technically sound, the article has reached the press and you may
need to respond to it. The best response will be available near term, an
audited TPC-C Benchmark for Sybase System 10 running on an OSF/1 V3.0 SMP
Sable. Beyond that, the Sybase Account Team will discuss the above test with PC
Magazine and, if it makes sense, invite them to test DEC OSF/1 with SMP.
Any updates to this issue will be posted in this notes file.
|
| 3410.21 | | CSC32::M_AUSTIN | Michael,804-237-3796,OLTP-EC | Mon Oct 03 1994 11:34 | 9 |
|
Re: .16
>Digital the Sybase SQL Server price/performance leader.
You will have a very tough time beating RDB any way you slice
it!!
Mike Austin
RDB Support
|
| 3410.22 | | QUEK::MOY | Michael Moy, DEC SQL Engineering | Mon Oct 03 1994 13:30 | 6 |
| re: -1
I think they're trying to say that Sybase SQL Server has the best pp on
Digital platforms.
michael
|
| 3410.23 | Do we have any benchmark results to counter with? | OSL09::OLAV | Do it in parallel! | Tue Oct 04 1994 13:01 | 4 |
| Can anyone point to be any database benchmark results comparing Sable with
Compaq ProLiant? Ideally both should be running Windows NT.
Olav
|
| 3410.24 | TPC-A results | MSBCS::STEINHARDT | | Tue Oct 04 1994 13:16 | 21 |
| TPC Benchmark (tm) A (TPC-A) Test Results:
Digital 2100-A500MP (1 CPU): 265 tps (second only to the 4-way 2100
in price/performance)
Digital 2100-A500MP (4 CPUs): 662 tps (the world record holder for best
price/performance)
COMPAQ ProLiant 2000 M5/66 (2 Pentium CPUs): 240 tps
The Digital 2100s are both running OpenVMS, Rdb, and ACMS
The COMPAQ ProLiant is running SCO UNIX, Oracle 7
The Bottom Line: The single processor 2100 outperforms the two
processor Pentium-base ProLiant on this benchamrk, and does so with
superior price/performance. All numbers are public and audited.
Regards,
Ken
|
| 3410.25 | Price/Perf | MSBCS::STEINHARDT | | Tue Oct 04 1994 13:29 | 15 |
| Price/Performance for the previous data:
Digital 2100-A500MP (4 CPU) 662 tps @ $4,401/tps (the world record)
Digital 2100-A500MP (1 CPU) 265 tps @ $4,405/tps (#2 in p/p)
COMPAQ ProLiant 2000 M5/66 (2 CPU) 240 tps @ $4,890/tps (#6 in p/p)
The top five positions for price/performance on the TPC-A benchmark
are held by Digital OpenVMS systems (4 Alpha, 1 VAX), and seven of the
top ten positions. One year ago Digital only had #6 and #7 among the
top ten.
Regards,
Ken
|
| 3410.26 | Is it on our World Wide Web? | OSL09::OLAV | Do it in parallel! | Tue Oct 04 1994 14:47 | 6 |
| Re: .24 and .25
Make sure theese numbers are published *highly* visible in our external
World Wide Web pages!
Olav
|
| 3410.27 | It is on the Web | MSBCS::MORGENSTEIN | Achilles loved Petroclus | Tue Oct 04 1994 16:50 | 11 |
|
Digital 2100-A500MP (4 CPU) 662 tps @ $4,401/tps (the world record)
Digital 2100-A500MP (1 CPU) 265 tps @ $4,405/tps (#2 in p/p)
This is not NT data. This is RDB with OpenVMS.
Ruth Morgenstein
CSG Performance Group
|
| 3410.28 | Who said it was NT???? | MSBCS::STEINHARDT | | Tue Oct 04 1994 18:01 | 5 |
| re:-1
see .24
-Ken
|
| 3410.29 | $/tps ... how's it calculated | KAOFS::R_DAVEY | Robin Davey CSC/CTH dtn 772-7220 | Tue Oct 04 1994 18:05 | 13 |
| re: .25
> Digital 2100-A500MP (4 CPU) 662 tps @ $4,401/tps (the world record)
> Digital 2100-A500MP (1 CPU) 265 tps @ $4,405/tps (#2 in p/p)
> COMPAQ ProLiant 2000 M5/66 (2 CPU) 240 tps @ $4,890/tps (#6 inp/p)
Can somebody tell me how the $/tps numbers are calculated or are the
above incorrect. Those numbers tell me that a Digital 2100-A500MP (4CPU)
system costs $2,913,462 and I somehow doubt that.
Robin
|
| 3410.30 | | WLW::KIER | My grandchildren are the NRA! | Tue Oct 04 1994 18:42 | 16 |
| > Can somebody tell me how the $/tps numbers are calculated or are the
> above incorrect. Those numbers tell me that a Digital 2100-A500MP (4CPU)
> system costs $2,913,462 and I somehow doubt that.
With the number of disk drives and controllers to support the
database that a 662 tps TPC/A requires (a fixed amount per TPS),
and the tape devices necessary to load those drives in a useable
amount of time and the front end communications necessary to
handle the 662*(10 or 100, I forget) simulated users I would expect
that number is very probably correct. A 662 tps TPC/A is *big*
(but think of our 3,000+ tps number on the big cluster).
Anyway, the full disclosure report should give you the breakout of
the cost information.
Mike
|
| 3410.31 | $2.9M = 5 yr. cost-of-ownership | MSBCS::MORGENSTEIN | Achilles loved Petroclus | Tue Oct 04 1994 18:45 | 14 |
|
You're right, Ken. I hadn't looked far enough back to see the full
text describing the results.
Robin, the $/tpsA contains the 5 year cost of ownership for all
the equipment in the test (including 10 * tpsA terminals). The TPC-A
specification detailed instructions on what gets priced.
The summary page from the Full Disclosure report for this test and
others is available in TPSYS::SW_PERFORMANCE:[TPC.SUMMARY]. It contains
a full price sheet for the test.
Ruth Morgenstein
CSG Performance Group
|
| 3410.32 | How about letting folks OUTSIDE know? | DPDMAI::HARDMAN | Sucker for what the cowgirls do... | Wed Oct 05 1994 02:01 | 11 |
| Great. So now the few folks that actually read this notesfile know how
fast our Aplha boxes can run a database. And some more folks might
stumble across it on the WWW.
The BIG question is.... What about the zillions of people that saw the
PC Rag article that said right there in black and white that our Alpha
boxes are the slowest on the planet? How is Digital going to let those
folks know the truth?
Harry
|
| 3410.33 | pay them to publish it - advertise | TROOA::BROWN | RPC - Really Practical Computing | Fri Oct 07 1994 03:58 | 6 |
| >> boxes are the slowest on the planet? How is Digital going to let those
>> folks know the truth?
*advertise* the audited results compared to others.
the more you pay ie.advertise, the less they are inclined to bash
|