Developing a benchmark program

[turbowrx]

Couple of questions.

I'm developing my own computer benchmark to use. Now, it isn't going to be on par with the well known ones, but the idea for me is to give myself a programming challenge, and a challenge it will be, and to offer another benchmark to fellow testers. It will be modeled after some of the futuremark benchmarks, but with more mathematical problems built in. I want to put together about ten tests, a mixture of visual and non-visual. I've thought of about 6 or 7 tests, all depending on how i look at them.

The benchmark will be available for people to use when it is finished and it will be accompanied by a website so that scores can be compared. Again, fashioned after futuremark, with subtle differences.

Now to the questions. I've started developing in java, but was wondering what everyone thought of that? Also, i would like to know what you, the user, wants to see in a benchmark? What tests you would like to see and what kind of user interface?

Thanks for your replies and any feedback will be helpful.

 

[vinnie]

Java for benchmarks? Isn't that a little... odd?

I'd have thought something without so many layers between the tests and the hardware would be more appropiate, unless of course you're trying to test the JVM on different systems...

 

[turbowrx]

Well, whatever code I choose, i've started in java but could easily switch, to write the program in will be the only one distributed. That means that the scores can be compared.

Java is the language i know best, which is why it was initially chosen. What i'm asking is would it be better to use a different language, ie. c# or c++?

In most of the tests i don't think the language really matters. Java can do compression and encryption just fine. Now doing a 3d test that measures fps is a different story. OpenGL is written in c, i believe. This is even okay as the jni can be used to access the functions. I want to know everyone's take on this. If you consider java a little odd, what would you recommend? Thanks for the feedback.

 

[STL]

Why write fake benchmarks when you can use real ones?

I wish hardware review websites would use benchmarks I actually cared about. For example, bootstrapping gcc, or bzip2ing a large file.

Java's a joke. Use C++.

> Java can do compression and encryption just fine

Ha! Ha ha ha.

 

[vinnie]

Originally posted by turbowrx
In most of the tests i don't think the language really matters.

Yes um... well... you're entitled to your opinion I suppose. I suggest you lean towards a real language, not an interpreted one. You're going to be running tests on a real machine from inside a virtual machine. You can't see the problem with that?

I say try C++.

As STL said, why not make something meaningful? Something people may actually want to compare? Compressing, whatever, something useful and different.

 

[turbowrx]

I understand what you are saying Vinnie and since the consesus, although only a couple of people, says go with c++, I may just do that. Remember, the most important part of this for me is the programming challenge, not building a top notch benchmark.

That being said, I would like to build something users could use and would want to use, which listening to you guys tells me I shoud write it in c++. So what kinds of tests would you users like to see. I'm planning right now on the standard decompress/compress, encrypt/decrypt, a 3d module, scientific calculations, possibly some audio/video file tests, and I would like to test the hyperthreading capabilities of the pentium processors by using threaded code.

Does anyone have any neat ideas or suggestions to offer?

 

[STL]

[vinnie]
> Yes um... well... you're entitled to your opinion I suppose. I
> suggest you lean towards a real language, not an interpreted one.

*giggle*

However, Java sucks even when compiled to machine code.

--

I've considered making my suffix tree BWT compressor into a benchmark. That's on my gigantic pile of things to do.

 

[turbowrx]

We could debate all day over which language is better overall. It's like debating on whether or not amd or intel is better. They both have their niches, and programming languages are the same way.

In terms of what I am attempting to do, you guys are probably correct in that if I want to create the best benchmark application I can then I should do it in C++. C++ is much better to use for hardware acceleration and for scientific calculations, which will account for a lot of what i'm doing.

I understand with having a lot of projects to do STL. I'm going to attempt this benchmark, along with run a small business and a web server for it and that doesn't even count my 9 to 5 job. This means I don't tend to do a lot of programming outside of work because I hang out with my girlfriend and love sports.

 

[doh]

Originally posted by STL
Why write fake benchmarks when you can use real ones?

I wish hardware review websites would use benchmarks I actually cared about. For example, bootstrapping gcc, or bzip2ing a large file.

Give the distcc benchmark suite (the suite) a look-see. That will stress your cluster. What it does is download the Linux Kernel Source and build it, and then build a whole bunch of other things. Hell on disk space, but fun to use.

 

[horndog]

write it in C++ or something, java has too much overhead which will affect the benchmarks, then the results would not be accurate

 

[Spleeze]

Originally posted by STL
[vinnie]
> Yes um... well... you're entitled to your opinion I suppose. I
> suggest you lean towards a real language, not an interpreted one.

*giggle*

However, Java sucks even when compiled to machine code.

Just curious why java sucks so much... Maybe in this application it is not appropriate... I'm interested in your take on java, since my school just switched to teaching java to incoming freshman instead of c++....

 

[STL]

[Spleeze]
> I'm interested in your take on java, since my school just switched
> to teaching java to incoming freshman instead of c++

"How do you write women so well?"
"I think of a man, and then I take away reason and accountability." - Melvin Udall, As Good As It Gets

Take C++. Then remove good feature after good feature from it. This includes:

* Operator overloading
* Templates
* Destructors
* Covariant returns
* Default arguments
* Multiple inheritance
* C's machine model
* Namespaces
* Function pointers
* Const correctness

Note that this implies the removal of:

* Smart pointers (which require operator overloading)
* The Standard Template Library (requires templates and destructors)
* Resource Acquisition Is Initialization (requires destructors)
* Functors (which require operator overloading)

Now mix in mandatory garbage collection and a wacky machine model. Congratulations! You've got a language with no "explicit" pointers that uses pointers everywhere! Let's give it to Alexander Stepanov and see what he says.

"I spent several months programming in Java. Contrary to its author's prediction, it did not grow on me. I did not find any new insights - for the first time in my life programming in a new language did not bring me new insights. It keeps all the stuff that I never use in C++ - inheritance, virtuals - OO gook - and removes the stuff that I find useful. It might be successful... but it has no intellectual value whatsoever" - Alexander Stepanov

There are only a few things I like about Java, and they really don't have anything to do with the language itself. (Those being precisely specified integer sizes, and a behemoth standard library. I'm ignoring for now that Java's standard library sucks.)

I spent about 3 months working with Java back in Caltech CS 3. I never understood what I was doing with it. When I started learning C++ sophomore year, I picked it up extremely rapidly - it just made sense. Now that I've learned how to use large portions of the Standard Template Library and Boost, I don't see how anyone could consider using Java.

What's funny is that the things I don't like about C++ aren't even problems in Java, because the relevant facilities aren't just imperfect, they aren't even there. My current annoyances with C++ include:

* No function template partial specialization (may appear in C++0x, can be worked around by using class template partial specialization) - Java doesn't even have templates
* No default template arguments for function templates (will appear in C++0x, can be worked around by writing separate overloads) - Java doesn't even have templates or default arguments, much less default template arguments

 

[:LJ:]

Ah good, STL's back. I've missed your endless discourse about how and why C++ will take over the world.

And have any of you actually considered that a benchmark written in Java would be a very good idea for measuring server performance? Especially since there are a hell of a lot of server apps out there written in Java. I'd certainly be interested, since it would give a good idea of the relative merits of various platforms.

Having said that, maybe I should junk all those incredibly useful, portable and lucrative applications in favour of redeveloping it all in C++. Yes....that's the way. I can feel my mind changing now. Must be all those long words STL uses. Kinda....hypnotic.....

EDIT : Removed potentially insulting comments. No point getting personal....

 

[turbowrx]

As I've already stated, both languages have their niche where they are incredibly useful. In fact, java has enough going for it, or at least the people making the decisions think so, that it is being used in many applications in my workplace, which is a lab. If java was so bad, there wouldn't be highly educated people all around me using it.

Anyways, this is about developing a useful benchmark.

:LJ - the server benchmark is a great idea. It wouldn't fit in with what I want to do, but it would be a nice project. Developing a benchmark for server systems is something that could be done down the road.

 

[:LJ:]

Originally posted by turbowrx
:LJ - the server benchmark is a great idea. It wouldn't fit in with what I want to do, but it would be a nice project. Developing a benchmark for server systems is something that could be done down the road.

In my opinion, the servers and enterprise environments are the only really useful area for a Java benchmark, since that's where most Java coding gets done. It's an interesting idea -I may take it up sometime.

 

[STL]

[turbowrx]
> If java was so bad, there wouldn't be highly educated people all
> around me using it.

There are lots of people who are seduced by bad things.

 

[turbowrx]

Your entitled to your opinion STL, but I think you are looking at the situation with a narrow mind and that's my opinion. The fact of the matter is that java fills a need, just as c++ does. Maybe you can't allow yourself to see that or your head has been filled by the opinions of your professors, I don't know. You seem to be very negative towards java, almost as if you blame it for something.

Think about this. Maybe your bias towards c++ is due to the fact that the programs you develop should be developed in c++?

 

[STL]

[turbowrx]
> Your entitled to your opinion STL, but I think you are looking at
> the situation with a narrow mind and that's my opinion.

You don't even know how to program. I don't think your opinions are worth anything.

> The fact of the matter is that java fills a need, just as c++ does.

You happen to be correct.

<ithil> Every language is a niche language. C++ is the least nichey

<ithil> Java is useful for RAD and web deployment. Both niches are pretty small.

I tend to disagree with ithil when it comes to rapid application development, though I concede that C++ can't do applets at this time.

Unlike you, ithil has experience in both Java and C++ - and he's a Good C++ Programmer - which is why I listen to him.

> Maybe you can't allow yourself to see that or your head has been
> filled by the opinions of your professors, I don't know.

Let's see what Caltech professors think.

Professor A: "Yay for Java!"
Professor B: "Yay for Scheme! Though most of my grad students work with Java."
Professor C: "Yay for Shitty, Common C++!"
Professor D: "Yay for OCaml!"

Uh huh.

(It may amuse the reader to learn that these are accurate portrayals.)

> You seem to be very negative towards java

Java is intellectually bankrupt.

Is that negative enough for you?

> Maybe your bias towards c++ is due to the fact that the programs
> you develop should be developed in c++?

Let's see what programs I have written/am writing so far.

1. Advanced data compressor.
2. 2D game.
3. Web-based forum.

Oh yeah, they all fit into a small niche, don't they?

C++ is the best general-purpose programming language. Other languages can beat it, but they all live in small little niches, and break terribly when you try to attack different problems with different approaches. C++'s problem is that it supports so many approaches, most people don't know what the good ones are. (See "Shitty, Common C++" comment above.)

 

[FuzzyDonkey]

Originally posted by STL
Professor A: "Yay for Java!"
Professor B: "Yay for Scheme! Though most of my grad students work with Java."
Professor C: "Yay for Shitty, Common C++!"
Professor D: "Yay for OCaml!"

Uh huh.

(It may amuse the reader to learn that these are accurate portrayals.)

This is common at several institutions, good and bad, not just at Cal Tech. After being in the real world, I've seen that my aversion to Java during college wasn't a bad thing, despite my compiler's prof praising it as a beautiful language. Sun's internal memo saying that Java (their language) sucked on Solaris (their OS) didn't discourage this either. I've lost the link to the original memo, here's an article.

http://www.eweek.com/article2/0,3959,873887,00.asp

Having seen a enterprise app developed in Java on Solaris, I'd have to say it wasn't pretty. Fortunately, I wasn't directly involved with the development, I was doing driver and OAM code for the system it interfaced with.

 

[STL]

[FuzzyDonkey]
> This is common at several institutions, good and bad, not just at
> Cal Tech.

Oh yes.

(The abbreviation is "Caltech", by the way.)

In academia, there's even less pressure to use languages which are good for developing large systems.

> After being in the real world, I've seen that my aversion to Java
> during college wasn't a bad thing

Giggle.

> despite my compiler's prof praising it as a beautiful language.

See, professors can say all sorts of things. Doesn't make them right.

> Having seen a enterprise app developed in Java on Solaris, I'd have
> to say it wasn't pretty.

Heh.

 

[tim_m]

Originally posted by STL
Take C++. Then remove good feature after good feature from it. This includes:

* Operator overloading
* Templates
* Destructors
* Covariant returns
* Default arguments
* Multiple inheritance
* C's machine model
* Namespaces
* Function pointers
* Const correctness

as you may or may not recall from my mentioning it way back in september i guess, my college just switched from teaching c++ to java this year. my professor keeps comparing things between java and c++ and anything that c++ did that java doesn't do that he's mentioned seems like a good thing. i mean, 'if (str1.equals(str2))' just sucks. ... not to hijack this thread into a good ol' c++ > java or anything like so many other times

 

[STL]

[tim]
> anything that c++ did that java doesn't do that he's mentioned
> seems like a good thing. i mean, 'if (str1.equals(str2))' just
> sucks

Correct.

Operator overloading is an important technique for making code comprehensible.

Consider:

str1 == str2
smart_pointer->something
functor(arguments)
some_vector[index]
some_map["key"]
stream << output;
bignum + 10
one_string += another_string
one_vector + another_vector (concatenation; I provide this myself since the STL doesn't)

It gets better, really. Suppose you have a vector<int> and you want to, say, quadruple everything in it. You can write:

transform(v.begin(), v.end(), v.begin(), _1 * 4);

Or suppose you have a vector<int *> and you want to sort it according to the values of the pointed-to ints (and not the pointers themselves):

sort(v.begin(), v.end(), *_1 < *_2);

The Boost Lambda Library uses mad dope crazy fly operator overloading to make this possible.

 

[ithil]

The incredibly annoying thing about equals() is it's a conceptual overload of == that does something entirely different. This is stupid, and leads to even more bugs than the idiotic =/== confusion that you find in both C++ and Java. At least a compiler will warn you about the latter (hopefully). Overloading a global concept by operator is even more evil than overloading it by case.

A week ago, I was on a phone interview with a possible future employer, and since my current project at work is in Java, he asked me how I felt about the language. I responded "ehhh..." and tried to figure out how to phrase it. He laughed and said, "Don't bother answering; your pause was enough. I don't really like the language either."

 

[turbowrx]

You don't even know how to program. I don't think your opinions are worth anything.

I'm gonna try to make this quick. You're not even close. I'm a professional engineer working for the u.s. air force. I program, test software, and deal with a bunch of other stuff. I work with excellent programmers, researchers, and top notch scientists from around the world. I have two degrees and excellent grades. You're still an undergraduate trying to get where I already am, no real offense there as I was in your shoes not that long ago. Basically, I know something, as I'm sure you do. Now if you remember, I'm not bashing c++. I think it is a great language for many things, a matter of fact a very good scientific language. It is also probably the best language to use for anything 3d. None of that takes away from java's importance. It is very good for large database oriented networked systems. Anyway, I don't think you've seen my point in what i've been saying all thread long. They are both very good for what they were developed for, and many more things!

 

[Time2mod]

Java is an easy language to learn. That is the biggest reason why it is so popular. I just don't think there's anything wrong with that. It may not be a very intellectual language, but so what?

Programers can and should learn more than one language. But when a company wants a project done, they oftentimes want it done as quickly as possible, and the cross-platform thing is of course a must-have for many large scale systems.

So I guess that what I am saying is that some people here are going back and forth on whether or not Java is good or not are talking about totally different things. For the most part, they are all right.

There are some things in Java that seem kind of ass-backwards, and the library is not as robust as C++. But then Java ain't that old, either.

Also, there are some non-technical things that are being overlooked. The first language an individual likes usually happens to the first one s/he did well in. So to those that start with Java, they are going to tend to prefer that over "deeper" languages, and those who first learned in C++ are going to dislike the differences in the Java language that were added to make the language simpler. Heck, I had a professor who still swears by CoBOL, and not OO-COBOL either.

Let's just not forget that the purposes for the creation and use of Java are different from what a more CS-type of person views as important. It's true, you don't learn much in Java, excepting logical models (much like you could learn from, say, Oracle or even MS Access), but you can learn some design basics and you can learn somewhat what it means to program.

Java never was designed as a science-type language. I doesn't really have a prominent place in CS, and shouldn't. But it's great for business apps and large-scale systems that depend on the cross-platform ability. It's great for segueing(sp) legacy systems into something modern.

That said, I wish that I had learned C++ first. It would have made my academic experience much, much better.

 

[STL]

[Time2mod]
> Java is an easy language to learn

I think C++ would be just as easy to learn if people would teach it correctly.

> the cross-platform thing is of course a must-have for many large
> scale systems.

"Java isn't platform independent; it is a platform" - Bjarne Stroustrup

In contrast, C++ is truly cross-platform (and C ever so slightly more so).

> So to those that start with Java, they are going to tend to prefer
> that over "deeper" languages

People who start with Java stand a good chance of being corrupted.

> and those who first learned in C++ are going to dislike the
> differences in the Java language that were added to make the
> language simpler

Simpler? Ha. Try "castrated".

> But it's great for business apps

They said that about COBOL too.

> and large-scale systems that depend on the cross-platform ability.

Java isn't /used/ for large-scale systems.

You're stringing words together, but they don't mean anything.

 

[Time2mod]

You have a way of just restating the same stuff without actually saying anything.

Originally posted by STL
I think C++ would be just as easy to learn if people would teach it correctly.

But they don't, do they? And they don't because oftentimes they don't know it well enough to do so.

Originally posted by STL
"Java isn't platform independent; it is a platform" - Bjarne Stroustrup

That's just double talk, and you know it.

Originally posted by STL
In contrast, C++ is truly cross-platform (and C ever so slightly more so).

Well, yes, if and only if the source code is available to everyone, and everyone spends time doing their own compiling. See, this is what I'm talking about. You are only looking fromthe angle of a code monkey. To anyone else that is not what cross-platform means.

Originally posted by STL
People who start with Java stand a good chance of being corrupted.

More double talking.

Originally posted by STL
Simpler? Ha. Try "castrated".

*Sigh* You act as though C++ grew in only a few short years into the near perfection that you think it is. You still only think that because its what you like, and not the other way around.

It's just a little silly to expect Java to contain all the stuff that it doesn't need. These are two totally different things. And the programmers who make databases are totally different from the programmers who delve as deeply as you do. Nobady said that Java should be used to create heavy-duty engineering simulations. It's used for coding things that are much simpler, and can be rolled out much more quickly without having every programmer be a true engineer or computer scientist. Do you really believe that the world is so simple that a programmer is a programer is a programmer?

Originally posted by STL
They said that about COBOL too.

And your point here is what? "They" were wrong about CoBOL, so "they" are wrong about Java? These things aren't related. Java is not like CoBOL. At the time, CoBOL performed the function it claimed to quite well, but it was only that one general function.

Originally posted by STL
Java isn't /used/ for large-scale systems.

Yes, of course it is. That is one of the purposes. Large-scale, cross-platform (the non-geek definition of cross-platform) systems are written in Java rather frequently.

Originally posted by STL
You're stringing words together, but they don't mean anything.

Funny that you would end this particular post with that, because that's what I was thinking of your canned responses all the way down.

 

[STL]

[STL]
> I think C++ would be just as easy to learn if people would teach it correctly.

[Time2mod]
> But they don't, do they? And they don't because oftentimes they don't know it well
> enough to do so.

People are idiots.

[STL]
> "Java isn't platform independent; it is a platform" - Bjarne Stroustrup

[Time2mod]
> That's just double talk, and you know it.

No, it's not. It's a Key Insight.

[STL]
> In contrast, C++ is truly cross-platform (and C ever so slightly more so).

[Time2mod]
> Well, yes, if and only if the source code is available to everyone,
> and everyone spends time doing their own compiling.

No. If people wrote portable code, then binaries could be built from closed source for different platforms with minimal hassle. For example, there's no reason (other than sheer incompetency) why every multiplayer game can't have a GNU/Linux dedicated server. Source code doesn't have to be openly available for it to be portable.

Duh.

> Java is not like CoBOL.

It's very popular, and it sucks. Seems similar enough to me.

> You act as though C++ grew in only a few short years into the near
> perfection that you think it is.

On the contrary, I understand better than you do how old C++ is, and how it's still developing.

By Stroustrup's beard, look at the timeline.

1971: New B.
1973: Essentials of modern C complete.
1978: K&R1.
1979: C With Classes.
1983: C++ used internally at AT&T. STL (me) born.
1985: First commercial C++ implementation, and TC++PL1.
1987: GCC 1.0 released.
1989: [C standardized for the first time. K&R2.]
1991: TC++PL2, adding templates and exceptions.
1992: GCC 2.0 released.
1994: Template metaprogramming discovered.
1998: C++ standardized for the first time, incorporating the STL. TC++PL3. MSVC6 released, a bane that still haunts us to this day.
1999: GCC 2.95.2 released on October 24, 1999. No further GCC releases until March 16, 2001.
2000: STL (me) enters Caltech. Still no good C++ compilers exist.
2001: GCC 3.0 released on June 18, the first GCC that really supports the C++ standard. MSVC .NET 2001 released. Still sucks, since it doesn't have partial specialization.
2003: MSVC .NET 2003 released, the first MSVC that doesn't suck. TC1 released, fixing various boo-boos in the C++ standard.
2004: GCC 3.3.3 released on February 14, 2004. Covariant returns, part of the C++ Standard since 1998, are still not supported. (Support is promised in 3.4.0, coming Real Soon Now.) STL (me) gets bored with trashing a guy who doesn't know what he's talking about.
200x: C++ standardized for the second time. STL (me) rejoices. Template support is still being promised in an upcoming version of Java.
201x: Duke Nukem Forever released.
The next day: Java gets template support. LOUSY template support. The C++ standard was at this point back in 1998, and compilers were at this point in 2001. Sigh.

> It's just a little silly to expect Java to contain all the stuff that it doesn't need.

Apparently eunuchs don't need their balls. But I'm not going to give mine up any time soon. Similarly, you'll pry destructors from my cold, dead hands.

 

[Time2mod]

Okay, this is pointless.

turbowrx, what tests have you already thought of? I'm not sure if you want to do actual graphics or only the math part. Color me curious.

 

[turbowrx]

On to what the thread is for.

At the moment, the usual stuff. Compression/Decompression, Encryption/Decryption, audio/video conversion, possibly something 3d (which will be the hardest ), and I would like to make threading involved in some way. That way hyperthreading or multi-processor systems could be tested versus systems that don't employ those features.

Now, initially I was going to do it all using java, even knowing that c++ would work better for it, but I may do it in c++. That really isn't the point though as this is practice for me and if something really cool comes out of it then I would setup a website so that users could download it and compare results, similar to pcmark or 3dmark. What I want to build is something that users will want to use, which is why I'm looking for ideas. User interface ideas, test ideas, and anything else anyone wants to offer.

Having a 3d test isn't something that needs to be there right away, but it is something that I would like in the "suite" in the future. I think users would like some glitz and glamour.

 

[turbowrx]

This has nothing to do with this thread, but I am downloading mandrake linux 10.0 and it is taking forever on my cable connection. Those sites must be getting some serious activity.

 

[jblair]

[turbowrx]

> Having a 3d test isn't something that needs to be there right away, but it is something that I
> would like in the "suite" in the future. I think users would like some glitz and glamour.

There's two things wrong with this:

1. A 3D Test? People have games and renders to test their hardware on. Unless you are absolutely certain you are going to design your Benchmark like a game/3D rendering program would be, the results are not likely to portray real-world performance.
2. A "Suite" of benchmarks of different varieties. Sure, some people can program in all sorts of wacky fields without any problems - and you may be one of those some. However it's much simpler to produce a series of benchmarks that gauge one thing really well, you can specialise in the area and properly figure out how to work in it.
   [/list=1] > which is why I'm looking for ideas. User interface ideas, test ideas, and anything else anyone
   > wants to offer.
   
   Game-breaking homerun: Benchmark computer stability.

 

[Time2mod]

Originally posted by turbowrx
Compression/Decompression, Encryption/Decryption, audio/video conversion, possibly something 3d (which will be the hardest ), and I would like to make threading involved in some way. That way hyperthreading or multi-processor systems could be tested versus systems that don't employ those features.

Or even better, against each other. The multi-threading aspect is something I would love to see, both in the "crunching" side and the graphics side. It is my understanding that as the hyperthreading processors get better, then the percentage increase caused by the hyperthreading will increase as well. I've never seen any numbers to back that up though. It would be interesting to see (if it is true, of course) at what point hyperthreading would actually be able to pass or come within negligible difference of SMP systems.

I have a dual processor rig right now, and I don't always know when the SMP is going to be helpful.

As far as ideas go, the only thing I would say at the moment is that I would love to see something like this that is easily customizable, and not just for programmers. For example, could it be made easy for a user running the video compression test to be able to select his/her own video? This would kind of futureproof the benchmark, and allow different niches to kind of create their own standards for comparison using the same program.

 

[turbowrx]

[jblair]

> Benchmark computer stability

Ha ha. I'm on it right now.

The problem is, I don't think there will ever be great computer stability unless consolidation is done. What I mean by that is computers have to be able to do so many different things that having a machine that is going to be happy doing everything you want it to is just foolish. For instance, new operating systems have to be built to run all the old software, which means they can't get rid of dependencies that the old software needs. It's a cluttered mess down there. Things are bound to crash.

Good idea.

The 3d test, like the server tests that were suggested earlier, might be better off in a different benchmark that tests a graphics subsystem. The rest of the tests I'm looking at right now fit together just fine I think. The 3d test would not be a game as the time that would be needed to build that right now is just not there. That is going to be a tough one.

 

[turbowrx]

[Time2mod]

That is a good idea. It throws another twist into benchmarking. For instance, a standard file I would include to compress would be of regular size, but not too huge. If the user was allowed to select his or her own file, a huge file could be chosen which might tax the memory more, therefore allowing a computer, with more ram but a slightly slower proc, that was beaten in compressing a smaller file win when compressing a larger file due to increased memory.

As far as giving the benchmark the ability to do that, no problem. My guess is that the big name benchmarks don't do that due to the fact that the comparisons wouldn't mean anything if different files were being used. Obviously, you could check the file that was used and slot that user accordingly, but that would make the website very spread out and much more painful I think. Readers would not want to navigate all over the place looking at times.

On the other hand, the website could only allow the uploads of the default test. This would allow users some tweaking ability on their own and allow for better website management. A separate forum could be setup for users to discuss using the benchmark differently.

As far as threading goes, it will be in the bm. It just depends on how I want to integrate it.

 

[Whatsisname]

Since you are writing a benchmark, speed is of absolute importance. There is no choice other than C with the real shit in ASM if you are hardcore.

 

[STL]

> Since you are writing a benchmark, speed is of absolute importance.

That's so laughably wrong, I won't even bother.

 

[Whatsisname]

Yes because a buggy and slow, inefficiant process is the right way to create and effective benchmark. Way to go.

Also, you suck at the quotes.

Originally posted by STL
> Since you are writing a benchmark, speed is of absolute importance.

That's so laughably wrong, I won't even bother.

 

[FuzzyDonkey]

Originally posted by Whatsisname
Yes because a buggy and slow, inefficiant process is the right way to create and effective benchmark. Way to go.

Also, you suck at the quotes.

Most code in the real world is buggy, slow, and inefficient. Since a benchmark should test real-world performance, why would you concentrate on making the code efficient as hell? Efficient code isn't what matters. Calibration is what matters. The code only needs to be efficient enough so that you spend the majority of the time doing what you think the benchmark is supposed to measure. Don't use an encryption algorithm that constantly swaps to disk if you're wanting to measure the speed of the ALU on the processor. But it shouldn't matter if the code squeezes every last bit of performance out of the system, either.

 

[:LJ:]

Depends what you're trying to test. If you're testing the hardware performance, then you write the most efficient code possible which hammers the subsystem in question. If you're testing a database system, then you'll probably concentrate on hammering it with the most load in particular areas. If you're testing an enterprise software platform, you'll most likely test memory handling performance (not just bandwidth) and network performance.

Unlike STL, you have to realise that the end product should result from a fit-for-purpose design, rather than any personal prejudice.

And STL : Have you ever considered that there may be a *reason* you're getting rejected from these places? If you opened your mind and stopped being such a one-trick-pony, you'd probably get a lot further. And this goes double when you get out into the real world. If your project manager says "Do this in Java/VB/Python/whatever" and you say "No, I won't do that because that language is crap" then the next thing you'll be doing is standing in the job centre queue.