Image Image Image




Post new topic Reply to topic  [ 399 posts ]  Go to page Previous  1 ... 12, 13, 14, 15, 16, 17, 18 ... 20  Next
Author Message
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Thu May 21, 2009 12:02 pm 
Offline
PokerAI fellow
User avatar

Posts: 7731
Favourite Bot: V12
Not sure.

_________________
indiana


Top
 Profile E-mail  
 
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Thu May 21, 2009 6:05 pm 
Offline
Senior member
User avatar

Posts: 223
Favourite Bot: N/A
birchy wrote:
indiana wrote:
Remark-4: Still not fast enough? Read the thread, where Adrian20XX claimed 450m+evaluatons per second with his port to C. To be eventually included in the distribution.

Any news on this? Is the source code available?

Indiana called it a port, but I don't consider that a port. This was only a skeleton to benchmark the performance once the table was already generated.

It basically uses performance optimizations, nothing super genius:
* Pointers to save the states, not ints
* Not counting the hands number inside the loop
* Exact fit for the cards (like for example when evaluating 7 cards hands, c1 goes until 47 only, not until 53).

An old version it's in page 14.
I substituted now from that version the clock for a High Precision clock to measure the time more reliable, and also I'm running it 100 times (but, I select the MINIMUM, not the mean as you are doing, because the minimum is closest to the real performance, the mean is affected for when the maching automatically starts to do other things without my control).

Now for sum only in my machine it's running at 412,854 million hands / second, while SteveB is running at 35,610 Millions Hand/second (an old comparative of C and C++ evaluators performance on my machine is at page 25).


Top
 Profile  
 
 Post subject: Re: My old 7 card evaluation code in VB6
PostPosted: Thu May 21, 2009 10:24 pm 
Offline
Senior member
User avatar

Posts: 223
Favourite Bot: N/A
birchy wrote:
An update for future reference...
Have benchmarked this VB6 version against my own C port HERE and n3m3sys' official C++ benchmark HERE.
Code:
n3m3sys' C++ benchmark: 4577ms
Birchy's VB6 evaluator: 6458ms
Birchy's C evaluator:   4341ms
Therea are some standard modifications that you should make there to your code (they are made in the attachment).
In my machine, your code as posted runs at 5694ms. With these standard modifications, it runs at 3510ms.
These are the modifications I've made:
* Saving the partial states
* Not updating the total evals within the loop
* Using exact fit for the cards, c1 has to go until 52-6 for example
* Using a shift to calculate the hand type from the hand value, not multiple ifs

Plus, you can use sum only instead of computing hand types now, to get a better benchmark of the "pure evaluation" (that does not include computing hand types).
birchy wrote:
I totally agree that ALL ranking functions do exactly the same thing - they take a 7 card hand and give you a relative value. Obviously some are faster than others but there shouldn't be any significant differences.
Obviously, I don't agree at all. If these simple modifications reduced the time to 61%, different algorithms with different ideas to optimize evaluations will come with gigantic performance differences, and that's why we have orders of magnitudes of differences between different evaluators.


Attachments:
BirchyEvalOptimized.zip [2.79 KB]
Downloaded 136 times
Top
 Profile  
 
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Sat Jul 18, 2009 10:52 pm 
Offline
Senior member
User avatar

Posts: 451
Favourite Bot: gimmick
I was reading this topic and i failed to understand a property of the benchmark used, that i think should be important enough.

When you say an evaluator can test X million hands per second, how the hands feed to the evaluator are calculated?

I think there may be big differences between performance of the algorithm between random hands and adjacent hands, depending on what kind of operation the algorithm does.

What setting has been used for these benchmarks?

EDIT: I found out the answer. I think there should be available a benchmark for monte carlo evaluation as full enumeration is mostly infeasible in real time botting.


Top
 Profile E-mail  
 
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Sun Jul 19, 2009 2:21 pm 
Offline
Senior member
User avatar

Posts: 360
Favourite Bot: Zander
Johanson wrote:
I think there should be available a benchmark for monte carlo evaluation as full enumeration is mostly infeasible in real time botting.

I made this point in an earlier post on this thread. The speed kiddies like to pretend nobody mentioned Monte Carlo because all the fast evaluators are highly optimized for full evaluations. They mostly fall flat on their face when fed random hands.

_________________
http://www.bespokebots.com


Top
 Profile  
 
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Sun Jul 19, 2009 2:37 pm 
Offline
Senior member
User avatar

Posts: 451
Favourite Bot: gimmick
From what i read around i understand that when accounting monte carlo evaluation pokersource evaluator suddenly becomes a first liner. Having done a little experimentation, do you agree with that?

It seems to me that for poker botting full enumeration is simply useless, except probably for headsup. Am i right here?


Top
 Profile E-mail  
 
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Sun Jul 19, 2009 5:08 pm 
Offline
Senior member
User avatar

Posts: 360
Favourite Bot: Zander
Everyone has their own preference, but i would much rather do a quick Monte Carlo and use the remaining time for more important stuff like player profiling, etc...

_________________
http://www.bespokebots.com


Top
 Profile  
 
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Tue Jul 21, 2009 12:32 am 
Offline
Senior member
User avatar

Posts: 223
Favourite Bot: N/A
birchy wrote:
The speed kiddies like to pretend nobody mentioned Monte Carlo because all the fast evaluators are highly optimized for full evaluations. They mostly fall flat on their face when fed random hands.
What a horrible statement, it's even disrespecting. The people that ignore it is simply because they are not interested in Monte Carlo.


Top
 Profile  
 
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Tue Jul 21, 2009 10:14 am 
Offline
PokerAI fellow
User avatar

Posts: 1239
Favourite Bot: my bot
Adrian20XX wrote:
birchy wrote:
The speed kiddies like to pretend nobody mentioned Monte Carlo because all the fast evaluators are highly optimized for full evaluations. They mostly fall flat on their face when fed random hands.
What a horrible statement, it's even disrespecting. The people that ignore it is simply because they are not interested in Monte Carlo.

Agreed. It's arrogant and unnecessary. And if I'm one of the "speed kiddies" its not even true: I raised the issue in the thread a long time ago.


Top
 Profile E-mail  
 
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Tue Jul 21, 2009 11:01 am 
Offline
PokerAI fellow
User avatar

Posts: 2342
Favourite Bot: My next one
So let's turn this unnecessary statement into a potentially useful question. Should I need to use few, unordered evaluations, are some of the evaluators better suited at that than others, or would any would do anyway since they are so fast ?


Top
 Profile E-mail  
 
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Tue Jul 21, 2009 12:01 pm 
Offline
PokerAI fellow
User avatar

Posts: 1239
Favourite Bot: my bot
It would need a bit of work to answer you properly and right now I'm short of time to do that. But as far as I remember, and refering to benchmarks in OP
- (1) RayW/404URL/Spears @143 Me/s very sensitive to hand order
- (2) Steve Brecher with partial state generation @ 38 Me/s sensitive to hand order
- (3) Steve Brecher without partial state generation @ 35Me/s is not sensitive to hand order.
- (3') Poker Eval @37Me/s is not sensitive to hand order.
- (4) RayW/404URL/Spears @ 24Me/s is not sensitive to hand order, but could be speeded up without a lot of effort or introducing order sensitivity.

Throwing completely random hands at the evaluators isn't really realistic either. In many cases you will have the board cards set, sometimes the private cards, and often it's just the future cards that are being changed through the evaluation.


Top
 Profile E-mail  
 
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Tue Aug 25, 2009 1:29 pm 
Offline
Junior member
User avatar

Posts: 23
Favourite Bot: mine
I know that Pauaul Sneeze's Hand Evaluation Indexer has been beaten from the nested Lookup implementation.
But I made some improvments to his indexer-function which I wanted to post here.
Of course this dosent beat the faster Hand-Evaluators, but could have some potential, because almost ALL actions inside are not related to each other and could thererfor be parallelised. Im trying to implement this on a CUDA Graphics Device as soon as I get my Card.

My approach is very similar to Paul Sneezes version.

I have tables for diffeent lookup-tasks, such as the bitcount and an index array that enumerates n choosen bit in 16 bits (could be downgraded to hold only n choosen in 13 bits as the rest is not needed).

My version takes the hand as a 64 bit integer where 13bit in each word represent the rank of the Card. The remaing 3 bits are unused and have to be set to zero.

Also I have added offset tables for 5 and 6 card evaluation as well.

In the attachment you will find the lookuptables and the code for my indexer.

Hope this is of some use for somebody here.


Attachments:
File comment: Slightly faster indexer for Paul Sneezes approach on hand evaluation. Also indexers for 5 card and 6 card hands.
Index.zip [142.37 KB]
Downloaded 121 times
Top
 Profile E-mail  
 
 Post subject: Re: 2+2 Hand Evaluator in Java
PostPosted: Mon Sep 07, 2009 2:35 pm 
Offline
PokerAI fellow
User avatar

Posts: 1613
Location: Austria
Favourite Bot: Marvin
Corrected my port of the lookup-based hand evaluator from the 2+2 forums to Java/Meerkat.

It uses a generated HandRanks.dat file and thus doesn't have to convert the cards to a different format. It uses plain Meerkat Cards and Hands.

The evaluator works correctly for 5-, 6-, and 7-card hands. It should be pretty fast, unfortunately my machine is not really up to heavy benchmark tests (lots of company stuff interfering with the CPU).

If you use it, you have to increase the Java heap size (usually using -Xms512M -Xmx512M or something like that). The evaluator is a singleton, but still reads the 128M file into memory once...

If the file doesn't exist, it's created, which takes up to a minute (on my machine, a 2 GHz CPU).

I'd love to see someone give it a performance test.

Edit: On this 2 GHz laptop CPU, 1,000,000 random lookups take ~4 seconds, but with hand conversion from Indiana format int[] to Meerkat API Card (see CorrectnessTest). It should be way faster if working with Meerkat Cards directly, and on a reasonably fast CPU.


Attachments:
File comment: LUT hand evaluator for 5/6/7 card hands
common.zip [44.78 KB]
Downloaded 124 times
Top
 Profile E-mail  
 
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Mon Sep 28, 2009 8:40 am 
Offline
New member
User avatar

Posts: 1
Favourite Bot: Gort
I have written a Java poker hand evaluator (attached) that I believe combines the best aspects of the Steve Brecher and Ray W (2+2) evaluators. For enumerations, the evaluator runs faster than Steve Brecher's HandEval and is in the same ballpark as the 2+2 evaluator, while using far less memory. For random hand evaluations, the evaluator again runs faster than HandEval while easily outperforming the 2+2 evaluator.

The evaluator uses a generated finite state transducer whose states are generated from the ranks of the cards of the hand. The suits of the cards are considered separately in order to keep the number of states of the transducer to a minimum. In this regard the evaluator is similar to the "DAG evaluator" Moritz Hammer wrote (http://www.pst.ifi.lmu.de/~hammer/poker/handeval.html) except that the implementation is far more efficient. The states of the FST are written as primitive-type arrays (Java 'char' arrays are used to save space) rather than being represented as Java objects as in the Hammer implementation.

In this respect, the evaluator is similar to the Ray W "2+2" one (with the FST arrays corresponding to the 2+2 lookup tables) expect that my 7-card evaluator arrays use only 626K of memory, far less than the 128M used by the 2+2 evaluator, and just under two times the memory used by the lookup tables of Steve Brecher's HandEval.

5-card, 6-card and 7-card evaluation methods are provided in the methods FastEval.eval5, FastEval.eval6 and FastEval.eval7. Each of the eval methods passes the CorrectnessTest in pokerai/game/eval.

The first time the evaluator is run, the FST arrays are generated, serialized and saved as files. Subsequent evaluations bypass this step and just read the serialized files. The HandFST generator requires a working correct evaluator in order to generate the FST equivalence class results. Steve Brecher's evaluator (with the 6-card straight-or-flush fix) is included in the jar file in the attachment for this use.


For an enumeration performance test, I modified pokerai.game.eval.PerformanceTest2 to add a seven-card testKlaatu method (shown below).

Here are the performance results on my machine (a 2.13GHz Core 2 Duo with 2G of RAM, Windows XP, Java 1.6.0_16) for seven-card evaluation. I modified the PerformanceTest2 harness to run each test ten times first to allow the Java JIT to do its thing before running the test whose times are reported. In my opinion this gives far more accurate results than reporting times from an initial test run from the first few seconds of the java virtual machine execution -- unless that's only as long as you plan on running your code that uses the hand evaluator.

Java -client (default) hands/second (enumeration test):

RayW (2+2): 214055296

Steve B: 18532284

Steve B "PSG": 26104304

Klaatu FastEval: 29729902

Java -server hands/second (enumeration test):

RayW: 203939878

Steve B: 29624570

Steve B "PSG": 29422600

Klaatu FastEval: 125855654


Notes:

Using the Sun JDK -server option to the java command results in much faster execution for all evaluators (except the 2+2 one, which is memory-constrained). IMO this is the more appropriate Java option for programs that execute for more than a few seconds. (On Windows, if you try using the -server option and get an error message about 'jvm.dll' not being found, try putting the Java executable directory first in your path or specifying the absolute path to the 'java' executable, or copying the file jvm.dll from C:/Program Files/<rel>/java/jre/bin/server/jvm.dll to C:/Program Files/<rel>/jre6/bin/server/jvm.dll.) With the -server option, the "Steve B" and "Steve B PSG" times are similar. I believe this is because the -server JIT does more aggressive optimizations, effectively rewriting the vanilla Steve B test code to be similar to the Steve B "PSG" test code. Also note that the FastEval -server performance is much better than the -client performance (at least after sufficient warmup).

Finally, I wrote some code to test random hand evaluation speed. The time reported in each result is the test time minus the time it takes just to generate the random cards (and also to sum the value of the first card of each run, in order to ensure that the JIT didn't optimize away the entire loop). The RayW 2+2 evaluator was, as expected much, much slower:


Java -client (default) hands/second (random access):

RayW (2+2): 4134281

Steve B: 10458063

Klaatu FastEval: 15496668



Java -server hands/second (random access):

RayW (2+2): 4252423

Steve B: 10223903

Klaatu FastEval: 22768670



For reference, here is the new PerformanceTest2 enumeration test:


Code:
import klaatu.eval.FastEval;
//
  public static void testKlaatu() {
    long sum = 0;

    int h1, h2, h3, h4, h5, h6, h7;

    long time = System.currentTimeMillis();
    sum = 0;

    for (h1 = 0; h1 < 52; h1++) {
      for (h2 = h1 + 1; h2 < 52; h2++) {
        for (h3 = h2 + 1; h3 < 52; h3++) {
          for (h4 = h3 + 1; h4 < 52; h4++) {
            for (h5 = h4 + 1; h5 < 52; h5++) {
              for (h6 = h5 + 1; h6 < 52; h6++) {
                for (h7 = h6 + 1; h7 < 52; h7++) {
                  sum += FastEval.eval7(h1, h2, h3, h4, h5, h6, h7);
                }}}}}}}


    //*

    print(sum, time, 133784560, 9);
  }


If someone is interested in porting the Fasteval evaluator to some C variant, it would be fairly easy to write Java code to spit out the generated FST arrays into a file as C static const data and then translate just the FastEval eval<n> methods (which are trivial).

All of the attached code is in the public domain and comes without warranty, express or implied.
Use as you like.


Attachments:
FastEval.zip [50.69 KB]
Downloaded 392 times
Top
 Profile E-mail  
 
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Thu Oct 01, 2009 6:39 pm 
Offline
PokerAI fellow
User avatar

Posts: 7731
Favourite Bot: V12
Nice post klaatu, I will link it in the OP in the todo section. I don't have the time to test and integrate this right now.

_________________
indiana


Top
 Profile E-mail  
 
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Fri Oct 23, 2009 1:46 pm 
Offline
Senior member
User avatar

Posts: 458
Location: Now in the mighty PolarBoar variant!
Favourite Bot: ...
All I've seen in this thread are Java and C/C++ evaluator, but has anyone written/ported an evaluator to C# yet?
If not I'll probably port one of available Java solutions to C# and post it here some time soon.


Top
 Profile  
 
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Fri Oct 23, 2009 2:38 pm 
Offline
PokerAI fellow
User avatar

Posts: 7731
Favourite Bot: V12
Polar, if you do that, please also port the functional correctness/performance test, so we can include it in the benchmarking.

_________________
indiana


Top
 Profile E-mail  
 
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Fri Oct 23, 2009 3:51 pm 
Offline
PokerAI fellow
User avatar

Posts: 2342
Favourite Bot: My next one
PolarBear wrote:
but has anyone written/ported an evaluator to C# yet?

Check this thread.


Top
 Profile E-mail  
 
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Sun Oct 25, 2009 8:26 pm 
Offline
Senior member
User avatar

Posts: 458
Location: Now in the mighty PolarBoar variant!
Favourite Bot: ...
I've found this project:

http://www.codeproject.com/KB/game/poke ... aldoc.aspx

It has nice features and was designed to be quite fast.


Top
 Profile  
 
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Mon Oct 26, 2009 1:36 am 
Offline
PokerAI fellow
User avatar

Posts: 2342
Favourite Bot: My next one
PolarBear wrote:
http://www.codeproject.com/KB/game/pokerhandevaldoc.aspx

I knew I had forgottent something... There's an updated version of that library with more code here : http://www.codeproject.com/KB/game/More ... ysis1.aspx

and here : http://www.codeproject.com/KB/game/More ... ysis2.aspx


Top
 Profile E-mail  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 399 posts ]  Go to page Previous  1 ... 12, 13, 14, 15, 16, 17, 18 ... 20  Next


Who is online

Users browsing this forum: Google and 7 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to: