Image Image Image




Post new topic Reply to topic  [ 399 posts ]  Go to page Previous  1, 2, 3, 4, 5, 6, 7, 8 ... 20  Next
Author Message
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Fri Apr 04, 2008 3:37 am 
Offline
Senior member
User avatar

Posts: 223
Favourite Bot: N/A
indiana wrote:
I simply measured how much memory is free before and after the initialization. So 140M is taken. If these arrays are kept staticly, then they will allocate the required memory. If they are not needed we have to clean them up, I can try this out. But to be honest 124 vs 140 isn't a big deal.

I've just browsed the code posted here, it's seems you have together the generation of the LUT table and the loading of the table.
The only load table program is in this post (C++ code), it should require the 124MB.
indiana wrote:
First, the other evaulators are not irrelevant, 140Mb (or 125) of memory is a serious thing. If you run VMWare setups for yur bots you can usually run them with 200M in total. If only the hand evaluator takes 150 of this, this is an issue. I think RayW did a great job, but Steve B (joiking or not) did that too :). And btw, saving/loading results from the files system like the current impl might also be problematic (or at least not nice) for bot environments.

If you look, I was refering to the other evaluators that do support holdem only. This is not the case of Steve Brecher's program. So without wanting to hurt your feelings, I still consider that the others are irrelevant, because there is no reason to use them, as they are "dominated" by other algorithms. Either in terms of pure speed, or speed/memory, or speed/game support.
Does this mean I don't want them listed in the OP? Of course not, there is a great service done to the community by letting know where not to spend time. And I don't say this in a ironic way, it took me time to compile evaluators that I later insta-discarded.
The programs might be very relevant to the author, and they still can have valuables ideas that we can not see without looking at depth in the code. For example on the 2+2 thread there were two evaluators that were dominated, and still brought good ideas that were new to me, combinadics and sorting networks. But if the ideas are difficult to spot in a thread full of explanations like the 2+2 thread, they'll be harder to spot looking at the code.

indiana wrote:
Second thing is, remember that one key difference between this thread and 2+2 (e.g.) is that I put in focus working and availble for immediate download source code. Theoretical discussions are of course great, but at the end if the spears port of RayW works for holdem and not for stud or omaha low, it runs for holdem.

So, does Indiana X, Cactus Kev, or University of Alberta work for stud or omaha low?
This was my point, by not updating the later evaluators that are "dominated" you were unfair with this one, but I still think you didn't on pourpose (otherwise I wouldn't ever bother to reply :-) ).

indiana wrote:
I understand very well the power of lookup tables, I have to admit that I did not realized that you can put in whatever values an external evaluator will calculate.

So, you didn't understood what this LUT was, you have to admit that too :-)
And I have to admit, that it stroke me yesterday by reading your post when you said it works only for holdem, I didn't realized that before, hell, this is a data structure to efficiently store values that we are using as an evaluator, but we can use this data structure to store whatever game of poker of 7 cards or less we want!!!

indiana wrote:
And that the bug with 5 and 6 cards can be only in the java port. But this is what anyone looking at OP can download now.
Adrian, if you are comfortable with C++ I can add and maintain also the cpp version of it in the archive. I'll try to get this up and running.

I keep every hand evaluator I've got, but I really think you should maintain the C++ version, but for the community and for you, you said "It's better if it's Java" so it was a contest open to other languages, I think you doing not good to the community and your self by not having the C++ evaluator there.

And a thing that is very relevant, I did a backup this week of my machine after a lot of months :-)


With all that said, I think that Steve Brecher should be beaten hands down even with low memory requirements, but It's just my feelings. If your VMs want low memory, I would try Paul Senzee's modification of Cactus Kev code (the 7 cards version, not the 5 cards), or Juk of York code, I think they might be a faster low memory requirement evaluator.

BTW, why is it better for bots to have multiple VMs? Doesn't this complicate things even further later when consolidating money from different accounts? (I think you have multiple accounts, one for each VM, right?)

TIA & Regards ...


Top
 Profile  
 
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Fri Apr 04, 2008 6:07 am 
Offline
Senior member
User avatar

Posts: 223
Favourite Bot: N/A
indiana wrote:
(2) Steve Brecher HandEval, http://www.stevebrecher.com/Software/software.html
--- Hands per second: 34659212, hands 133784560, checksum 3895822421317564, total time: 3860
Remarks: No heavy memory usage, compact implementation and support of other poker variants

I've just realized that you might not be using the proper loop optimizations for Steve Brecher's code, and it resulted that this was true, you can speed it up by building "key" partially.
This is Steve's implementation of the partial state generation, as usual in C:
Code:
//deck is a 52-element array of __int64
//with one bit set per element
int main(void)
{
    clock_t timer;

    int ix1, ix2, ix3, ix4, ix5, ix6, ix7;
    Hand_T board1, board2, board3, board4, board5, board6;
    int handTypeSum[9];

    Init_Hand_Eval();
    InitDeck();
    for (ix1 = 0; ix1 < 9; ix1++)
        handTypeSum[ix1] = 0;

    timer = clock();

    for (ix1 = 0; ix1 < 46; ix1++) {
        board1 = deck[ix1];
        for (ix2 = ix1+1; ix2 < 47; ix2++) {
            board2 = board1 | deck[ix2];
            for (ix3 = ix2+1; ix3 < 48; ix3++) {
                board3 = board2 | deck[ix3];
                for (ix4 = ix3+1; ix4 < 49; ix4++) {
                    board4 = board3 | deck[ix4];
                    for (ix5 = ix4+1; ix5 < 50; ix5++) {
                        board5 = board4 | deck[ix5];
                        for (ix6 = ix5+1; ix6 < 51; ix6++) {
                            board6 = board5 | deck[ix6];
                            for (ix7 = ix6+1; ix7 < 52; ix7++) {
                                handTypeSum[Hand_7_Eval(board6 | deck[ix7]) >> 24]++;
                            }
                        }
                    }
                }
            }
        }
    }

    timer = clock() - timer;

    for (ix1 = 0; ix1 < 9; ix1++)
        printf("%d\n", handTypeSum[ix1]);
    printf("seconds = %.2f\n", (float)timer/CLOCKS_PER_SEC);
}

Regards ...


Top
 Profile  
 
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Fri Apr 04, 2008 8:57 pm 
Offline
PokerAI fellow
User avatar

Posts: 7731
Favourite Bot: V12
OK Adrian,

You continue to terrorize me but apparently this is (at least half of it) very producive and leads to good improvements and fixes! :)

On the questions:

1)
Memory: I fixed OP to 128 Mb memory. I in fact used calculcations of 1MB = 1,000,000 bytes as HDD vendors do, instead of the correct 1024 bytes = Kb. I guess this was the reason from the start.

2)
That everything except Spears/RayW and Steve B are irrelevant: Might be but this is not the point. I will maintain them in the ranklist and the code in the OP. Otherwise if I don't do it, new people that join this thread will ask "what about Alberta/Aaron evaluator?", etc. This is originally a competition so all that made it to the finish are shown :) An the fact that I skipped the remarks for everyone after #1 and #2 is exactly that I would expect that the people who want to use code they take one of these two for actual use, and look at the rest, and the thread, for ideas only.

3)
I always knew what LUT is. I have solved at least 1000 computer science problems with LUT. What I didn't know (and maybe still don't know) is if the current Spears/RayW is easily adaptable for other poker variants, i.e. its internal design - is decoupled enough and if one can easily extend it (e.g. get it runnign for another variant by just replacing the initialization and not touchign the rest).

4)
Keeping C++: I have a bit of a problem here, as I plan to extend later on this public code library with more things (EHS^2 e.g.), and it's all java, I suppose it's not a good idea to mix it up with C++ in the same archive. But I can just include all the source code (includnig the test) in a separate subfolder until late decide what to do with it (this is todo, if you like to help me attach again zip with your latest fixes I can retest, offer for download, and publish remarks about the C++ in the OP)

5)
Cactus Kev - I think I tried it, it's slower than Steve.

6)
Paul Senzee's modification of Cactus Kev code - attach it, I'll test & include it! :)

7)
Yuk's code - attach it, I'll test & include it! :)

8)
Why is it better for bots to have multiple VMs? -- I don't want to discuss this thing in this thread. Just mentioned it as requirement, and it is a valid one. Less memory is always a valid requirment. 128M is not much, but also not little.

9)
I did a retest with the suggestions of how to speed up further Steve's code. It's 10% faster, which is not much (frmo 37m to 40m evaluations per second). Not sure if I did it all right:

Code:
    for (h1 = 0; h1 < 52; h1++) {
      key1 = key | (0x1L << (h1));
      for (h2 = h1 + 1; h2 < 52; h2++) {
        key2 = key1 | (0x1L << (h2));
        for (h3 = h2 + 1; h3 < 52; h3++) {
          key3 = key2 | (0x1L << (h3));
          for (h4 = h3 + 1; h4 < 52; h4++) {
            key4 = key3 | (0x1L << (h4));
            for (h5 = h4 + 1; h5 < 52; h5++) {
              key5 = key4 | (0x1L << (h5));
              for (h6 = h5 + 1; h6 < 52; h6++) {
                key6 = key5 | (0x1L << (h6));
                for (h7 = h6 + 1; h7 < 52; h7++) {
                  key7 = key6 | (0x1L << (h7));
                  sum += HandEval.hand7Eval(key7);
                }}}}}}}

_________________
indiana


Top
 Profile E-mail  
 
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Fri Apr 04, 2008 10:25 pm 
Offline
Senior member
User avatar

Posts: 223
Favourite Bot: N/A
indiana wrote:
1) Memory: I fixed OP to 128 Mb memory. I in fact used calculcations of 1MB = 1,000,000 bytes as HDD vendors do, instead of the correct 1024 bytes = Kb. I guess this was the reason from the start.

Once the array was generated, you don't further need the ID array. This was my previous suggestion, you can see how RayW uses the array in here, but it is pretty obvious, just load the array from disk and loop the cards.

indiana wrote:
2) That everything except Spears/RayW and Steve B are irrelevant: Might be but this is not the point. I will maintain them in the ranklist and the code in the OP.

Didn't I said that they should be kept in my opinion in the OP and this was a service to the community to know where not to go by seeing the low performances?

indiana wrote:
5) Cactus Kev - I think I tried it, it's slower than Steve.

I think this too, but didn't tested the 5 cards evaluator with the 7 cards version (included in the same code).

indiana wrote:
6) Paul Senzee's modification of Cactus Kev code - attach it, I'll test & include it! :)

This is Paul Senzee's modification Cactus Kev¡s 5 cards version of the evaluator. You can also use it as a 7 card evaluator. I did compiled, it's almost 3 times faster than Cactus Kev for 5 hands (didn't checked 7 hands).
And this one is the complete 7 cards version of Paul Senzee, but now re-reading his post it looks like it uses even more memory than RayW. But, didn't compiled it.

indiana wrote:
7)
Yuk's code - attach it, I'll test & include it!

Seems Juk of York is using even more memory :-(, didn't compiled it neither.

indiana wrote:
9) I did a retest with the suggestions of how to speed up further Steve's code. It's 10% faster, which is not much (frmo 37m to 40m evaluations per second). Not sure if I did it all right:
Code:
    for (h1 = 0; h1 < 52; h1++) {
      key1 = key | (0x1L << (h1));
      (code continues here).

I don't know Java, but it seems you did made a small mistake. Steve represents cards as a 64 bits integer with only 1 bit set. And a 7 cards hand is a 64 bits integer with exactly 7 bits sets. So what he is doing in the code I've sent, he is constructing partially the hand, instead of doing all the ors at the inner loop he takes ors out of the inner loop. So In the first one you have to assign the card without doing the xor, as he does in the C code:
Code:
board1 = deck[ix1];

Anyway, the way to double check it is to count the hand values in the test, as everyone does in the 2+2 thread.
BAD!! = 0
High Card = 23294460
Pair = 58627800
Two Pair = 31433400
Three of a Kind = 6461620
Straight = 6180020
Flush = 4047644
Full House = 3473184
Four of a Kind = 224848
Straight Flush = 41584
Total Hands = 133784560

I use this and also for a debugging version I compare all the combos against a benchmark evaluator, just to be sure every single number is perfect (I also check that all the combos with the same cards but different order map to the same state in the case of the LUT).

So, as you are doing one or operation more than what you need, you should probably have a better improvement than 10%, but still almost 10%.

And now you got to a point were 10% is really a lot, improvements now don't start to get the same kind of rewards, I still have a couple of improvements to implement by I don't expect much from the major part of them. 30% more will be heaven counting all of them :-)

Regards ...


Top
 Profile  
 
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Fri Apr 04, 2008 11:16 pm 
Offline
Senior member
User avatar

Posts: 223
Favourite Bot: N/A
indiana wrote:
Code:
                for (h7 = h6 + 1; h7 < 52; h7++) {
                  key7 = key6 | (0x1L << (h7));
                  sum += HandEval.hand7Eval(key7);
                }

Oh, and you neither need key7, now the hand is key6 | card 7. You can see in Steve's code there is no such assignment, you directly calculate the value.
Regards ...


Top
 Profile  
 
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Fri Apr 04, 2008 11:34 pm 
Offline
PokerAI fellow
User avatar

Posts: 7731
Favourite Bot: V12
OK, I'll reread again, but the highest prios I see (I'm gonna try tomorrow) is
- utilizing the check for correctness (good idea, I won't have it as part of the benchmark, but running it once is good idea).
- cleanup a bit the benchmark files
- some microoptimizations in S.B.'s code

_________________
indiana


Top
 Profile E-mail  
 
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Mon Apr 07, 2008 2:34 am 
Offline
Senior member
User avatar

Posts: 223
Favourite Bot: N/A
Hi,

I have a few questions for the people that are really using 7-cards evaluators, I'm just programming a seven cards evaluator for now.

1) Now that there is here a faster evaluator like 5 times the previous one, are you using it now?
2) Did the performance improvement allowed your programs to tackle more complex problems and did this resulted in a better bot in the case of people programming bots?
3) If you had evaluators for holdem, 5, 10, 100 or 1000 times faster than RayW, what is you think you could do with them? Do you think they will make a difference beside the performance? If this evaluators would be for specific scenarios, like for example a scenario may be 10 or less players having hands or hand ranges pre-flop, and the evaluator gives you directly the odds for each player, which scenarios would you be interested?

TIA & Regards ...


Top
 Profile  
 
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Mon Apr 07, 2008 10:08 am 
Offline
PokerAI fellow
User avatar

Posts: 1239
Favourite Bot: my bot
Adrian20XX wrote:
I have a few questions for the people that are really using 7-cards evaluators, I'm just programming a seven cards evaluator for now.

1) Now that there is here a faster evaluator like 5 times the previous one, are you using it now?
2) Did the performance improvement allowed your programs to tackle more complex problems and did this resulted in a better bot in the case of people programming bots?
3) If you had evaluators for holdem, 5, 10, 100 or 1000 times faster than RayW, what is you think you could do with them? Do you think they will make a difference beside the performance? If this evaluators would be for specific scenarios, like for example a scenario may be 10 or less players having hands or hand ranges pre-flop, and the evaluator gives you directly the odds for each player, which scenarios would you be interested?


I'm not using an evaluator at all right now but have done in the past and expect to in the future. If this disqualifies me from participating, or if my response doesn't answer your questions, just ignore me.

On the flop, ignoring your own hand, there are only 211876 evals for all seven card hands. SB's evaluator will get through those in 7ms, so there is no need to use 120MB to reduce this to 2ms.

The higher level problem that I want to solve is this. For any number of players and given a probability distribution of those players hands, what are the chances of any hand held by any player winning at showdown? A degenerate form of this question is "what are my chances of holding the best hand at showdown", but the more general question helps answer the questions "what are my opponents holding?" and "what do my opponents think their opponents are holding?"

Also see http://pokerai.org/pf3/viewtopic.php?f=3&t=496


Top
 Profile E-mail  
 
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Mon Apr 07, 2008 2:55 pm 
Offline
Senior member
User avatar

Posts: 223
Favourite Bot: N/A
spears wrote:
I'm not using an evaluator at all right now but have done in the past and expect to in the future. If this disqualifies me from participating, or if my response doesn't answer your questions, just ignore me.

On the flop, ignoring your own hand, there are only 211876 evals for all seven card hands. SB's evaluator will get through those in 7ms, so there is no need to use 120MB to reduce this to 2ms.

The higher level problem that I want to solve is this. For any number of players and given a probability distribution of those players hands, what are the chances of any hand held by any player winning at showdown?

Thanks I appreciate the answer, actually I wanted to know what people that do real programs beside evaluators do, as I said I'm improving my evaluator.

The case you want to know is a refinement of what I called scenario 1, the evaluation of 10 or less players pre-flop to do the river, including weights. I was going to say the additional requirement of adding weights should not make much difference, but I've realized that for the LUT now every single operation that must be made for all combos can make significant differences. Anyway I think the adition of weights should take the same "order of magnitude".

I'm gonna look at the other thread, and post there a comment that I think might be usefull there, thanks again.

Regards ...


Top
 Profile  
 
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Mon Apr 07, 2008 7:18 pm 
Offline
Senior member
User avatar

Posts: 223
Favourite Bot: N/A
indiana wrote:
4)
Keeping C++: I have a bit of a problem here, as I plan to extend later on this public code library with more things (EHS^2 e.g.), and it's all java, I suppose it's not a good idea to mix it up with C++ in the same archive. But I can just include all the source code (includnig the test) in a separate subfolder until late decide what to do with it (this is todo, if you like to help me attach again zip with your latest fixes I can retest, offer for download, and publish remarks about the C++ in the OP)

I've just re-formulated my benchmark a bit.
At the end is the code I'm currently using, it is self-contained, does not require any .h file, and only requires the file HandRanks.dat.
Note that this is the C version (it is a pure C program), that keeps the pointers so Java guys have to modify the savings of intermidiate states.

So, this is what I did and why:
1) I now run the tests ten times instead of one. I do this to generate a more stable timing, as now the timings are really really short.
2) I've took away all the code and just loaded the table from disk. I did this because the generation may put things in the processor cache that can help the evaluation later, so in this way I can compare two LUT generated in different ways and know the performance in the evaluation is not influenced by the generation. The performance with this in my machine is 205.222.519 hands / second.
3) I removed the hand type evaluation and did like Indiana an addition, because the calculation of hand type is not needed (we are benchmarking pure 7 cards evaluation speed) and now every single operation can affect the speed. The performance with this in my machine is 429.347.112 hands / second.
4) I did the evaluation also with an exact fit for the hand range of the loops of the cards (for example, c1 had previously the range 1-52 and now it has 1-46 that is the real range). I did this because as I said every single additional operation can affect the evaluation. The performance with this in my machine is 464.691.073 hands / second.

Code:
   Seconds   #Hands   Hands / Second   Relative Speed
Hand Types, Not Exact Fit   0,6519   133784560   205.222.519   44,16%
Sum Only, Not Exact Fit   0,3116   133784560   429.347.112   92,39%
Sum Only, Exact Fit   0,2879   133784560   464.691.073   100,00%

Perhaps someone may think that this things are not relevant, but I think they are because we do care about the final evaluation time, not if the code was optimized and the traversal of the hand space not. And of course later if the code needs to perform additional operations for each combos (like updating the time each player wins), it will be performed and this will add an aditional constant time to all the evaluators, so in this way we isolate the pure evaluator performance.

And as a final note and shocking and totally unexpected for me, using iSum+=p6[c7] is faster than using iSum=p6[c7]

Code:
#include <stdio.h>               // Includes definitions for printf(), fprintf(), getchar() and stderr
#include <time.h>               // Includes definitions for clock_t, clock() and CLOCKS_PER_SEC

// #define ADRIAN_LUT
#define SUM_ONLY
#define EXACT_FIT

#define NUMBER_OF_TESTS 10


#ifdef ADRIAN_LUT
#define BASE_ADDRESS 0
#define INPUT_FILE_NAME "AdrianLUT.dat"
#define ARRAY_SIZE (32380116+1)
#define HAND_TYPE_SHIFT 24
#define HAND_TYPE_START 0
#else
#define BASE_ADDRESS 53
#define INPUT_FILE_NAME "HandRanks.dat"
#define ARRAY_SIZE (32487834)
#define HAND_TYPE_SHIFT 12
#define HAND_TYPE_START 1
#endif

#ifdef EXACT_FIT
#define END_CARD1 47
#define END_CARD2 48
#define END_CARD3 49
#define END_CARD4 50
#define END_CARD5 51
#define END_CARD6 52
#else
#define END_CARD1 53
#define END_CARD2 53
#define END_CARD3 53
#define END_CARD4 53
#define END_CARD5 53
#define END_CARD6 53
#endif
#define END_CARD7 53

int p[ARRAY_SIZE];

int main()
{
   clock_t clock_tTimer;
   FILE *fin;
   int c1, c2, c3, c4, c5, c6, c7;
   int *p1, *p2, *p3, *p4, *p5, *p6;
   int i;
#ifdef SUM_ONLY
   int iSum=0;
#else
   int iHandTypes[10]={0,0,0,0,0,0,0,0,0,0};
#endif

   fin = fopen(INPUT_FILE_NAME, "rb");
   if (!fin) {
      fprintf(stderr, "Can not open file %s\n", INPUT_FILE_NAME);
      return -1;
   }
   fread(p, ARRAY_SIZE*sizeof(*p), 1, fin);
   fclose(fin);

   clock_tTimer=clock();
   for (i=0;i<NUMBER_OF_TESTS;i++) {
      for (c1 = 1; c1 < END_CARD1; c1++) {
         p1 = p+p[BASE_ADDRESS+c1];
         for (c2 = c1+1; c2 < END_CARD2; c2++) {
            p2 = p+p1[c2];
            for (c3 = c2+1; c3 < END_CARD3; c3++) {
               p3 = p+p2[c3];
               for (c4 = c3+1; c4 < END_CARD4; c4++) {
                  p4 = p+p3[c4];
                  for (c5 = c4+1; c5 < END_CARD5; c5++) {
                     p5 = p+p4[c5];
                     for (c6 = c5+1; c6 < END_CARD6; c6++) {
                        p6 = p+p5[c6];
                        for (c7 = c6+1; c7 < END_CARD7; c7++) {
#ifdef SUM_ONLY
                           iSum=p6[c7];
#else
                           iHandTypes[p6[c7] >> HAND_TYPE_SHIFT]++;
#endif
                        }
                     }
                  }
               }
            }
         }
      }
   }
   clock_tTimer=clock()-clock_tTimer;

#ifdef SUM_ONLY
   printf("OK: 7 Cards benchmark SAVING intermidiate states SUM ONLY, clocks: %d, seconds: %2.4f, sum %d\n\n", clock_tTimer, (float) clock_tTimer/CLOCKS_PER_SEC, iSum);
#else
   if (iHandTypes[HAND_TYPE_START]==NUMBER_OF_TESTS*23294460 && iHandTypes[HAND_TYPE_START+1]==NUMBER_OF_TESTS*58627800 && iHandTypes[HAND_TYPE_START+2]==NUMBER_OF_TESTS*31433400 && iHandTypes[HAND_TYPE_START+3]==NUMBER_OF_TESTS*6461620 && iHandTypes[HAND_TYPE_START+4]==NUMBER_OF_TESTS*6180020 && iHandTypes[HAND_TYPE_START+5]==NUMBER_OF_TESTS*4047644 && iHandTypes[HAND_TYPE_START+6]==NUMBER_OF_TESTS*3473184 && iHandTypes[HAND_TYPE_START+7]==NUMBER_OF_TESTS*224848 && iHandTypes[HAND_TYPE_START+8]==NUMBER_OF_TESTS*41584) {
      printf("OK: 7 Cards benchmark SAVING intermidiate states CALCULATING HAND TYPES, clocks: %d, seconds: %2.4f\n\n", clock_tTimer, (float) clock_tTimer/CLOCKS_PER_SEC);
   } else {
      printf("ERROR in Number of Hand Types\n");
      for (i=0;i<10;i++) {
         printf("Array Place %d, Value: %d\n", i, iHandTypes[i]);
      }
   }
#endif
}

Regards ...


Top
 Profile  
 
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Mon Apr 07, 2008 9:53 pm 
Offline
PokerAI fellow
User avatar

Posts: 7731
Favourite Bot: V12
Wow.

Can you give me the generator so I can test as well on my side? My serialized format is different.

Otherwise on the perfromance hints, I experimented with the Java version:
1- using exact fit improved the performance from about 140m to abotu 147m/second
2- running the test 10 times in a row instead of one, reduced (?!?) the performance from 147m to about 133m/sec!
3- ok, i already read from the disk instead of doign a generation, so nothing to do here
4- you seem to removed the additon? i.e. you don't have += in your code

Seems your machine is faster, or the low level optimizations in C helped really helped you. Hm, 140m vs 450 is appealing, although at this ponit it seems to me that any practical calculation will eliminate the benefots of the hand evaluator ultra speed.

On usages (your previous question) - I use it for ...:
1) Ok same as spears said, hand strength. See EHS/EHS^2 thread. I will retest retest my EHS implementation with Ray's evaluator.
2) 3, 4, 5 etc players preflop matchups

Realtime I mostly haev a higher level caches, so I stick to low memory evaluator (SB). For offline, I'm with RayW. For other games SB.

Actually, I might start a separate thread, for the fastest EHS, EHS^2 implementation, if there is any interest.

_________________
indiana


Top
 Profile E-mail  
 
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Tue Apr 08, 2008 1:11 am 
Offline
Senior member
User avatar

Posts: 223
Favourite Bot: N/A
indiana wrote:
Wow.
Can you give me the generator so I can test as well on my side? My serialized format is different.

The numbers I've put there are for RayW's generator. At the end it saves the array in the original C code. It is in the original RayW's post.
If you are using RayWs code, you are using the same generator (btw, a generator for me is different from another generator if they generate a different table, not if they generate the same table different).
As you may have noticed, I have my own generator, but as I've said it seems I've tackled the wrong problem and I've improved the generation from 36 seconds on my machine with RayW's code, to 0.8 seconds in my machine with my own generation code. But, my LUT later performs 20% slower than RayW's, so I'm not sure what is going to happen with that. Plus, it has over 10 thousand lines of code (really, not joking).
If you want I can send you for now the LUT, it has the same representation for eval results that Steve Brecher for now (in fact it is my current benchmark to check the results are correct), but I will later move to a smaller integer representation 90% sure.

indiana wrote:
2- running the test 10 times in a row instead of one, reduced (?!?) the performance from 147m to about 133m/sec!

With the load only code? May be the cache from the generation might affect your performance. With these speeds the run time is very sensitive, that's why I've switched to 10 iterations. Previously I was doing 10 generations and selecting the best of 10, now I do 5 evaluations of 10 times each one, and select the best one.
Also not doing the sums against the assignment is in my machine slowing the evalution like 10%.

I still have the open question if someone think we can get more stables results using a VM, or any other method.

indiana wrote:
4- you seem to removed the additon? i.e. you don't have += in your code

I forgot, it was the last test I've done, and this ubelivable aumented the evaluation time. So if we want to be closer to the pure evaluation performance, the assignment does more work at least in C than the sum. Don't ask me why, it is true altough I can not explain.

indiana wrote:
Seems your machine is faster, or the low level optimizations in C helped really helped you.

The saving of the pointers might in the intermediate states might help the C version here.

indiana wrote:
Hm, 140m vs 450 is appealing, although at this ponit it seems to me that any practical calculation will eliminate the benefots of the hand evaluator ultra speed.

On usages (your previous question) - I use it for ...:
1) Ok same as spears said, hand strength. See EHS/EHS^2 thread. I will retest retest my EHS implementation with Ray's evaluator.
2) 3, 4, 5 etc players preflop matchups

Realtime I mostly haev a higher level caches, so I stick to low memory evaluator (SB). For offline, I'm with RayW. For other games SB.

Actually, I might start a separate thread, for the fastest EHS, EHS^2 implementation, if there is any interest.


Sure, there is always interest for code, altough as you can see I'm stucked badly with the evaluator, now I have a personal thing with RayW I need to solve :-)

BTW, what about the speed improvement from the current numbers from 5, 10, 100 and 1000 times? What would you do with that? Something usefull or just run the same code faster? I thing there is room to not less than between 5 and 10 times if I blur the lines between the generation and the evaluation.

TIA & Regards ...


Top
 Profile  
 
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Tue Apr 08, 2008 2:33 am 
Offline
Senior member
User avatar

Posts: 223
Favourite Bot: N/A
indiana wrote:
2) 3, 4, 5 etc players preflop matchups

Here I share some tips for bluring the lines between the evaluator and the enumeration.
I have a very clear idea of what I need to do here, I'm delaying now I don't know why, perhaps because I can not take the beat from RayW without fighting it to the end.

At the end, this evaluator without enumerator will be dead against the one that blurs the line.

These are a lot of posts from Andrew Prock, the author of PokerStove.
I outline the main points.
Andrew Prock wrote:
Agnostic vs. Zealous
The zealous approach usually requires more code, more code maintenance, and tends to be more brittle (susceptible to bugs). The *size* of the zealous solution is much larger in several dimensions. On the other hand, the zealous solution can often be faster, since optimizations can be applied across the conceptual levels that the agnostic solution separates ...
All of the public poker code I've read, and most of the poker software I've read about produces agnostic solutions to the problem of evaluating hands. While this is good for providing a general low level library, most hand evaluations occur in very specific contextual circumstances. In particular, many of these solutions treat evaluation and enumeration as two separate problems...
The point is that the general state of poker evaluators has been relatively stagnant for the last half decade. It is time to actually sit down and do the hard work of producing quality evaluators which take a more zealous approach. Hand evaluators may be a rather dry aspect of poker, but it's either come up with more clever ways of doing complex enumeration/evaluation tasks, or wait 90 years for computers to "catch up"...
P.P.S. I'd like to give credit where credit is due. The underlying ideas behind this kind of evaluator didn't originate with me. I was inspired by ancient posts made by Steven R. Jacobs regarding video poker evaluation/enumeration algorithms...

I really recommend the previous article to everyone.
Andrew Prock wrote:
Post in the 2+2 evaluators thread
PokerStove can do about 2 billion evaluations per second in the optimized Enumeration code. In the unoptimized code, it does about 10 million evals per second when enumerating...
If you know what you are trying to calculate, you can do it significantly faster by taking the whole calculation into consideration instead of just relying on fast 7 card hand evaluators...

This shows that the 10 times faster is way behind what we can expect from a combined solution.
Andrew Prock wrote:
Another Post in the 2+2 evaluators thread.
There's a distinct difference between lumping evals together and a fast generic evaluator. You can certainly do evals rather quickly if you are able to "share results" across cases. Under specific circumstances this is a fairly straightforard process. In general, it's rather difficult

Andrew Prock wrote:
Yet Another Post in the 2+2 evaluators thread.
PokerStove can do over 1.5 billion evals/sec by grouping evals together. But that's different from a generic evaluation

Now if someone wants to get more ideas, I would suggest to dig for the posts that Andrew suggests of Steven R. Jacobs on Video Poker.

Unfortunately he has too many posts as you can see, and I didn't had the patience to look at all of them, neither the luck to find the ones that are relevant to this using the logical search keywords (they are very old posts, may be the "poker language" for source code functions was different from the current one, altough I don't think so).

I would really love to see the relevant posts from Steven Jacobs.

If I don't recall wrong, I think Darse Billings at some point wrote something about this.

Regards ...


Top
 Profile  
 
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Mon Apr 21, 2008 7:17 pm 
Offline
Senior member
User avatar

Posts: 223
Favourite Bot: N/A
Why don't you add the Java code from Andrew Prock (the author of PokerStove) to the evaluators thread?
I didn't checked it yet.
http://www.pokerstove.com/download/jpoker.tar.gz
Regards ...


Top
 Profile  
 
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Mon Apr 21, 2008 7:34 pm 
Offline
PokerAI fellow
User avatar

Posts: 7731
Favourite Bot: V12
Tested that, and included in the archive. With a very unfair 7-cycles evaluation, creating a Hand object on every iteration (which I took from jpoker examples), the results were:

( 9) Pokerstove jpoker, http://www.pokerstove.com/download/jpoker.tar.gz
--- Hands per second: 4420292, hands 133784560, checksum 4270106005454684, total time: 30266

Code:
  public static void testPokerstove() {
    int[] hand = new int[7];
    long time = System.currentTimeMillis();
    long sum = 0;
    int h1, h2, h3, h4, h5, h6, h7;
    for (h1 = 0; h1 < 52; h1++) {
      for (h2 = h1 + 1; h2 < 52; h2++) {
        for (h3 = h2 + 1; h3 < 52; h3++) {
          for (h4 = h3 + 1; h4 < 52; h4++) {
            for (h5 = h4 + 1; h5 < 52; h5++) {
              for (h6 = h5 + 1; h6 < 52; h6++) {
                for (h7 = h6 + 1; h7 < 52; h7++) {
                  // need to have all the assignments here, as the array is sorted internally
                  hand[0] = h1;
                  hand[1] = h2;
                  hand[2] = h3;
                  hand[3] = h4;
                  hand[4] = h5;
                  hand[5] = h6;
                  hand[6] = h7;
                  pokerai.game.eval.jpoker.Hand h = new pokerai.game.eval.jpoker.Hand(hand);
                  sum += h.evaluate();
                }}}}}}}
    print(sum, time, 133784560, 8);
  }

_________________
indiana


Top
 Profile E-mail  
 
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Mon Apr 21, 2008 7:43 pm 
Offline
PokerAI fellow
User avatar

Posts: 7731
Favourite Bot: V12
Ok, after a bit of a modding (getting read of Hand object at all, but simply exposing a static method in HandEvaluator that accepts int[]), I now reached:

( 9) Pokerstove (Andrew Prock's) jpoker, http://www.pokerstove.com/download/jpoker.tar.gz
--- Hands per second: 8031732, hands 133784560, checksum 4270106005454684, total time: 16657

So nothing so great, but thank you for pointing me to yet another HandEvaluator that beats Indiana's :-)

_________________
indiana


Top
 Profile E-mail  
 
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Tue Apr 22, 2008 3:35 am 
Offline
Regular member
User avatar

Posts: 76
Favourite Bot: Poki
I just tried the PerformanceTest2.java in my laptop.

So, for testRayW(), i was getting an "java.lang.OutOfMemoryError: Java heap space" exception, until I started running it with:
java -Xmx300m ...

With -Xmx256m it got the same Exception.

Onto the results:
It takes about 40seconds (!!!) to load the evaluation tables.
Afterwards it evaluates 74m hands per sec.

I think I'll keep using SB's code for now. Indiana, also I'd like to know if you have finished the "little" changes for the "partial state generation", as I seem to have lost myself in between the posts by you and Adrian.


Top
 Profile E-mail  
 
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Tue Apr 22, 2008 7:33 am 
Offline
PokerAI fellow
User avatar

Posts: 7731
Favourite Bot: V12
altair wrote:
Onto the results:
It takes about 40seconds (!!!) to load the evaluation tables.
Afterwards it evaluates 74m hands per sec.


If you ran it for a first time it generates the ealuation tables and loads them. Second run will only load them. On my side it takes less than 2 seconds to load them (about 128 Mb file). If you persistently get bad results, post your JDK version, OS, hardware conf etc ...

altair wrote:
I think I'll keep using SB's code for now. Indiana, also I'd like to know if you have finished the "little" changes for the "partial state generation", as I seem to have lost myself in between the posts by you and Adrian.


Partial state generation? I even cannot recall what little changes, I have to look back.

_________________
indiana


Top
 Profile E-mail  
 
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Tue Apr 22, 2008 10:19 pm 
Offline
Senior member
User avatar

Posts: 223
Favourite Bot: N/A
altair wrote:
Indiana, also I'd like to know if you have finished the "little" changes for the "partial state generation", as I seem to have lost myself in between the posts by you and Adrian.

This is what was called the partial states generation, it only affects the evaluation, it's on page 5 of this thread (I can't find the way to link to the specific part).

Code:
   for (c0 = 1; c0 < 53; c0++) {
      u0 = HR[53+c0];
      for (c1 = c0+1; c1 < 53; c1++) {
         u1 = HR[u0+c1];
         for (c2 = c1+1; c2 < 53; c2++) {
            u2 = HR[u1+c2];
            for (c3 = c2+1; c3 < 53; c3++) {
               u3 = HR[u2+c3];
               for (c4 = c3+1; c4 < 53; c4++) {
                  u4 = HR[u3+c4];
                  for (c5 = c4+1; c5 < 53; c5++) {
                     u5 = HR[u4+c5];
                     for (c6 = c5+1; c6 < 53; c6++) {
                        handTypeSum[HR[u5+c6] >> 12]++;
                        count++;
                     }
                  }
               }
            }
         }
      }
   }

But, you can also take out of this the count++ and directly compute outside the number of evaluations done, and you can also remove the shift that is used in order to get the hand type if you want to.
C and C++ coders can save pointers and get another performance improvement, but it seems it is not your case.

Regards ...


Top
 Profile  
 
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Tue Apr 22, 2008 10:58 pm 
Offline
PokerAI fellow
User avatar

Posts: 7731
Favourite Bot: V12
Still don't get what is "partially" generated here.

Currently my benchmark of RayW is this one, which looks the same as Andrian's:

Code:
 
    sum = 0;
    for (h1 = 1; h1 < 47; h1++) {
      u0 = handRanks[53 + h1];
      for (h2 = h1 + 1; h2 < 48; h2++) {
        u1 = handRanks[u0 + h2];
        for (h3 = h2 + 1; h3 < 49; h3++) {
          u2 = handRanks[u1 + h3];
          for (h4 = h3 + 1; h4 < 50; h4++) {
            u3 = handRanks[u2 + h4];
            for (h5 = h4 + 1; h5 < 51; h5++) {
              u4 = handRanks[u3 + h5];
              for (h6 = h5 + 1; h6 < 52; h6++) {
                u5 = handRanks[u4 + h6];
                for (h7 = h6 + 1; h7 < 53; h7++) {
                  sum += handRanks[u5 + h7];
                  /*
                  handRank = handRanks[u5 + h7];
                  handEnumerations[handRank >>> 12]++;
                  numHands++;
                  equivalencyEnumerations[handRank >>> 12][handRank & 0xFFF]++;
                  */
                }}}}}}}

_________________
indiana


Top
 Profile E-mail  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 399 posts ]  Go to page Previous  1, 2, 3, 4, 5, 6, 7, 8 ... 20  Next


Who is online

Users browsing this forum: Google and 8 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to: