Image Image Image




Post new topic Reply to topic  [ 399 posts ]  Go to page Previous  1, 2, 3, 4, 5 ... 20  Next
Author Message
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Sat Feb 09, 2008 9:31 am 
Offline
PokerAI fellow
User avatar

Posts: 7731
Favourite Bot: V12
Timmy wrote:
(6) Spears port of 2+2 evaluator ordered test
I'll see about getting this running tomorrow. All Java's package/import stuff mixes me up.

(7) Timmy's PSim library (poker-eval wrapper) (The enumeration is quite different, but this is a real world test.)
Now we will benchmark with the hand: Ah 9s
We took 63.768 seconds to do 2,099,692,386 evaluations.
--- Hands per second: 32,927,054

UofA is still beating Indiana-1. :lol:


The Java stuff is a bit mixed up because many evaluators come with same named classes, e.g. class "Hand". package is basically a namespace, and it should be the same as the folder structure. If you import a package, you use all classes in it directly, like "new Hand()". If you import two packages that both contain "Hand", then you need to specify which hand you want to instanticate, hence the statements "new pokerai.game.eval.hand.spears.Hand()".

What means a real world test? OK, I wait to see some binaries, but the results looks promising if this is Java already.

UofA is still beating Indiana-1? OK, as I have Indiana-3 now that really doesn't matter :twisted: but I'm curious. What JDK are you using to compile it (java -version) and how much memory (although I don't see how memory could be related).

Timmy wrote:
While the speeds you guys are getting is impressive, there's something about imagining all that memory usage on VMs...


For real-time I still use (and plan to use) my evaluator (or one that uses less memory). Actually, I have evaluator usages only in very rare cases, in most of them I have some higher level caching. For example, I cache/precalculate all SAIE tables, which then I use for equilibrium calculations, so handeval is not used.

The spears evaluator however is of great value. We discussed in the forums how to precalculate the exact SAIEs for 3 players, I still haven't done it (exact, I have approximations). If that was going to take me 1 month, now it's going to take me 2 days on 1 CPU ... so such stuff.

OK, going to the dll thread now.

_________________
indiana


Top
 Profile E-mail  
 
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Sat Feb 09, 2008 3:38 pm 
Offline
PokerAI fellow
User avatar

Posts: 686
Location: Midwest, USA
Favourite Bot: N/A
By real world test I mean enumerating a real hand to actually calculate the equity (% of showdowns won). It's just not a fair comparison to your benchmark enumeration manner. I wouldn't say one's better than the other, they just can't be compared straight across. (in general, you're benchmark enumeration is faster because it doesn't worry about dead cards, etc.)

My java version is 1.6.0_02. I've been running the tests from the command line like so:

javac pokerai\game\eval\TestIt.java
java -Xms500m -Xmx700m pokerai/game/eval/TestIt

That gets your TestIt program running fine, but I'm still having trouble getting Spears's enumeration to run. I'll have another look later (unless someone wants to farther clue me in).

BTW, how do you define SAIE? (I know it stands for subjective all-in equity, but what does it mean, exacly?)


Top
 Profile  
 
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Sat Feb 09, 2008 4:14 pm 
Offline
PokerAI fellow
User avatar

Posts: 7731
Favourite Bot: V12
Hi

I though I'm already doing an enumeration, but looking once again, I ain't really. Feel free to propose enumeration, I'll include it, i.e. post please coding (hopefully short) that produces a hand that consist of 7 ordinal hand values (0 - 51). Note that the evaluators included currently have two different ways to produce ordinal values (i.e. mine and alberta/spears work with different ordinal values, that's why the cycles are slightly different).

Or by enumeration you mean something more in here, i.e. the enumeration is intermixed with the hand value calculation (that would make sense, as after you generate 2-3 hands you can precalculate stuff for them once)?

Next, all looks fine with your execution. The difference is really strange. Only other thing I can say is, I use 64bit windows, I ain't sure about the jkd if it's 32/64. But I reproduced the same results on a slower laptop, all win32, jdk5. anyhow - doesn't matter, as this particular contest (indiana-1 vs alberta) is somewhere for the bottom of the ranklist :)

SAIE - is subjective all-in equity, meaning you equity at showdown if there is no more betting. So AA vs KK preflop is simply the matchup value (82% for AA, 18% for KK excluding some tie %). Similarly you can make this more complex, by:
- looking for 3-way (and so on) matchups, i.e. equity of AA vs KK vs QQ
- looking for matchups given board and opponent belief vector (i.e. if I have AA on board TT7 and he has AT+, 77+) what's my showdown win%?

There are all things that pokersource do (but for this I'll write in the other thread).

_________________
indiana


Top
 Profile E-mail  
 
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Sat Feb 09, 2008 4:21 pm 
Offline
PokerAI fellow
User avatar

Posts: 1239
Favourite Bot: my bot
Timmy wrote:
That gets your TestIt program running fine, but I'm still having trouble getting Spears's enumeration to run. I'll have another look later (unless someone wants to farther clue me in).


I thought I'd try to help, but can't unzip eval.zip with winzip (bad archive) or jar (zulch). Is this an intelligence test? Have I failed?


Top
 Profile E-mail  
 
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Sat Feb 09, 2008 4:43 pm 
Offline
PokerAI fellow
User avatar

Posts: 7731
Favourite Bot: V12
spears wrote:
Timmy wrote:
That gets your TestIt program running fine, but I'm still having trouble getting Spears's enumeration to run. I'll have another look later (unless someone wants to farther clue me in).


I thought I'd try to help, but can't unzip eval.zip with winzip (bad archive) or jar (zulch). Is this an intelligence test? Have I failed?
I used Winrar. Just the extension was zip, winrar will usually open this. Now i fixed this, it is a ZIP which you can open with winzip.

Timmy, what problem do you have with spears evaluator? Note that in SixCardEvaluator.java you have to change this line:
private static final String ranksPath = ".\\scr-public\\pokerai\\game\\eval\\spears\\ranks";
to point to the file "ranks.", this is a relative path so it will also work if you start it from the parent of "scr-public" after unzip.

_________________
indiana


Top
 Profile E-mail  
 
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Sat Feb 09, 2008 7:57 pm 
Offline
PokerAI fellow
User avatar

Posts: 1239
Favourite Bot: my bot
Timmy wrote:
...but I'm still having trouble getting Spears's enumeration to run. I'll have another look later (unless someone wants to farther clue me in).


Does this help?

Code:
package pokerai.game.eval;

import pokerai.game.eval.spears2p2.StateTableEvaluator;

public class TestIt2 {
   
   public static void main(String[] args) {
       StateTableEvaluator.initialize();
         int[] handRanks = StateTableEvaluator.handRanks;

         int c0, c1, c2, c3, c4, c5, c6;
         int u0, u1, u2, u3, u4, u5;
         int numHands = 0;
         int handRank;
         int[]          handEnumerations = new int[10];
         int[][] equivalencyEnumerations = new int[10][3000];
         String[] handDescriptions = {"Invalid Hand", "High Card", "One Pair", "Two Pair", "Three of a Kind",
                                    "Straight", "Flush", "Full House", "Four of a Kind", "Straight Flush"};

         long time = System.currentTimeMillis();
         
         for (c0 = 1; c0 < 53; c0++) {
             u0 = handRanks[53 + c0];
             for (c1 = c0 + 1; c1 < 53; c1++) {
                 u1 = handRanks[u0 + c1];
                 for (c2 = c1 + 1; c2 < 53; c2++) {
                     u2 = handRanks[u1 + c2];
                     for (c3 = c2 + 1; c3 < 53; c3++) {
                         u3 = handRanks[u2 + c3];
                         for (c4 = c3 + 1; c4 < 53; c4++) {
                             u4 = handRanks[u3 + c4];
                             for (c5 = c4 + 1; c5 < 53; c5++) {
                                 u5 = handRanks[u4 + c5];
                                 for (c6 = c5 + 1; c6 < 53; c6++) {
                                    handRank = handRanks[u5 + c6];
                                    handEnumerations[handRank >>> 12]++;
                                    equivalencyEnumerations[handRank >>> 12][handRank & 0xFFF]++;
                                    numHands++;
                                 }
                             }
                         }
                     }
                 }
             }
         }
         
        
          time = System.currentTimeMillis() - time; // time given is start time
          long handsPerSec = Math.round(1000 / ((time*1.0)/ numHands));
      
          System.out.println(" --- Hands per second: [b]" + handsPerSec + "[/b], hands " + numHands);
          System.out.println();
         
         System.out.println("Enumerating hand frequencies and equivalency classes...");
         System.out.println("SEVEN-CARD POKER HAND FREQUENCIES AND EQUIVALENCY CLASSES\n");
         System.out.printf("    %-17s %15s %15s\n", "HAND", "FREQUENCY", "CLASSES");
         System.out.println("    -------------------------------------------------");

         int sumEquivalency;
         int numClasses = 0;
         for (int i = handEnumerations.length - 1; i >= 0; i--) {
            sumEquivalency = 0;
            for (int j = 0; j < equivalencyEnumerations[i].length; j++) {
               if (equivalencyEnumerations[i][j] != 0) sumEquivalency++;
            }
            numClasses += sumEquivalency;
            System.out.printf("    %-17s %15d %15d\n", handDescriptions[i], handEnumerations[i], sumEquivalency);
         }
         System.out.println("    -------------------------------------------------");
         System.out.printf("    %-17s %15d %15d\n", "TOTAL", numHands, numClasses);
      }

}


Top
 Profile E-mail  
 
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Sun Feb 10, 2008 7:17 am 
Offline
PokerAI fellow
User avatar

Posts: 686
Location: Midwest, USA
Favourite Bot: N/A
Spears, thank you very much. That worked perfectly. You probably saved me an hour of screwing around with it.

Anyway,
--- Hands per second: 110,657,204, hands 133,784,560

Very impressive. I updated my previous post too.


Top
 Profile  
 
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Sun Feb 10, 2008 12:25 pm 
Offline
PokerAI fellow
User avatar

Posts: 1239
Favourite Bot: my bot
Indiana - If you're looking for a quick pure Java evaluator that is not a memory hog you might consider porting Keith Rule's C# evaluator at codeproject. It reports 8 Mh/s on my machine and runs in < 10Mb.


Top
 Profile E-mail  
 
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Sun Feb 10, 2008 11:47 pm 
Offline
PokerAI fellow
User avatar

Posts: 7731
Favourite Bot: V12
I updated OP: I ported now Steve Brecher's hand evaluator, and it beated everything, with almost 30m hands per second. It also does not have any special memory requirements. Not sure if the test was somehow in favor of it, I also did a few more optimizations when encoding the hand in comparison to the spears2p2 - but despite this, all in all it looks as the best option so far (especially that it's more compact, and als includes razz and omaha evaluators).

(5) Spears port of 2+2 evaluator, http://pokerai.org/pf3
--- Hands per second: 22091946, hands 49353408, checksum 139477642848

should be 34: 34
(6) Steve Brecher HandEval, http://www.stevebrecher.com/Software/software.html
--- Hands per second: 28980275, hands 49353408, checksum 1480156650577328

I'm also going to look at and incorporate the second, ordered test in the benchmark in the next days, and also update the ZIP.

_________________
indiana


Top
 Profile E-mail  
 
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Wed Feb 27, 2008 2:10 pm 
Offline
Senior member
User avatar

Posts: 131
I am still trying to figure out: Is there any way of using any of these evaluators with AutoIT? Any suggestions gratefully received!

_________________
http://www.pokerspreadsheets.com


Top
 Profile E-mail  
 
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Wed Feb 27, 2008 3:47 pm 
Offline
Regular member
User avatar

Posts: 74
Favourite Bot: ...
longfields wrote:
I am still trying to figure out: Is there any way of using any of these evaluators with AutoIT? Any suggestions gratefully received!


If you can find some one to convert them to a C/C++ dll then you could use that, I don't know of any that would be usable with AutoIt, unfortunatly. But some one else might know of one. Anyone know the answer to this?

-acidie

_________________
I'm back... well almost, no, no I'm gone again.
I'm in ya shitz, haxoring ya boxz!
Nothing is secure, everything is permitted.


Top
 Profile  
 
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Wed Feb 27, 2008 7:01 pm 
Offline
PokerAI fellow
User avatar

Posts: 1239
Favourite Bot: my bot
longfields wrote:
I am still trying to figure out: Is there any way of using any of these evaluators with AutoIT? Any suggestions gratefully received!


I've never used AutoIt but I see from their web page you can call functions in dlls. So maybe you could use Timmy's dll http://pokerai.org/pf3/viewtopic.php?f=3&t=353 or make one from steve brechers C code. I think it would be hard and slow to call java from AutoIt.


Top
 Profile E-mail  
 
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Wed Feb 27, 2008 11:48 pm 
Offline
Senior member
User avatar

Posts: 131
OK, thanks for the hints. I'm going to continue this over at http://pokerai.org/pf3/viewtopic.php?f=3&t=353

_________________
http://www.pokerspreadsheets.com


Top
 Profile E-mail  
 
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Mon Mar 24, 2008 3:19 pm 
Offline
PokerAI fellow
User avatar

Posts: 7731
Favourite Bot: V12
(a)
Just a reminder to benchmark Ian Fellows' Evaluator from in this thread.

(b)
A PM I got on 2+2

Guy on 2+2 by PM wrote:
If the best evaluator there is Steve's Brecher, it seems that you still do not have compiled "the Beast" for Java. On my machine the C++ version of RayW code is running 208.387.165 evaluations per second, against the Steve Brecher's C version that does only 34.569.654 evaluations per second.
There were some posts in the 2+2 thread on porting RayW's code to Java, and it seemed to be worked after fixing one error.

_________________
indiana


Top
 Profile E-mail  
 
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Mon Mar 24, 2008 7:46 pm 
Offline
Regular member
User avatar

Posts: 76
Favourite Bot: Poki
indiana wrote:
(a)
(b)
A PM I got on 2+2

Guy on 2+2 by PM wrote:
If the best evaluator there is Steve's Brecher, it seems that you still do not have compiled "the Beast" for Java. On my machine the C++ version of RayW code is running 208.387.165 evaluations per second, against the Steve Brecher's C version that does only 34.569.654 evaluations per second.
There were some posts in the 2+2 thread on porting RayW's code to Java, and it seemed to be worked after fixing one error.



That's really interesting...
Are you going to look into this Indiana? If so, can you keep us posted? :)

Also, regarding the comparations you made for the 7-card hand evaluators, you didn't include Steve Brecher's hand evaluator in the source code for comparing? why?


Top
 Profile E-mail  
 
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Mon Mar 24, 2008 8:02 pm 
Offline
PokerAI fellow
User avatar

Posts: 7731
Favourite Bot: V12
I found some bugs in my handeval and in the process of fixing this I forgot to update the OP's code. In the meantime, you can download SB's evaluator from SB's website, and I also attach the SB code (in pokerai packages as part of the benchmark) to this post.

I'm going to followup on RayW code, but if anyone is willing to get the C++ it running from the 2+2 thread, would be appreciated. In the beginning of this thread spears posted about this, and it seems like RayW code works very fast for ordered hands, it works at about 2m for random hands (and we don't know yet what kind of test is the one that I started with, yet another to do/cleanup).

You see, there are todos/cleanups needed but as long as we maintain and accumulate working, faster and faster code in the OP, I think we are moving forward with this. I guess the corresponding 2+2 thread had much more power and momentum, but looking at its outcome (and postfactum) it's pretty messy and not really usefull for anyone beyond the people that participated in it.

_________________
indiana


Top
 Profile E-mail  
 
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Mon Mar 24, 2008 8:14 pm 
Offline
Regular member
User avatar

Posts: 76
Favourite Bot: Poki
True, it's much more clean to keep the 1st and the 2nd post of this thread with the most up-to-date, and tested, version of the best evaluators.
So keep up the good work.


Top
 Profile E-mail  
 
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Mon Mar 24, 2008 9:43 pm 
Offline
PokerAI fellow
User avatar

Posts: 7731
Favourite Bot: V12
Forgot to attach Steve's code (done now).

I also got followup PM on 2+2:

AdrianXXXX wrote:
This is the evaluator from Ray Wooten:
http://archives1.twoplustwo.com/showthreaded.php?Cat=0&Number=8652348&page=0&vc=1

It had a bug that only appeared when it gets ported to Java, so you have to correct that:
http://forumserver.twoplustwo.com/showthread.php?p=1487721#post1487721

mark007 was porting RayW's code to Java:
http://archives1.twoplustwo.com/showthreaded.php?Cat=0&Number=8747706&an=&page=0&vc=1

I didn't compared the performance without using enumeration, but later in the thread Steve Brecher posted a program to be able to use either Ray W's code or Steve's code, with enumeration or with random hands.

Regarding Steve's or Ray W's code, I think there is no doubt that if you evaluate Holdem the way to go is RayW's code (perhaps unless you are doing Montecarlo). And it is not close, for me Steve is evaluating at 23,56% the speed of RayW, so you can evaluate more than 4 times the speed with Ray's code. The point here is raw speed, not the ability to evaluate other games.

For other games there might be better evaluators, I didn't looked at them (and I'm currently not interested, I'm a holdem guy for now :- ).

Regards ...


Attachments:
stevebrecher.zip [9.3 KB]
Downloaded 110 times
Top
 Profile E-mail  
 
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Sun Mar 30, 2008 8:43 pm 
Offline
Senior member
User avatar

Posts: 223
Favourite Bot: N/A
indiana wrote:
I'm going to followup on RayW code, but if anyone is willing to get the C++ it running from the 2+2 thread, would be appreciated. In the beginning of this thread spears posted about this, and it seems like RayW code works very fast for ordered hands, it works at about 2m for random hands (and we don't know yet what kind of test is the one that I started with, yet another to do/cleanup).


You mean you didn't got Ray's code to work in C++, and you need help to compile it in C++? If this is the case I can help you out. Also, you can get the LUT file generated with the C++ code, and then use the file from Java, just have to use the same numbers for the cards.

Regards ...


Top
 Profile  
 
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Sun Mar 30, 2008 9:22 pm 
Offline
PokerAI fellow
User avatar

Posts: 7731
Favourite Bot: V12
Hi Adrian, welcome to the forums!

All of the evaluators we tried so far were in Java. There are 5 or 6 of them already, available for download. We made also one estimate comparison to poker-eval within the thread.

Can you attach to the thread a C++ evaluator of RayW, with a simple example? I can then test it with the test that we used for benchmarking all evaluators so far (I wouldn't say this test is perfect or super representative, we simlpy started with it. At some point, which we postponed several times, we can look for other kind of tests, the good thing is that we maintain all the time working versions of all benchmarked evaluators, so a full retest is not a big problem).

So if you can simlpy attach the C++ solution of Ray I can have a look at it in the next days. I still haven't tried that due to other issues.

Keep in mind all source we publish here
http://pokerai.org/pj2/code/eval.zip
(Steve Brecher is not yet in this ZIP, it's instead attached few posts above)

There is also one C evaluator in there (my personal translated to C for the purpsoe of comparing C vs Java). You can have a look at what it is currently benchmarking and even post an integrated test would be even better.

_________________
indiana


Top
 Profile E-mail  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 399 posts ]  Go to page Previous  1, 2, 3, 4, 5 ... 20  Next


Who is online

Users browsing this forum: Google and 8 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to: