Image Image Image




Post new topic Reply to topic  [ 399 posts ]  Go to page Previous  1 ... 10, 11, 12, 13, 14, 15, 16 ... 20  Next
Author Message
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Fri Apr 10, 2009 10:10 am 
Offline
PokerAI fellow
User avatar

Posts: 7731
Favourite Bot: V12
Hm, you got me here. I can't fully remember why I named it like that. I think this was not RayW code, but different (slower) LUT based approach - with several different LUTs for straights, etc. Spears, do you remember?

_________________
indiana


Top
 Profile E-mail  
 
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Fri Apr 10, 2009 11:16 pm 
Offline
Senior member
User avatar

Posts: 223
Favourite Bot: N/A
indiana wrote:
Hm, you got me here. I can't fully remember why I named it like that. I think this was not RayW code, but different (slower) LUT based approach - with several different LUTs for straights, etc. Spears, do you remember?

It seems you got that from the second page of this thread.
indiana wrote:
OK,
Did a retest. Spears 2+2 port runs 23m/sec, i'll do some more tests for if it isn't buggy.
(1) Spears port of 2+2 evaluator, http://pokerai.org/pf3
--- Hands per second: 23401331, hands 49353408


Both the code that you have in directories "twoplustwo" and "spears2p2" seem to be a port of RayW's code.
And, if they do generate the same table as I guess is the case, then the results should be the same (you can have different results if you do different types of evaluations of course like sum only or calculating hand types, or saving states). But that would be a result of the traversal only.
If I recall correctly, when this thread was starting for a while you were convinced that RayW's code was not good, may be that was when you had the test run as you did in the beggining? (i.e. not ordered, so RayW's performance goes down there).
Anyway, I'm pretty sure the first post in the thread is at least misleading regarding what you call "Spears port of 2+2 evaluator" performance.


Top
 Profile  
 
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Sat Apr 11, 2009 9:39 am 
Offline
PokerAI fellow
User avatar

Posts: 7731
Favourite Bot: V12
I've ported once a LUT based approach (wuith 5 independant LUT tables) which wan't good. But in that my starting point was not 2+2 at all, I think I found it as a C code somewhere on google code, etc.

I just reviewed the thread. So Spears says he used RayW ideas and code plus some helper classes from elsewhere for this port. That's what he says:

Spears wrote:
Just for the record, the state table evaluator java was lifted from http://forumserver.twoplustwo.com/showt ... age=0&vc=1
and http://forumserver.twoplustwo.com/showt ... age=0&vc=1 and then I added my port of Cactus Kev's 5 card evaluator. This credit is shown in the source.


I changed it now (based on that) to:
Quote:
(4) Spears first adaptation of RayW/2+2/Cactus Kev evaluators, http://pokerai.org/pf3

_________________
indiana


Top
 Profile E-mail  
 
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Sat Apr 11, 2009 4:30 pm 
Offline
Senior member
User avatar

Posts: 223
Favourite Bot: N/A
indiana wrote:
I changed it now (based on that) to:
Quote:
(4) Spears first adaptation of RayW/2+2/Cactus Kev evaluators, http://pokerai.org/pf3
I think it could be better to rename it to something like:
Quote:
(4) Spears adaptation of 2+2 user "404url" port to Java of RayW LUT hand evaluator.
Basically because that might flag that the inefficiency might be due to "404url" port (so that could be a bad starting point), at the same time give credit to "404url" for what is due (even the inefficiency port if that turns to be the case), and mantain consistency when referencing to RayW's starting code that is the same for both (4) and (1) (otherwise a non-informed user could also think there are two different RayW's evaluators).
Regards ...


Top
 Profile  
 
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Sat Apr 11, 2009 6:50 pm 
Offline
PokerAI fellow
User avatar

Posts: 7731
Favourite Bot: V12
OK, if spears confirms that 404url's Java code was what he used (and nothing else outside the 2+2 thread), then I'll rename it exactly like proposed. Or you (adrian) are already sure in that?

_________________
indiana


Top
 Profile E-mail  
 
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Sat Apr 11, 2009 8:21 pm 
Offline
Senior member
User avatar

Posts: 223
Favourite Bot: N/A
indiana wrote:
OK, if spears confirms that 404url's Java code was what he used (and nothing else outside the 2+2 thread), then I'll rename it exactly like proposed. Or you (adrian) are already sure in that?
I'm not sure.
Just that I feel that the OP is now not clear, if it take you that gathered together all the code a couple of days to figure out what was the difference and you are still not sure, imagine a guy that reads it for the first time.
BTW, I also think that there are some things also that are not clear:
indiana wrote:
(1) Spears adaptation of RayW LUT hand evaluator, http://pokerai.org/pf3, http://forumserver.twoplustwo.com
--- Hands per second: 142779680, hands 133784560, checksum 1528972275512, total time: 937
(4) Spears first adaptation of RayW/2+2/Cactus Kev evaluators, http://pokerai.org/pf3
--- Hands per second: 23783922, hands 133784560, checksum 450334402748, total time: 5625
And then the directories are named spears2p2 and twoplustwo, it's not clear what directory corresponds to (1) and (4).

You know I'm a C guy and not a Java guy, but my feeling is that both generate the same table, is that right (just looking at the tables sizes from the code)? I would start from there, because if they both generate the same table I wouldn't even classify them as different, the difference will be in the traversal as I said in the previous post.


Top
 Profile  
 
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Sat Apr 11, 2009 10:57 pm 
Offline
PokerAI fellow
User avatar

Posts: 7731
Favourite Bot: V12
Yeah, you are right, that's confusing with the packages. If one look into the benchmark examples he'll eventually find out, but better to say which is it. I would wait a bit for spears (if he recalls what he did :), and then update all as discussed.

_________________
indiana


Top
 Profile E-mail  
 
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Mon Apr 13, 2009 3:05 am 
Offline
Senior member
User avatar

Posts: 223
Favourite Bot: N/A
A few things as I've revisited the performance of some evaluators and I've noticed before I forget.

Remember I'm compiling them in C, while Indiana is doing it in Java.
And, Indiana seems to be using bigger integers for the sum than me, I'm using the same integer type as the hand value uses (except at Cactus Kev and Paul Senzee where I'm using 32 bits integers). So he might be doing integer conversions for the sum and that will hurt the faster evaluators (personally I prefer to use the native type of the evaluator as we don't introduce the overhead of the integer conversions and so we have a better indication of the raw evaluation performance).

SUM only SAVING is the only option I really care about, because not saving is ineficcient and hand type computation does introduce an extra overhead that is not needed for real evaluations (plus, it's not uniform for every evaluator there, as sum have a lot of ifs).

The main point is the performance for RayW SAVING doing SUM ONLY is very different from what I get. Indiana is getting 283% the time I get, while at Steve Brecher saving he is getting 94%, on Indiana-3 he is getting 97% and on Catcus Kev he is getting 93%. So beside the integer conversions (that can be a problem for the ultra fast evaluators), I would check two other points, one is cache size of his processor (altough I would guess he has a better processor than me, but I don't know if he is doing the benchmarks in a VM), and another one that I would not overlook will be Java performance.

BTW, my tests were done on a Core 2 duo 6320, 1,86GHz, 2GB with Windows Vista Ultimate.

Here is a graph of the results I have and the actual numbers.


Code:
Algorithm   Calculation   Saving   Full Name   Best   Ray Wooten   Adrian20XX LUT   Paul Senzee   Steve Brecher   Indiana   Cactus Kev
Generation            988,002   25680,000   994,779            
7 Cards (milliseconds)   SUM  ONLY   SAVING   7 Cards, SUM  ONLY, SAVING (milliseconds)   330,145   330,145   350,774   N/A   3756,904   71876,539   164323,756
7 Cards (milliseconds)   SUM  ONLY   WITHOUT SAVING   7 Cards, SUM  ONLY, WITHOUT SAVING (milliseconds)   500,704   505,249   500,704   N/A   4674,088   71876,539   165007,197
5 Cards (milliseconds)   SUM  ONLY   SAVING   5 Cards, SUM  ONLY, SAVING (milliseconds)   17,534   17,534   25,762   44,079   54,598   124,343   147,643
5 Cards (milliseconds)   SUM  ONLY   WITHOUT SAVING   5 Cards, SUM  ONLY, WITHOUT SAVING (milliseconds)   19,662   19,662   29,213   44,765   67,371   124,343   148,640
7 Cards (milliseconds)   HAND TYPE   SAVING   7 Cards, HAND TYPE, SAVING (milliseconds)   585,116   585,116   707,604   N/A   3792,720   72786,604   165435,831
7 Cards (milliseconds)   HAND TYPE   WITHOUT SAVING   7 Cards, HAND TYPE, WITHOUT SAVING (milliseconds)   1163,118   1485,317   1163,118   N/A   5036,211   72786,604   165668,100
5 Cards (milliseconds)   HAND TYPE   SAVING   5 Cards, HAND TYPE, SAVING (milliseconds)   24,553   24,553   41,339   54,191   55,706   127,617   153,926
5 Cards (milliseconds)   HAND TYPE   WITHOUT SAVING   5 Cards, HAND TYPE, WITHOUT SAVING (milliseconds)   35,126   35,126   43,709   55,955   67,569   127,617   158,332
7 Cards (Million Hands/sec.)   SUM  ONLY   SAVING   7 Cards, SUM  ONLY, SAVING (Million Hands/sec.)   405,230   405,230   381,398   #VALUE!   35,610   1,861   0,814
7 Cards (Million Hands/sec.)   SUM  ONLY   WITHOUT SAVING   7 Cards, SUM  ONLY, WITHOUT SAVING (Million Hands/sec.)   267,193   264,789   267,193   #VALUE!   28,623   1,861   0,811
5 Cards (Million Hands/sec.)   SUM  ONLY   SAVING   5 Cards, SUM  ONLY, SAVING (Million Hands/sec.)   148,224   148,224   100,883   58,961   47,602   20,902   17,603
5 Cards (Million Hands/sec.)   SUM  ONLY   WITHOUT SAVING   5 Cards, SUM  ONLY, WITHOUT SAVING (Million Hands/sec.)   132,182   132,182   88,966   58,058   38,577   20,902   17,485
7 Cards (Million Hands/sec.)   HAND TYPE   SAVING   7 Cards, HAND TYPE, SAVING (Million Hands/sec.)   228,646   228,646   189,067   #VALUE!   35,274   1,838   0,809
7 Cards (Million Hands/sec.)   HAND TYPE   WITHOUT SAVING   7 Cards, HAND TYPE, WITHOUT SAVING (Million Hands/sec.)   115,022   90,071   115,022   #VALUE!   26,565   1,838   0,808
5 Cards (Million Hands/sec.)   HAND TYPE   SAVING   5 Cards, HAND TYPE, SAVING (Million Hands/sec.)   105,851   105,851   62,869   47,959   46,655   20,365   16,884
5 Cards (Million Hands/sec.)   HAND TYPE   WITHOUT SAVING   5 Cards, HAND TYPE, WITHOUT SAVING (Million Hands/sec.)   73,990   73,990   59,461   46,447   38,464   20,365   16,415
7 Cards Relative Performance   SUM  ONLY   SAVING   7 Cards, SUM  ONLY, SAVING Relative Performance   100,000%   100,000%   94,119%   #VALUE!   8,788%   0,459%   0,201%
7 Cards Relative Performance   SUM  ONLY   WITHOUT SAVING   7 Cards, SUM  ONLY, WITHOUT SAVING Relative Performance   100,000%   99,100%   100,000%   #VALUE!   10,712%   0,697%   0,303%
5 Cards Relative Performance   SUM  ONLY   SAVING   5 Cards, SUM  ONLY, SAVING Relative Performance   100,000%   100,000%   68,061%   39,779%   32,115%   14,101%   11,876%
5 Cards Relative Performance   SUM  ONLY   WITHOUT SAVING   5 Cards, SUM  ONLY, WITHOUT SAVING Relative Performance   100,000%   100,000%   67,306%   43,923%   29,185%   15,813%   13,228%
7 Cards Relative Performance   HAND TYPE   SAVING   7 Cards, HAND TYPE, SAVING Relative Performance   100,000%   100,000%   82,690%   #VALUE!   15,427%   0,804%   0,354%
7 Cards Relative Performance   HAND TYPE   WITHOUT SAVING   7 Cards, HAND TYPE, WITHOUT SAVING Relative Performance   100,000%   78,308%   100,000%   #VALUE!   23,095%   1,598%   0,702%
5 Cards Relative Performance   HAND TYPE   SAVING   5 Cards, HAND TYPE, SAVING Relative Performance   100,000%   100,000%   59,394%   45,308%   44,076%   19,240%   15,951%
5 Cards Relative Performance   HAND TYPE   WITHOUT SAVING   5 Cards, HAND TYPE, WITHOUT SAVING Relative Performance   100,000%   100,000%   80,363%   62,775%   51,985%   27,525%   22,185%


Attachments:
EvaluatorsPerformance.jpg
EvaluatorsPerformance.jpg [ 70.81 KB | Viewed 2226 times ]
Top
 Profile  
 
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Mon Apr 13, 2009 8:54 am 
Offline
PokerAI fellow
User avatar

Posts: 7731
Favourite Bot: V12
Did not get what is "sum only", and how is that different than what I do?

Conversion shouldn't take so much. I never did these tests really in a "clean lab environment", simply because of the big difference (RayW vs Steve Brecher vs Indiana/Alberta). Although in your graph Steve Brecher's performs worse than I observed (And SB is mu current preferred evaluator due to usage of less memory/working for more games).

Integer conversion shouldn't be an issue. I'm not sure if I used 64bit JDK, if I did actually int would be an issue (slower), as the native type is 8 bytes, i.e. usage of int will lead to artificial conversions. But AFAIR I used 64 bit OS, 32 bit JDK, which would mean same speed and conversions for int and long (Never really tried that explicitely ...).

My setup is: Q6600, no VMWare.

If you like maybe you can attach the VS project / EXE that produce the above graph, and we can have a look / retry it?

_________________
indiana


Top
 Profile E-mail  
 
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Mon Apr 13, 2009 4:32 pm 
Offline
Senior member
User avatar

Posts: 223
Favourite Bot: N/A
indiana wrote:
Did not get what is "sum only", and how is that different than what I do?
From your tests, you do "sum only". Sum only means adding the hand values, instead of checking what type of hands it belongs to and incrementing that type by one like it's done at the benchmarks at twoplustwo, producing an output like this:
Code:
BAD!! = 0
High Card = 23294460
Pair = 58627800
Two Pair = 31433400
Three of a Kind = 6461620
Straight = 6180020
Flush = 4047644
Full House = 3473184
Four of a Kind = 224848
Straight Flush = 41584
Total Hands = 133784560
indiana wrote:
In your graph Steve Brecher's performs worse than I observed (And SB is mu current preferred evaluator due to usage of less memory/working for more games).
It makes sense to me, your processor is faster than mine (I was expecting this as all the benchmarks beside RayW were running a bit faster on your machine, and you have some Java performance impact while I have native code).
indiana wrote:
Conversion shouldn't take so much.
Integer conversion shouldn't be an issue. I'm not sure if I used 64bit JDK, if I did actually int would be an issue (slower), as the native type is 8 bytes, i.e. usage of int will lead to artificial conversions. But AFAIR I used 64 bit OS, 32 bit JDK, which would mean same speed and conversions for int and long (Never really tried that explicitely ...).
When you are only looking at a memory access and an addition as in RayW, a conversion at each inner loop can be noticeable, so that's why I took care to do all the sums at the proper type that the evaluator is returning. In the "original" code, almost all produce 32 bits in my machine (either long or int, signed or unsigned). Except for Cactus Kev and Paul Senze that return a short (on these ones I'm also doing the operation at 32 bits, so they have a bit of overhead over the others).
indiana wrote:
If you like maybe you can attach the VS project / EXE that produce the above graph, and we can have a look / retry it?
Here it is for RayW. It does not produce the graph, it produces the number than later I put in an Excell Worksheet that produces the graph. It's 32 bits (and, it's not storing pointers for states that could produce code that will run a tiny tiny bit faster).
Oh, it will stop for at the begining for like 25 seconds while it is producing the table, and while starting the hand type evaluations also, while producing the table for the second time.
And at the end, you have to press enter, I did this so I can run it also double clicking at the file whithout the screen dissapearing when it ends.


Attachments:
RayWBenchmarks.zip [55.16 KB]
Downloaded 100 times
Top
 Profile  
 
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Mon Apr 13, 2009 6:03 pm 
Offline
PokerAI fellow
User avatar

Posts: 1239
Favourite Bot: my bot
Sorry for the delay. I've been in the mountains.

Yes, 404url was the originator, but that was based on RayW. 404url used Cactus Kevs code but it wasn't in the listing at 2+2 so I added it.

I think the 24 M evals/sec test had lots of inefficient fluff to translate from a "nice" hand representation, whereas the 143 M evals/sec test was the ordered test that RayW did. I'll check again when I'm back at my own machine in a bit of quiet.


Top
 Profile E-mail  
 
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Mon Apr 13, 2009 7:18 pm 
Offline
PokerAI fellow
User avatar

Posts: 7731
Favourite Bot: V12
Adrian is this producing all the benchmarks (the table above), or only the RayW results?

If you attach the VS project for this and agree to publish all the code under GPL (OK, with the exception of yours, rest should be under GPL, or proprietary free license, as far as I investigated this once ...).

P.S. I updated the OP with the info about spears port, and added in the todo list to make the packaging a bit more clear.

_________________
indiana


Top
 Profile E-mail  
 
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Mon Apr 13, 2009 7:49 pm 
Offline
Senior member
User avatar

Posts: 223
Favourite Bot: N/A
indiana wrote:
Adrian is this producing all the benchmarks (the table above), or only the RayW results?

If you attach the VS project for this and agree to publish all the code under GPL (OK, with the exception of yours, rest should be under GPL, or proprietary free license, as far as I investigated this once ...).

P.S. I updated the OP with the info about spears port, and added in the todo list to make the packaging a bit more clear.

Only RayW.
The problem for uploading a project is that now everything is packed with some new code of mine that is still in too early stages and I don't want to put it out there. And, decoupling it will take a while, and the evaluators for me are like a black hole that when I get close to them I get absorved as it just happened, I didn't played poker for 13 days, that's why I made the post to end that phase, and point that you seem to have a performance performance problem there (I really think the problem is the language now).
Anyway, RayW's benchmark is toooo close to his original code (except for the 100 times run and the sum only), so the original code will benchmark like the executable I've uploaded.

How many milliseconds is this producing in your machine? Because that will give you the hint.


Top
 Profile  
 
 Post subject: Re: 7-card Hand Evaluators
PostPosted: Mon Apr 13, 2009 9:18 pm 
Offline
PokerAI fellow
User avatar

Posts: 7731
Favourite Bot: V12
I will try to run the exe later ...

Well, OK, if at some point you want to organize all the C/C++ evaluators zipped together with the benchmark, like the Java files are, then I would attach this to the OP. This will for sure be helpful to people that don't code in Java.

_________________
indiana


Top
 Profile E-mail  
 
 Post subject: Re: 2+2 Hand Evaluator in Java
PostPosted: Mon Apr 20, 2009 12:18 pm 
Offline
PokerAI fellow
User avatar

Posts: 1613
Location: Austria
Favourite Bot: Marvin
flounderhead wrote:
Just curious (don't speak Java):
- how does the lookup performance compare for *random* (non-sequential) lookups, did you measure that?

I once ported the code to c# and found random access lookups to be much slower than in native code.


I didn't measure. I can't actually imagine, why random lookups would be much slower than when using native functions. However, it makes sense that they're way slower than sequential lookups, due to how CPU internal memory caches work.

If you're preparing card ranges for mass lookups, this is definately something to keep in mind...


Top
 Profile E-mail  
 
 Post subject: Re: 2+2 Hand Evaluator in Java
PostPosted: Mon Apr 20, 2009 12:29 pm 
Offline
PokerAI fellow
User avatar

Posts: 1613
Location: Austria
Favourite Bot: Marvin
indiana wrote:
Is the generator different than the Java port of the 2+2 evaluator?


Now it is in four aspects:

- Used the Meerkat classes for Card, Hand, ... to generate the LUT, saves code
- Split the evaluator and the generator for readability (especially for users of the evaluator)
- In the generic rankHand() function, made sure Hand.size() is >= 5.
- Saved the +1 increments for each card in the lookup code, should increase performance slightly

All in all:

- For < 4-card hands, the generic function is save to use now (returns -1)
- Performance during lookup should be a bit better now, especially if you know how many cards the hand has (perfect for statistical 7-hand scenario testing)
- Performance when creating the LUT is probably a little bit worse due to the Meerkat classes

I think it is worth the changes for readability / API compliance, and performance during lookups.

Cheers, Supersonic.

P.S.: I'd be excited to see performance results, and of course feedback, if anyone bothers to check. Please use the rankHand7() function when benchmarking (due to thew safety check for hand size in the generic rankHand() function, this will be slower).


Attachments:
File comment: 5-7 Card Hand Evaluator
common.zip [42.25 KB]
Downloaded 111 times
Top
 Profile E-mail  
 
 Post subject: My old 7 card evaluation code in VB6
PostPosted: Mon May 18, 2009 1:56 am 
Offline
Senior member
User avatar

Posts: 360
Favourite Bot: Zander
Seeing as i'm no longer a Windows fan, i dug out this old VB6 project. It may be of use to anyone out there who's still a die-hard VB6 fan (like i was a few years ago).

Performance is around 7sec for a full enumeration on my old AMD 3000 box which isn't too shabby considering that everyone says VB6 is "slow". As i always say, never judge until you understand what you're judging.

I'm sure i had a Monte Carlo version as well but i can't find it. :(

Feedback/mods/benchmark results/etc would be nice.


Attachments:
VB6_eval_7.zip [5.14 KB]
Downloaded 124 times

_________________
http://www.bespokebots.com
Top
 Profile  
 
 Post subject: Current evaluation benchmarks?
PostPosted: Mon May 18, 2009 1:21 pm 
Offline
Senior member
User avatar

Posts: 360
Favourite Bot: Zander
I'm returning to botting, having had a break for a couple of years. Can't get my head around pokersource, so i have written my own (simplistic) routine in C. Not really sure how good or bad it is because i have nothing to benchmark against...i would therefore appreciate some guidance on the latest developments and/or some C code which i can use as a benchmark on my machine. Criticisms, modifications, suggestions, etc are all welcome.

This is my first crack at anything useful in C. The code is not optimized and only enumerates at present, however i am particularly interested in Monte Carlo simulations and these will be added in the near future.


Attachments:
c_pokereval.zip [2.82 KB]
Downloaded 110 times

_________________
http://www.bespokebots.com
Top
 Profile  
 
 Post subject: Re: My old 7 card evaluation code in VB6
PostPosted: Mon May 18, 2009 6:25 pm 
Offline
PokerAI fellow
User avatar

Posts: 7731
Favourite Bot: V12
Have you benchmarked that to see what this worths compared to our beasts in the 7-cards hand evaluator thread?

_________________
indiana


Top
 Profile E-mail  
 
 Post subject: Re: My old 7 card evaluation code in VB6
PostPosted: Mon May 18, 2009 6:51 pm 
Offline
Senior member
User avatar

Posts: 360
Favourite Bot: Zander
indiana wrote:
Have you benchmarked that to see what this worths compared to our beasts in the 7-cards hand evaluator thread?


No, and i'm not even sure how to benchmark it. :oops:
It is very old code, and i don't think it will rub shoulders with Java or C, but i guess it would be nice to know how it compares. How would i go about benchmarking it against the other beasts?
I've also ported this method to C, which i've posted on another thread.

_________________
http://www.bespokebots.com


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 399 posts ]  Go to page Previous  1 ... 10, 11, 12, 13, 14, 15, 16 ... 20  Next


Who is online

Users browsing this forum: Google and 7 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to: