Basic StdLog parser for statistics/ranking

[sk89q]

I rolled a basic StdLog parser last night. No extensive testing has been done on it, but theory-wise, it should all work. At present, the parser only aggregates statistics, but it doesn't conclude upon it (no "score" or anything of that nature). (If someone gives me equations/expressions/math, then a score could be added.)

[edit: a new version is available in a post later in this thread]

I don't know how well it compares speed/efficiency wise, as I chose to write the code itself to be as correct as possible, but a quick test on my server tells:

Code: [Select]

ss1# ./teststdlog.php ../pball/server27921.log 0
Log filesize: 6.74 MB
Games parsed: 523
Time to parse: 7.24498 secs
I'll see if I can post the system's specs as I remember/find them.

Example output: http://digitalpaint.therisenrealm.com/downloads/sdk/libdpstdlog/output.txt

Test script: http://digitalpaint.pastebin.com/f9b5e11d

Note that the class does not eat up the file on its own -- it is used merely to parse input. This way, I could write a real-time parser that would sit in the background and broadcast best K/D, best kill streak, etc. information on my server when the game ended. parse will return the data when a round has ended, so for overall server stats, you'll need to roll your own thing.

The only known bug, although not intrinsically with my class but rather the StdLog output, is with bots that have the same conflicting names will cause inconsistent player states. Turning off DigitalPaintStdLogStateParser->strict will prevent the InconsistentStdLogPlayerState exception from being thrown.

=======

Stats tracked:

For each game:
start_time (when the map was loaded, absolute UNIX timestamp)
game_start_time (when the first round started)
map
game_mode
first_kill (predator, victim, weapon, time, and predator ping)
last_kill (predator, victim, weapon, time, and predator ping)
kill_weap (list of weapons used, and how much)
death_weap (list of weapons used, and how much)
rounds (list)
run_time
play_time
scores (for team games)
winner (for DM)
players (list)

For each round:
start_time
end_time
elapsed_time
result (who won, which is available for every round but the last)
kills
accountable_kills
team_kills
suicides
flags_saved
flags_base_grabbed
flags_captured
flags_team_captured
flags_white_captured (siege and white)
kill_weap (list of weapons used, and how much)
death_weap (list of weapons used, and how much)

For each player:
name
is_bot
aliases (names they used)
kills
accountable_kills (non-team kills)
team_kills
kill_streak (best)
kill_weap (list of weapons used, and how much)
deaths
accountable_deaths (non-team deaths)
team_deaths
death_streak (most)
death_weap (list of weapons used, and how much)
suicides
flags_saved
flags_base_grabbed
flags_captured
flags_team_captured
flags_white_captured
flag_bonus_times
first_kill_for_team (times killed first in a round for team)
first_death_for_team (times died first in a round for team)
ping_avg
connected_time
play_time
k_d (kill/death ratio)

[Eiii]

I like how no one's responded to this yet.
Anyway, it's amazing. Again. <3

[sk89q]

Well, frankly I don't know how many people either have (1) the need for it, or (2) the know how to make this useful for their server, or (3) even the understanding of what the heck I am doing.

On specs, my server has dual dual-core processers. Okay, okay, so that's not too shabby for your average server. Running on my crappy home system with a single core Intel something ~2GHZ? (note, the log has now changed):

Code: [Select]

Log filesize: 6.98 MB
Games parsed: 541
Time to parse: 17.17650 secs
Not great, but good enough for a log parser (I think).

==

By the way, if someone intends to use this, drop me a line, because I have to finalize/fix a few things, like removing the redundant weap_deaths for each round (weap_kills == weap_deaths), make sure suicides are counted in deaths, etc.

[blaa]

3=alot

[b00nlander]

without having tested your script (will hopefully do that later) I can only say FINALLY.  If this is complete and bug-free, it will be a great base to run stats from, at least as long as people are waiting for XBain to finish his stats stuff.
I think it is also good that this does not actually calculate stats, because that leaves options to those who want to implement your coding.  I hope you are distributing it freely (GPL / similar)? I might give a full stats script a shot then, someday.

Funny thing is: I told Reed about 2 weeks ago that it would be great to have such a parser to build other apps on And now it's here.

[Reed]

<3 that is all I shall say.

[lekky]

[sk89q]

Fixed a few things, added PHPDoc, class constants, and the auxiliary readme, license, etc. files:
http://digitalpaint.therisenrealm.com/downloads/sdk/libdpstdlog/libdpstdlog_1_2_1.zip

That should be complete, although, like I said, I haven't done extensive testing, especially having done the bulk of it in one night. However, I was meticulous in writing it, so hopefully I caught most of any problems. Nothing seems wrong, of course.

One way to easily do the stats system is to make a PHP script that runs through a log every 30 minutes or so, adding each game to a database table and indexing players separately. The DBMS can do the statistical and analytical things you need to do and put it on a webpage. In addition, the stats script can also run in real-time, and add the game directly to the DB right away after a game has ended, and saying the round ID # into the game console chat after a game. The hardest part is considering what math is going into calculating the score and possibly how to keep the stats script from corrupting your data should it be stopped abruptly (or another process is started). The laborious part is the website. Have fun!

========

To all those clueless: the StdLog is a special log that stores precise data about the game, such as who killed who, what their ping was at the time, who captured what flag, when they did it, etc. It's not entirely complete, and there are a lot of things left desired, but from what is available, a parser can be used to gain accurate statistics and numbers on how good a player is in a game. The class/script I wrote does the hard part of understanding this log and aggregating the data.

[cusoman]

w00t!!! Thats really sweet Sk89q, pretty soon, if you get server admins to run this script, and can get it to output to a data file, we could see individual players stats online, and with the equations/math you were referring too, this community could have a nice ranking system.  Good work
-Cusoman

[sk89q]

On request of PURE, it now tracks the count of who killed who and with what weapon (kills_against, deaths by). Added in 1.3.

1.3.1 fixes a few small bugs and (hopefully) all the E_NOTICE problems.

http://digitalpaint.therisenrealm.com/downloads/sdk/libdpstdlog/libdpstdlog_1_3_1.zip

[sk89q]

http://digitalpaint.therisenrealm.com/downloads/sdk/libdpstdlog/libdpstdlog_1_3_2.zip

Now stores ip and port for each player.

[StandAloneComplex]

Is anyone actually using this on their servers?

[XtremeBain]

I've begun some testing with this and have it importing stats into my database.  There's a couple bugs, and I'll have to extend the class a little further to get all of the stats I need out of it, but so far it's pretty good.  I just wish I didn't hate OO'd PHP.

[atmays]

Well I tried to make a post gettin people to vote for the ranking system. Since everyone go to the paintball general section, i posted there, and it got a few more votes. Then they locked it and said im setting bad precedence. W/E.

Guess noone will vote for this feature.

By the way go here to vote for it.

http://dplogin.com/dplogin/featurevote/feature.php?id=10078

:D

[vLaD]

<3 this  good job

[i_am_a_pirate]

where do u extract it to?

[ViciouZ]

£5 you don't know what to do with it

Anywhere, as long as PHP will handle it.

[ben@ryehills]

Looks good but any install and configure instructions? Plz

Basically a dummys guide 

As we all no nowt, lol 

[i_am_a_pirate]

yeah, anyone know where this thing goes, how to use it, etc?!?!?!?

[ViciouZ]

First you need PHP. Then you need to extract the file to anywhere. Now copy the logfile you want to parse into that directory. Open a command prompt, cd to the directory, and type:

Code: [Select]

./teststdlog.php ./logfile.here 0It should create a file called output.txt that looks like this: http://digitalpaint.therisenrealm.com/downloads/sdk/libdpstdlog/output.txt

Pitfalls:
Unless you host a server you won't have a logfile.
Until someone writes a nice script to make a webpage from it, the output is fairly difficult to read.