Digital Paint Discussion Board
Development => General Development => Topic started by: jitspoe on June 29, 2007, 03:29:14 PM
-
Since this is by far the most requested feature, I might as well kick up some discussion on plans for the implementation.
For starters, and this isn't really related to the faster part, I think there needs to be a progress bar outside of the console, so people don't think the game just locked up at the "loading" screen.
Now, I know a lot of people have suggested HTTP downloads, but I don't think that will work very well. Here's why:
- As I discovered from the server list and later the login system, a number of people have weird proxy configurations that don't allow a basic HTTP connection.
- LAN servers may not have Internet access, and those are the types of servers that people should be downloading from the fastest.
- Who is going to foot the bill for the bandwidth?
- What if the map repository doesn't contain the map a server is running?
- What if the version of the map is different?
I think the most effective method is for the server hosting the game to transfer the files, too, using the same method of communication as game traffic (UDP). If you can connect to the server, you can download maps. This is the way map downloads are currently handled. It just needs to be beefed up a bit.
Currently the way file transfers work is this:
Client requests a download ->
<- Server checks if the file exists and responds accordingly
Client requests first 1k block of file ->
<- Server sends first 1k block of file
Client requests second 1k block of file ->
<- Server sends second 1k block of file
...
until the transfer is finished.
Two key problems with that. #1, the server runs at 10hz, so even under optimal conditions, the maximum file transfer rate is 10k/s. #2, the server waits for the client to request the next block to send it, and the client waits until it receives a block to request the next one. If the latency is high, it drastically impacts the overall throughput. In short, your download speed is determined by your ping + it's rounded up to the nearest 10th of a second, so if you have a ping of over 100ms, you'd probably be getting transfer rates of 5k/s.
Now, the new file transfer system I think should work something like this:
Client requests a download ->
<- Server checks if the file exists, and returns the filename of an appropriate file if it exists (this could be a pak file, if the file is part of a package, or perhaps an alternate image format, like jpg instead of wal)
Client OK's or denies the download of the file (won't overwrite existing files or files outside of the paintball2 directory) ->
<- Server sends first block of file
<- Server sends second block of file
<- Server sends third block of file
Client acknowledges first block of file ->
<- Server sends fourth block of file
Client acknowledges second block of file ->
...
If a client fails to acknowledge a block after a set period of time, the server will re-send that block and slow down the rate at which it sends data. If the client acknowledges all the blocks, the server will increase the rate at which it sends data.
-
You can already chat like an obs when you are loading but you can't see the chat of other people.
-
I have a cunning plan...
Spread the server loads. E.G. system as it is now, but other servers on the serverlist server multiple parts, like a multi source p2p download. More traffic is taken from the less empty full servers.
-
I think that would make thigs overly complex, and issues with missing/different file versions and LAN servers wouldn't be resolved that way, either.
-
how about the client only acknowledging a failed block? Would the server still to send blocks if the client had a problem and was unable to send anything back to the server :/
-
How would the client know if a block failed? And if the client didn't send an ack, the server would keep sending that block. If the client had a problem sending anything back to the server, it wouldn't be able to play anyway.
-
All fast ingame download systems I know of use HTTP.
Just do it.
-
Um.
User friendly.
Period.
-
All fast ingame download systems I know of use HTTP.
Just do it.
I agree with Smokey here. I'm kinda worried about lagging the server, if the chunks it sends are that much bigger.
-
Tremulous and most ET mods use curl, maybe you should check it out?
http://curl.haxx.se/
-
Steam Source-based games (and I think HL-based) allow a server operator to configure an HTTP download location for maps -- typically, it is some server they control. Even if the server is on the same box as the game server, it is still faster than the speed limits imposed by the game's protocol.
I don't recall if any anti-leeching stuff is provided. However, you could do the following to discourage it:
1) Use a standard file (ie: "paintball2-map-allow.txt") in the root of the HTTP maps directory that determines if that directory is okay for paintball2 in-game map leeching. If it does not exist, it is assumed that it isn't okay.
2) Use a header (ie: "X-paintball2-server-referrer") that includes the ip:port of the server we are leeching a map to play on with. This header would be sent when retrieving "paintball2-map-allow.txt" and map files, so that the server could straight out deny or generate a dynamic "allow" file based on the referring pball server (you could dynamically deny/allow access and block map downloads directly using this header).
3) Have the server respect "paintball2-map-allow.txt" and refuse to give the download HTTP location to clients if the HTTP server does not allow map downloads.
4) The client may need to check "paintball2-map-allow.txt" before a download too, since a game server may lie in order to leech bandwidth (of course, since the client sends a referring header, the HTTP server operator can always manually block requests).
If a map desired is not on the given HTTP server, the client might ask the game server for an alternate location that might possibly have it, then eventually default to the normal map download process if it is unable to complete an HTTP download.
Like HL, you could probably expand this to include all media types.
You might allow for rotation of the HTTP download servers in case for some reason or another there are different sets of maps at different locations, or the server operator feels they need to load balance a bit to allow optimum HTTP download speeds.
-
I really don't like the idea of making server setups even more complicated. Having to set up an HTTPD server alongside of the paintball server is going to cause a skyrocket in the already high "Why won't my server work?" posts. cURL doesn't resolve the issue of secured proxies (already tested this a while back for the server browser). Map downloads via UDP aren't going to lag the server any more than HTTP. In fact, they will probably have even lower overhead since the UDP headers are smaller.
-
The downloads aren't that slow. No slower than any other game.
-
They are, you probably dont realise as these maps are smaller in size to most other games (i expect).
-
I really don't like the idea of making server setups even more complicated. Having to set up an HTTPD server alongside of the paintball server is going to cause a skyrocket in the already high "Why won't my server work?" posts. cURL doesn't resolve the issue of secured proxies (already tested this a while back for the server browser). Map downloads via UDP aren't going to lag the server any more than HTTP. In fact, they will probably have even lower overhead since the UDP headers are smaller.
If they have proxy issues, then I'm sure they can deal with slow downloads. :)
-
They are, you probably dont realise as these maps are smaller in size to most other games (i expect).
No, I think Fred187 is right. I mean, RtCW has a average download rate of about 10 Kb/s. But then again, the server admin could also set the max download rate.
-
No, I think Fred187 is right. I mean, RtCW has a average download rate of about 10 Kb/s. But then again, the server admin could also set the max download rate.
I've never gotten below 150kb
I think these maps, for being <1meg to >3mb should download much faster than 4-8 minutes on my connection.
-
I think these maps, for being <1meg to >3mb should download much faster than 4-8 minutes on my connection.
Same here. I have 6 MBPS download connection and it takes right around 4 minutes at the least.
-
Steam Source-based games (and I think HL-based) allow a server operator to configure an HTTP download location for maps -- typically, it is some server they control. Even if the server is on the same box as the game server, it is still faster than the speed limits imposed by the game's protocol.
I don't recall if any anti-leeching stuff is provided. However, you could do the following to discourage it:
1) Use a standard file (ie: "paintball2-map-allow.txt") in the root of the HTTP maps directory that determines if that directory is okay for paintball2 in-game map leeching. If it does not exist, it is assumed that it isn't okay.
2) Use a header (ie: "X-paintball2-server-referrer") that includes the ip:port of the server we are leeching a map to play on with. This header would be sent when retrieving "paintball2-map-allow.txt" and map files, so that the server could straight out deny or generate a dynamic "allow" file based on the referring pball server (you could dynamically deny/allow access and block map downloads directly using this header).
3) Have the server respect "paintball2-map-allow.txt" and refuse to give the download HTTP location to clients if the HTTP server does not allow map downloads.
4) The client may need to check "paintball2-map-allow.txt" before a download too, since a game server may lie in order to leech bandwidth (of course, since the client sends a referring header, the HTTP server operator can always manually block requests).
If a map desired is not on the given HTTP server, the client might ask the game server for an alternate location that might possibly have it, then eventually default to the normal map download process if it is unable to complete an HTTP download.
Like HL, you could probably expand this to include all media types.
You might allow for rotation of the HTTP download servers in case for some reason or another there are different sets of maps at different locations, or the server operator feels they need to load balance a bit to allow optimum HTTP download speeds.
I like this idea. It puts the responsibility of bandwidth with the person running server. Simple access control. Would work on a LAN.
-
Community funded central server, unlimited badwidth, cURL, proxies go kill urslef kthnx.
-
Community funded central server, unlimited badwidth, cURL, proxies go kill urslef kthnx.
You make a compelling argument.
Either way, speeding up the UDP download is going to be done. HTTP would be a very nice addition, but it is one of those fringe features that isn't necessary right now.
-
Community funded central server, unlimited badwidth, cURL, proxies go kill urslef kthnx.
You make a compelling argument.
Either way, speeding up the UDP download is going to be done. HTTP would be a very nice addition, but it is one of those fringe features that isn't necessary right now.
Speeding up UDP is blah, just go HTTP.
-
Why are you so anti-UDP?
-
Why not do something like this? Get the speeds and still get the compatibility.
+ Does the game server specify a HTTP server?
- Yes, it does
+ Is this HTTP server authorized? (i.e. robots.txt kind of thing)
- Yes, it is
+ Does the file exist?
- File found
- Download through HTTP
- File missing on HTTP server
- Ask the game server for a download through UDP
- No, it isn't
- Ask the game server for a download through UDP
- No, it doesn't
- Ask the game server for a download through UDP
-
Why are you so anti-UDP?
Because I know how simple HTTP would be.
-
What are the cons to using libcurl?
-
- Adds bloat.
- Doesn't get around secure proxies and some of the other issues people had downloading the server list.
- Won't speed up LAN server downloads (unless somebody sets up an HTTP server on the LAN).
-
maybe thttpd can help
http://www.acme.com/software/thttpd/
-
Paintball2 can double as a webserver, as well! :D
I don't think that's entirely a good idea. >_> Just one more port for people to open, or hell might happen if someone connected to a paintball2 server through a web browser.
Also, the name of that library is fun to pronounce.
-
I'd prefer to steer clear of HTTP. Adding an HTTP server to paintball2 would potentially open a whole new can of security worms, and as Eiii said, it's just one more port to deal with. Plus many networks block HTTP traffic. UDP is guaranteed to work if you can connect to the server (and, if you can't, it doesn't matter if files transfer or not). Theoretically, it might even be possible to transfer at a faster rate, too, since there is less header overhead.
-
I think UDP is what you should use it seems to be a better choice.
-
Jits, please forgive my inexperienced ramblings, I've never programmed anything like a network game engine before, but after reading your initial statement something occurs to me:
your proposed idea seems to have similar logic to TCP's error recovery - why not use TCP-based file transfer anyway?, then all error recovery is catered for in-protocol. This way is also less affected by latency times as your TCP ack's not only take care of themselves but probably also are also more reliable than your UDP client requests. For instance, if I have a high ping (200+), and receive a segment from the server ok, yet my request for the next block is dropped on the way to the server due to some congestion or CPU bottleneck (or in some cases traffic shaping if your ISP is as rubbish as mine) I'll have to wait for the client timeout value to count off before I even have a chance of receiving the next desired segment. Whereas a TCP request would mean the server and client wouldn't really need to worry about timeouts and missing acks and things as it would all be taken care of for it by the TCP stack + traffic shapers and line congestion would be less likely to drop the ack/request. On reading that back, it strikes me that this may not speed up file transfers in general, just for those who are losing out due to the nature of UDP's unreliable nature mixed with unfortunate next hops.
Why does the server run at 10hz? I imagine theres a good reason for this, but like I say I have no idea of the ins and outs of netcode.
- Rick
-
Well, what I plan to do would essentially behave like TCP, except it would be done at the application level. It's more or lest a "best of both worlds" approach. You get good transfer rates regardless of latency, but without the hassle of adding TCP ports into the mix.
As for the server running at 10Hz, it was just a matter of using minimal resources, I guess. It doesn't exactly run at 10Hz, but it sends packets out at that rate and does game/physics calculations at that rate as well. It can receive packets at a higher rate, and with some slight adjustments that I've already done, it will be able to send them faster as well (but only for downloads).
-
- Rick
:o :o :o You!
Question....from Loial Loialess: How will this effect the servers, specifically in regards to cpu and bandwidth usage? Just curious?
-
I noticed some of your code updates in the CVS for build 20/faster downloading, but when I updated my repository the new code didn't compile because of "e" not being defined in this code:
if (e.x)
e.x(Cmd_Argv(1), atoi(Cmd_Argv(2)));
Bah, hohum oh well.
-
Ahh ok, so really the current implementation is very similar to TFTP, but limited to 10 req's a second.
Loial, I wouildn't have thought running the file transfers over TCP would impact CPU usage much at all, and while TCP headers are larger than UDP's (about 4 times the size) I doubt the few extra blocks would take a noticable amount of time more than with a UDP datastream.
It looks like you want to implement something similar to TCP's flow control, but don't forget that TCP ack's the next seq number in the stream from the last one it received, not each segment just later on. Are you planning a sliding window? I don't know what the realities of efficiency would be like over UDP if you did once the windows start getting rather large, especially over poor quality connections, but then again I doubt this would harm the performance of the server if its only serving 10 requests per client per second.
- Dr Rick Dagless M.D
-
Loial: I imagine the end result will be this: Players will spend less time downloading and more time playing, causing the game to be more enjoyable and increasing the likelihood that they will continue to play. New players won't have to wait 10 minutes to download a map and play, only to have the map change and wait another 10 minutes for the next map and be turned off, never to play again, but will instead increase the player base and draw in more players. The servers will have more people connected, and naturally use more CPU and bandwidth.
Of course, that's in an ideal world, but I'm sure it will help some. The results of the file transfers themselves won't be any different than people downloading files off of the site or accessing the forums and other activities.
y00tz: Code on the CVS probably isn't in a compilable state. That's more or less my backup so I don't lose the code I've been working on. The fast downloading system isn't complete yet, anyway.
Rick: I will have a sliding window. It'll probably start out ~10k/s. I'm not sure how fast I should make it "slide", though. If the window becomes large, it's not going to hurt the server performance any. It'll just send packets out at a slower rate. I'm not quite sure I follow why TCP ack's the next sequence... why would it ack something it hasn't received yet?
-
IIRC its called forward acknowledgement or something, the basic jist of it is that the ack's keep track of how much has been sent - the seq numbers aren't arbitrary, they are the number of the first byte in the block. If a block goes missing, the server will receive a lower ack seq number than it was expecting and know to send a block again starting from that number. That way you only have to ack at the end of your receive window or on a block timeout, instead of after every block and the server will still know which block has gone missing.
I'm not sure what you mean about how fast you'd make it slide - if for every consecutive error-less window you increase the window size, then it will slide up as high as it can, then when it starts to see retransmission requests, stop or make it slide down a little, that way your speed will ramp up quite quickly to its optimum level.
- Dr Rick
-
Dr. Jitless Dagpoe (MD),
If you were the host what would you do when yearly net income started to drop because of a software upgrade your client just uploaded? I would be a bit pissed and looking for payback literally.
So parcentage wise what are we looking at? Dagletus said it was about 4 times the size.
I saw this upgrade needed back in 2000...so I am not backing out or questioning the logic of thy upgrades. Infact I fully support this. *shame on Jitler trying to show me the already known logic. I considered that a slap in the face. * i sorry nathan, craig miss read.
Here is your turn to slap me back. Just help me forcast additional expense, if any.
Live long and prosper DP.
(I say all this owning Jitspoe and Snipen money, i digress and poop backwards)
-
You make a compelling argument.
Not just compelling but is it more cost efficient, Ironphistah?
-1 post me...
-
I'm not sure what you mean about how fast you'd make it slide - if for every consecutive error-less window you increase the window size, then it will slide up as high as it can, then when it starts to see retransmission requests, stop or make it slide down a little, that way your speed will ramp up quite quickly to its optimum level.
What I mean by that is how much should it "slide up" (or down)?
The approach I have is actually a bit different than TCP. It may be a bad idea, but I think it will be fairly effective:
The files will be broken up into 1KB chunks. The size can be changed, but I'm using 1K for simplicity. When the download starts, the server creates an array for the status of each chunk: 0 means it hasn't been sent, -1 means it has been sent and acknowledged, and any other value is the timestamp for when the chunk was sent. The chunks start sending at a rate of 10k/s. Each time a chunk is sent, the rate slows down a bit. When a client receives a chunk, it ack's it. When the server receives the ack, it sets the chunk status to -1 and bumps the rate back up. After all the 0 status chunks have been sent, the server goes back and finds the oldest chunk and re-sends it and continues until all chunks have been ack'd. This could be tweaked so that chunks past a certain age get sent before chunks that haven't been sent at all, but it doesn't really matter. In the end, the client will get all of the chunks, and the order they're in doesn't matter. Each chunk is ID'd and put in the right order on the client side.
See any major flaws with that logic? It's about half implemented now, so hopefully I'll be able to do some testing soon.
loial: In the end, the bandwidth used would be the same. The overhead of the headers isn't going to make much of a difference. Essentially, what would make the difference is how many people are on the server. Maps would download faster, so people would be more likely to stick around and play.
-
what would make the difference is how many people are on the server.
So if more people are on the server......which is our goal.....then a differnece would be seen?
Duh
I am asking will how much larger of usage would there be if the same amount of people connected. Regarding the current version vs the newer one?
-
Hmm if i understand this right, you probably don't need a window at all.
If the current downloads are slow because of the inefficient error correction algorithm, then if you just send all the blocks out as fast as you can (without impairing the servers bandwidth required for players! I imagine something like specifying your servers bandwidth in the config file when you are setting up your server, and then taking the number of connected players, multiplying that by a safe average bandwidth requirement per client, subtracting that from the specified bandwidth in the cfg, then further dividing that by the number of clients in the downloading state) and quench slightly if you get excessive numbers of missing ACKs (e.g. if you have 1kB blocks, if you receive < (transmit rate in kBps)/2) a second), and the multiple-passes will take care of anything else. This way, you don't need to make any decisions about sliding the window up and such.
Thinking about it, this way is quite simple really and probably pretty effective, as long as you start high and go down as the level of lost acks becomes known, instead of needing logic to increment speeds after a number of sucessful runs (which might need tolerance levels and stuff for normal lost packets (acks) to function most efficiently). Having said that, I don't know what the bandwidth situation is like when running popular servers, it might be you need a highly scalable method dependant on how much free bandwidth you can possibly use if the servers are at high capacity.
Nice ideas anyway, can't wait to see it implemented - If only so I don't have to shed a tear everytime someone complains about having to wait ages to download KungFu...
- Dag
-
Downloads are slow now because the server runs fairly slowly and sends out small packets. >_>
-
yeah good point, I had forgotten about the 10hz thing. So how fast will the server be able to send out packets now?
- Rick
-
yeah good point, I had forgotten about the 10hz thing. So how fast will the server be able to send out packets now?
- Rick
1000hz. I figure that's plenty, as people won't likely be getting over 1000k/s.
While your idea to just send data out as fast as the server can would technically work, saturated bandwidth tends to go a lot slower (acks might get dropped, and packets would have to be re-sent multiple times). Plus it would just mean lots of wasted server bandwidth. Even if it takes a while to get up to speed, the transfer rate will be remembered by the server, so transferring lots of small files should be more efficient than with TCP.
-
1000hz. I figure that's plenty, as people won't likely be getting over 1000k/s.
is that 1000Hz or 1000kbs?
I don't thinkg my dialup could handle 1000kbs ;D
-
Both. 1000Hz transferring 1KB blocks.
-
Once you decide which form you will use which appears to be almost done how difficult do you think it would be to implement? Will you just put it in for build 20?
-
Like I said, it's already about halfway implemented. I will probably put it in build 20, but have it as a separate command for testing purposes in case there are any bugs or other issues that need to be worked out, then it will be the default method of transfer in a future version.
-
Hmmm... that sounds good to me, I wish you good luck as their is not much I can help you with in this.
-
lol, whoops.
-
Whoa. 160% of Carpathian is like 12 maps put together, sweet.
-
That looks really really messed up lol.
-
:D
-
I'm making progress. The performance isn't quite as good as I want it to be yet, though. Seems the server tends to max out at 250-500 hz (2-4 ms delay between packets). That's about 300k/s. Much better than it used to be, but still slow for LAN connections. I think I can get around it by sending multiple packets per frame. I'd consider making larger packets, too, but I think packet fragmentation can be a problem. I'm not sure how that works with UDP.
-
Calrathan whipped out his networking book and found the TCP specifications. I've basically got TCP over UDP right now. It works well enough. I was able to transfer at 35-42k/s from cable connection to cable connection. On LAN connections, it's a bit slower than it should be, but naturally a lot faster than the old system. While the file transferring itself is functional, I still have a lot of cleanup, error checking, etc. to work on, so I'm not sure when the next release will be.