Amazon EC2 for map compilation

[DrRickDaglessMD]

Hey all,

I've been thinking about using the virtual computing platform offered by Amazon's Elastic Computing Cloud (EC2) to speed up the compilation of complex maps.

At the moment it takes me more than an hour to fastvis+rad Grave2 and the other side project I'm working on, which is stifling really as a) it ties up my pc for that hour, b) I don't have that much time in the first place to work on stuff, c) it really protracts anything like testing lighting placements, etc...

I was just wonder if anyone had any thoughts about using such a third party processing platform?

For the moment I think the problem with using the EC2 service is that at this time there is no windows platform offered (apparently there is one in development to be released this fall, albeit at a higher price due to licensing) and I don't really have any knowledge of any linux platform BSP/VIS/RAD compilers.

- Dagless

[Eiii]

From what I can see, it looks to be about $0.20/hour for a machine that'll do that work pretty darn quickly...

I don't know how feasible it actually is, though-- does Wine work on the compilers?

[ViciouZ]

Linux has versions of qbsp3, qvis3 and qrad3, possibly deWan's versions, but no arghrad - q2wmap features arghrad sun and phong shading support and is open source but is dodgy with directory setups and linux compatibility for now.

[jitspoe]

q2wmap was very difficult to get directories set up on - requires environment variables to be set and such.  I've been tempted to take the source and refactor it so it works a little better for q2/pb2 stuff.

Are you using detailed brushes a lot?  It shouldn't take long to do a full compile if you've properly detailed everything.  After which, rad will be faster as well.  It only takes a few minutes to final compile midnight2.

[DrRickDaglessMD]

I'm not sure if the EC2 instances support WINE, but apparently you can get windows running on it with QEMU and Linux. I'm not sure if it's really worth the hassle or just wait for the official windows support (depending on the price!).

jits - It's the approach and complexity of the map which is causing the ramping up of development time in my case. Specifically, I've been working on a new map which relies heavily on careful lighting, custom textures and if prudent tricky brushwork. Getting this to play nicely is a tough job, but im accomplishing it slowly (I can't WAIT to show you guys) with r_speeds being at a high of about 1700 - a stunningly low figure for the visible scene + reflective water, trust me. Ideally I need to know how complex a scene is, polish the VIS carefully, work out the lighting, work on the textures (and alphas), models, etc... in that order. Detailing everything prior to polishing the VIS isn't a task I really relish; the fact that BSP makes detail brushes translucent makes texture work very difficult, especially on the number of small and often complex brushes I'm working with. un-detailing them to fix the textures then redetailing them is a chore, but I guess its what I'm faced with unless I can mitigate the problem with something like 24ghz of processing power from EC2! Be assured though, I'm well aware that the map's VIS in its current form is RIDICULOUS and is the cause of the long compile time - I'm almost at the maximum numportals with half the map's architecture in place, with no detail brushes.

Spook has very kindly offered to donate his killer new rig to help me compile maps, an offer I may well take up for Grave2 as the VIS on that can never be improved (because it's such a godawful layout. Why did people have to like this concept? it's an absolute nightmare). Cheers man, I appreciate it.

One concern I have with the cloud computing solution is that does the virtual server instance actually scale as high as it possibly can? And can the compilers use all that capacity?

- Dag

P.S. as a side note, I'm curious how long Zorch's new map, Abandon All Hope takes to RAD - perhaps if you see this Zorch you could let us know? All those phong surfaces have piqued my curiosity!

[Zorchenhimer]

the fact that BSP makes detail brushes translucent makes texture work very difficult, especially on the number of small and often complex brushes I'm working with. un-detailing them to fix the textures then redetailing them is a chore

Take a look at the pic I attached.

And for Abandon All Hope, it doesn't take even close to an hour.  It takes about 10-15 minutes.

Edit:
15 minutes, 27 seconds for a final compile.

[DrRickDaglessMD]

haha, I thought there might be a setting like that but gave up looking after I didn't see anything on the preferences dialogue.

That ought to make life a bit easier then. Grave2 will still be an ordeal, however!

- Dag

[jitspoe]

By the way, you are aware you can just click the "+Det" button to detail something, right?  Combine this with the search by texture name fetaure, and you can detail things pretty quickly.

[DrRickDaglessMD]

aye, although I prefer a more economic use of textures, i.e. if a wall of a structure is intended to be included in the VIS and happens to be made out of wood, it's really likely that the same texture is used elsewhere on more detail-friendly brushes. This is simply because I don't want to have to source loads of wood textures and love using it all over - I'd just like to avoid the chore of having to fine-tooth comb the map for correct detail brushes more than once. I can see how that'd be helpful though, there are definately areas where I could use it.

I guess its a case of wait and see whether the EC2 windows offering is cheap enough to be a viable time-saver.

- Dag

[jitspoe]

Just go to group | omit detail, then anything you see in the map that shouldn't block visibility, you select and +det.  Texture search isn't always useful, but sometimes you'll have something like a light, crate, barrel, etc texture that's ONLY used on brushes that should be detailed.

[Spook]

I'm still willing to loan out my rig if price is an issue, I mean, it takes no time to compile all the maps I've ever created.