The terror of forking and LWP::Parallel
I’m not sure about anyone else but I hate using the raw fork() command in Perl. It may just be a psychological thing but even in programs I know will run well with good sychronization, I alway harbor a deep fear that the parent will start spawning zombie processes like an undead rabbit. Up until now, I’ve had to deal with that fear in my campaign updating processes. I fork both Google and YSM updates and it saves a ton of time.
Today, however, I was directed toward LWP::Parallel. It allows you to run HTTP calls in parallel. There’s also an LWP::Parallel::UserAgent. This is perfect. I currently inherit the LWP::UserAgent module and overload the request methods for both Google and YSM. Then I just use these modules as I would LWP::UserAgent. Now I’m going to add some logic to take advantage of this parallelization. I’ll keep you apprised of the details.
January 31st, 2006 at 9:51 pm
Google for:
site:stonehenge.com LWP fork
you might be surprised at some of the articles I’ve already done on that.
February 1st, 2006 at 7:36 am
October 2002. It looks like I’m late to the party on this one. These articles are excellent though, thanks for the resource.