plainblack.com
Username Password
search
Bookmark and Share

    

"Safer" hacking of the WebGUI core on production instances

User dan
Date 6/8/2007 8:10 am
Views 6507
Rating 10    Rate [
|
]
Previous · Next
User Message
dan
Folks,

I often find that I need to make minor adjustments to core WebGUI
features / behaviors based on requests from clients.

This is a simple - but effective - way of hacking the WebGUI core on
production sites without losing track of what files you've tweaked and
how. It requires changes only to "preload.perl".

Create a directory "$webguiRoot/custom/lib/"

Edit "preload.perl" to add the path you just created to @INC "below" the
main $webguiRoot/lib directory:

########################
my $webguiRoot;

BEGIN {
       $webguiRoot = "/home/webgui/WebGUI";
       unshift (@INC, $webguiRoot."/lib");
       unshift (@INC, $webguiRoot."/custom/lib");
}

########################

Now copy the WebGUI modules you're going to hack into an equivalent
directory structure under $webguiRoot/custom/lib.  The idea is that you
don't touch the distributed modules under $webguiRoot/lib at all - you
copy them to the new $webguiRoot/custom/lib directory first.

Restart WebGUI and your tweaked modules in custom/lib/ should take
effect. Supposedly, you can modify @INC at a virtualhost level via
apache directives (giving you the ability to hack the core on a
per-client basis), though that can kill your ability to use shared
memory. I haven't experimented with virtualhost-level @INC hacking yet.

Before your next upgrade, diff files in $webguiRoot/custom/lib against
the distributed ones and you should be able to apply your tweaks to the
new distributed WebGUI codebase (more) easily.

It might be nice to see this idea supported by default in WebGUI. . .

-DJCP

--
-**---****-----******-------********---------**********
Daniel Collis-Puro
Software Engineer
End Point Corp.
dan@endpoint.com
(office) 781-477-0885
**********---------********-------******-----****---**-



Back to Top
Rate [
|
]
 
 
knowmad

Hi Dan,

Thanks for sharing this tip. I've used other systems (like RT by Best Practical) which provide a mechanism like this for overriding core modules/methods. I'd like to see this in the core. Have you submitted an RFE with this patch?

As for per-virtualhost support, although you can tweak @INC, I'd think that the first virtualhost to run and pull the WebGUI libraries into mod_perl's memory would be the one that stays in memory when the user goes to visit another site hosted by the same Apache instance.

Until something like the defunct Apache 2.0's perchild MPM is built, I don't think there is a safe way to do this without running multiple instances of Apache for each host. This wouldn't be too hard as it would simply be a matter of setting up additional modperl servers on higher ports.

There is the worker MPM for Apache 2.2 which I'm not familiar with. Maybe that would work to allow a separate Apache child to load a separate set of libraries from different @INC directories.

 

Cheers,
William



Back to Top
Rate [
|
]
 
 
dan
william@knowmad.com wrote:
> knowmad wrote:
>
> Hi Dan,
>
> Thanks for sharing this tip. I've used other systems (like RT by Best
> Practical ) which provide a mechanism
> like this for overriding core modules/methods. I'd like to see this in
> the core. Have you submitted an RFE with this patch?
>
Ha. I'm very familiar with RT. We run it via FastCGI and suexec and take
advantage of the built-in core hacking features.

I'll happily submit a patch to support this scheme if JT thinks it's an
OK idea.
>
> As for per-virtualhost support, although you can tweak @INC, I'd think
> that the first virtualhost to run and pull the WebGUI libraries into
> mod_perl's memory would be the one that stays in memory when the user
> goes to visit another site hosted by the same Apache instance.
>
The per-VirtualHost @INC modifications are documented here:

http://perl.apache.org/docs/2.0/user/config/config.html#Modifying_C__INC__on_a_Per_VirtualHost

As long as you've got an Ithreads enabled perl (you probably do) it
should work, assuming you trust the docs.

But "the prefork processes will be huge" (from the docs). I'd imagine
they'd only increase in size relative to the number of modules you've
overridden in each VirtualHost, so even then the increase may not be too
bad.
>
> Until something like the defunct Apache 2.0's perchild MPM
>  is built, I don't
> think there is a safe way to do this without running multiple
> instances of Apache for each host. This wouldn't be too hard as it
> would simply be a matter of setting up additional modperl servers on
> higher ports.
>
We run our WebGUI mod_perls as unprivileged users specifically so we can
have some kind of suexec-like user/privilege separation. I plan to
automate the setting up of per-user WebGUI mod_perl / apaches at some
point.

The neat thing about this kind of setup is you can still share the same
WebGUI codebase - you just run the apache/mod_perl process under a
normal user account. Combine that with @INC overrides and you've got
another way to hack the core safely on a per-WebGUI install basis.

-DJCP

--
-**---****-----******-------********---------**********
Daniel Collis-Puro
Software Engineer
End Point Corp.
dan@endpoint.com
(office) 781-477-0885
**********---------********-------******-----****---**-



Back to Top
Rate [
|
]
 
 
patspam

Fantasic, thanks Dan and knowmad. I have used the idea to achieve a clean separation of WebGUI core and project-specific files.

First the motivation:

Each project that I do involves a bunch of Wobjects, Activities etc.. spread throughout the WebGUI source tree. There are two issues with this:

  1. Version control - I keep each project in its own subversion repository (or at least its own sub-dir of a repository). I only want to keep the project files under version control, not the entire WebGUI source tree with my project files spread throughout it. Up until now I have been using svn to ignore everything except my project files but this is not a great solution
  2. Multiple projects, single dev machine - I do all my dev on a single WebGUI vm on my laptop but spreading files from different projects across the same WebGUI vm is a recipe for insanity, and not realy possible with svn (you can't have files from different repositories in the same directory). Obviously I could have a separate VM for each project but that's a lot of VMs.. and now I don't have to!

Here's my approach: 

  1. In the same way as Dan I put all my project files in a completely separate directory (eg. /projects/Project1/lib) and maintain the same directory structure as /data/WebGUI/lib
  2. Add the custom dir to @INC in preload.perl:
    my $webguiRoot;
    my $webguiRoot1; # new custom project dir

    BEGIN {
            $webguiRoot = "/data/WebGUI";
            $webguiRoot1 = "/projects/Project1";
            unshift (@INC, $webguiRoot."/lib");
            unshift (@INC, $webguiRoot1."/lib");
    # new custom project dir
    }
  3. All the modules in your custom Wobjects dir need to get added to @modules, so add the following subroutine at the end of preload.perl:

    #----------------------------------------
    sub getCustomWebGUIModules {
            my $originalWebguiRoot = $webguiRoot;
            $webguiRoot = shift;
            File::Find::find(\&getWebGUIModules, $webguiRoot."/lib/WebGUI");
            $webguiRoot = $originalWebguiRoot;
    }
    And below line 47:

    File::Find::find(\&getWebGUIModules, $webguiRoot."/lib/WebGUI");

    Add a call to the new subroutine for each of your custom project directories:

    getCustomWebGUIModules($webguiRoot1);

I'm in the process of refactoring a bunch of my projects to use this new strategy, will report back if it leads to any problems or any rethinking in terms of the best place to keep automated test files etc..

Cheers,

Patrick 



Back to Top
Rate [
|
]
 
 
JT
Though I can't recommend or condone that people host multiple versions of a single module on a single Apache instance whether Apache allows for it or not, I do agree that having a mechanism for loading custom modules outside of the core is both useful and desirable. To that end I've added a mechanism to preload.perl so that you can create a preload.custom file which contains a list of extra lib folders to search. This functionality will be included in 7.4. Thanks for the idea Dan.

Back to Top
Rate [
|
]
 
 
patspam
Awesome. I refactored a couple of projects to use custom modules outside of the core (not multiple versions or core modules though) and it worked a treat. Great to have it built into the core to make this even easier.

On 6/22/07, jt@plainblack.com <jt@plainblack.com> wrote:
JT wrote:

Though I can't recommend or condone that people host multiple versions of a single module on a single Apache instance whether Apache allows for it or not, I do agree that having a mechanism for loading custom modules outside of the core is both useful and desirable. To that end I've added a mechanism to preload.perl so that you can create a preload.custom file which contains a list of extra lib folders to search. This functionality will be included in 7.4. Thanks for the idea Dan.

http://www.plainblack.com/webgui/dev/discuss/safer-hacking-of-the-webgui-core-on-production-instances/4

Unsubscribe




--

Plain Black&#44; makers of WebGUI
http://plainblack.com




Back to Top
Rate [
|
]
 
 
    



© 2019 Plain Black Corporation | All Rights Reserved