WebGUI hangs on package export.
Initial investigation of this problem, with help from preaction, has resulted in the following conclusions:
a) our WebGUI database contains some assets of which the links to files in the upload directory tree are missing [ cause yet unknown ]; although the folder that we attempt to export as a WebGUI package does not directly contain these assets, one of our templates possibly does: our webgui.log file contains some, possibly related, messages, log file excerpt given below:
2007/01/16 11:47:16 - ERROR - www.main.vir.conf - main::[[undef]] - Could not set uid/gid on '/data/domains/www.main.vir/public/uploads/temp/CC/CC5p6D2jCyLRF6ercd8Bqg/000001000001000098000004.json'2007/01/16 11:47:17 - ERROR - www.main.vir.conf - main::[[undef]] - Could not set uid/gid on '/data/domains/www.main.vir/public/uploads/8D/hr/8DhrdqhM4KvqkCtu6ISApw/thumb-a1.jpg'
2007/01/17 10:47:38 - ERROR - www.main.vir.conf - main::[[undef]] - Couldn't copy file /data/domains/www.main.vir/public/uploads/xK/O3/xKO3cefe_opJtW2Sq8n_1Q/changelog to /data/domains/www.main.vir/public/uploads/9L/PJ/9LPJXEoGIQJ3zW3jULlGtA/changelog because Is a directory
2007/02/07 10:25:40 - ERROR - www.main.vir.conf - WebGUI::Storage::Image::generateThumbnail[129] - Couldn't read image for thumbnail c reation: Exception 435: unable to open image `/data/domains/www.main.vir/public/uploads/qa/Hd/qaHd-bvzPVBY_EiQ8bXz-w/Templates/KnK/003.000.002/green/images/headleft_green.jpg': No such file or directory2007/02/07 10:25:40 - ERROR - www.main.vir.conf - WebGUI::Storage::Image::generateThumbnail[129] - Couldn't read image for thumbnail creation: Exception 435: unable to open image `/data/domains/www.main.vir/public/uploads/Xq/pm/XqpmRezA_yDIcRiomZx3BA/Templates/KnK/003.000.002/green/images/mainmenubg_green.gif': No such file or directory
2007/02/07 10:54:28 - ERROR - www.main.vir.conf - WebGUI::Storage::Image::generateThumbnail[129] - Couldn't read image for thumbnail creation: Exception 430: unable to open file `/data/domains/www.main.vir/public/uploads/cv/mU/cvmUyydK8xPvyislITKtzQ/Templates/KnK/003.000.002/green/un_green.png'2007/02/07 10:54:28 - ERROR - www.main.vir.conf - WebGUI::Storage::Image::generateThumbnail[129] - Couldn't read image for thumbnail creation: Exception 435: unable to open image `/data/domains/www.main.vir/public/uploads/ib/HQ/ibHQr-GON2CsRns9XPBhHg/Templates/KnK/003.000.002/green/images/headleft_green.jpg': No such file or directory
b) /data/WebGUI/lib/WebGUI/Storage.pm gets confused because of this and attempts to access various invalid locations (e.g. ./etc/lvm/, ./etc/cups/, ./proc, etc) and - assumably - tries to enumerate these recursively, in an attempt to compile a package export tarball; apache error log excerpt given below:
c) WebGUI hangs in the export because of this (browser interface is unresponsive) and the server slows down and eventually mysql and apache crash with zero swap space problems.
A replacement for Storage.pm line 792, as suggested by preaction, is given below (first the original line, next the replacement line):
original line 792:
chdir $self->getPath;
replacement line 792:chdir $self->getPath or warn "Could not chdir to '".$self->getPath."': $!";
The earlier log file excerpt was without this change. With this change, similar error messages (but reported at line 795, i.s.o. 794, as I commented out the original line). The log file produced by Storage.pm with the replaced line, does show this (indicative of an asset without reference):
Could not chdir to '/data/domains/www.main.vir/public/uploads/2_/e2/2_e2ymt4vU1OwwPiPYis1Q': No such file or directory at /data/WebGUI/lib/WebGUI/Storage.pm line 793.
--
AMH_mari
I have investigated further, targeting some error messages in the webgui.log file. These are listed below:
Couldn't create storage location: /data/domains/www.main.vir/public/uploads/FP/gA : Permission denied Couldn't create storage location: /data/domains/www.main.vir/public/uploads/FP/gA/FPgAZ0FtPIbSa5aO-kQW3A : No such file or directory
A ls -l of this location on the file system is listed below:
# ls -la /data/domains/www.main.vir/public/uploads/FPtotal 96drwxr-xr-x 7 root root 4096 Apr 11 12:16 .drwxr-xr-x 4074 nobody root 69632 Apr 17 10:22 ..drwxr-xr-x 3 root root 4096 Apr 11 11:57 1Jdrwxr-xr-x 3 root root 4096 Apr 11 12:13 6Wdrwxr-xr-x 3 root root 4096 Apr 11 12:16 nSdrwxr-xr-x 3 root root 4096 Apr 11 12:10 W2drwxr-xr-x 3 root root 4096 Apr 11 11:56 Xy
There seems to be no apparent reason why WebGUI would not be able to create the gA directory in the above path.
But (some thoughts):
a) the WebGUI installation instructions (source install) tell to execute a chown -Rf nobody uploads command after the initial upload directory has been copied from /data/WebGUI/www, thereby changing to owner nobody;
b) during regular WebGUI use new entries appear in the upload directory, owned by root? Is this normal?;
c) If, in the above log file excerpt, WebGUI complains with a Permission denied message, is that possibly because some process, executing with nobody privileges, attempts to create new content in a upload subdirectory owned by root?
# ps -ef |grep spectroot 2357 1 0 07:12 pts/0 00:00:07 perl spectre.pl --daemonroot 3241 2159 0 10:40 pts/0 00:00:00 grep spect# ps -ef |grep httproot 2361 1 0 07:12 ? 00:00:25 /usr/sbin/httpdnobody 3025 2361 0 09:45 ? 00:00:07 /usr/sbin/httpdnobody 3026 2361 0 09:45 ? 00:00:09 /usr/sbin/httpdnobody 3027 2361 0 09:45 ? 00:00:04 /usr/sbin/httpdnobody 3029 2361 0 09:45 ? 00:00:03 /usr/sbin/httpdnobody 3030 2361 0 09:45 ? 00:00:00 /usr/sbin/httpdnobody 3031 2361 0 09:45 ? 00:00:01 /usr/sbin/httpdnobody 3033 2361 0 09:45 ? 00:00:03 /usr/sbin/httpdnobody 3139 2361 0 10:15 ? 00:00:06 /usr/sbin/httpd
> b) during regular WebGUI use new entries appear in the upload directory, owned by root? Is this normal?;
Found out why this happens: a bug in my template import script...
[quote]
There seems to be no apparent reason why WebGUI would not be able to create the gA directory in the above path.[/quote]
The reason you get the permission denied errors is because these files are owned by root and in this case only the owner is allowed to write. The owner of your uploads directories should be the user as whom apache runs.
# ps -ef |grep spectroot 2357 1 0 07:12 pts/0 00:00:07 perl spectre.pl --daemonroot 3241 2159 0 10:40 pts/0 00:00:00 grep spect# ps -ef |grep httproot 2361 1 0 07:12 ? 00:00:25 /usr/sbin/httpdnobody 3025 2361 0 09:45 ? 00:00:07 /usr/sbin/httpdnobody 3026 2361 0 09:45 ? 00:00:09 /usr/sbin/httpdnobody 3027 2361 0 09:45 ? 00:00:04 /usr/sbin/httpdnobody 3029 2361 0 09:45 ? 00:00:03 /usr/sbin/httpdnobody 3030 2361 0 09:45 ? 00:00:00 /usr/sbin/httpdnobody 3031 2361 0 09:45 ? 00:00:01 /usr/sbin/httpdnobody 3033 2361 0 09:45 ? 00:00:03 /usr/sbin/httpdnobody 3139 2361 0 10:15 ? 00:00:06 /usr/sbin/httpd[/quote]
In this case apache is running as nobody and thus the uploads directories should be owned by nobody. Spectre is running as root, which is not necessary! Spectre can be run by a nonpriviledged user as long as you tie it to a port > 1024. By default spectre uses port 32133 so that should not be a problem.
Does this information solve your problem?
My apologies for the delay in answering.
The issue was caused by my template importer script. I ran it as root and did not set effective user and group to nobody. The files owned by root in the storage location were a result of this.
One issue remains though: the fact that WebGUI crashes when trying to export to a package any WebGUI content that has broken storage location references. One of the people on the webgui IRC channel mentioned a code fix (workaround) that I mentioned in the initial bug report.
The problem was indeed caused by a script running with root privileges. I have changed this so that effective uid and gid are set to that of the nobody user.
A bug remains though, as WebGUI crashes when an attempt is done to export to a package WebGUI content which has broken references to the storage location.
Thanks for your info.
