Part of the EllisLab Network
   
 
Logging humongously slow!!
Posted: 21 January 2007 11:23 AM   [ Ignore ]  
Summer Student
Total Posts:  10
Joined  01-15-2007

Hi,
  I am working this app of mine and its going smoothly. Suddenly, today I notice its taking longer than usual to load. First it was around 1 second .. then it went up to 4 seconds. I was really confused, I even put the benchmarking code inside the Loader class and all but still couldn’t detect it. I commented out all the stuff in my controller and put in an

echo ' foo ';

. Still very slow. I tried searching the forum about this but couldn’t find anything.

Finally, I thought I would check the log and saw that I had to scroll a long way.I deleted the stuff in the log file and Voila!! things are back to normal!!. Then I searched this forum about logging and saw a post on it. I had put log threshold to 4 for some debugging stuff before. I didn’t know it would have such an impact.

  Anyway, isn’t there any way this can be sped up a little? I saw that log.php only flocks the file and then writes to it. Is the overhead that much? And why does it get slower only as the file grows larger?

Thanks,
Thomas

Profile
 
 
Posted: 21 January 2007 02:41 PM   [ Ignore ]   [ # 1 ]  
Administrator
Avatar
RankRankRankRankRankRank
Total Posts:  6762
Joined  03-23-2006

Yeah, logging is slow, and that’s why most people recommend that you only use it for development and testing.  The only solution is to lower the threshold, or do what you’re doing and going in and periodicly deleting the contents of the log file.

 Signature 

DerekAllard.com - CodeIgniter, ExpressionEngine, and the World of Web Design
BambooInvoice - Open Source, CodeIgniter powered invoicing.

Profile
MSG
 
 
Posted: 26 January 2007 11:57 AM   [ Ignore ]   [ # 2 ]  
Grad Student
Avatar
Rank
Total Posts:  35
Joined  08-04-2006

If you are running linux as your server, try using logrotate to manage dynamically your log file size.

Profile
 
 
Posted: 05 April 2007 03:27 PM   [ Ignore ]   [ # 3 ]  
Grad Student
Avatar
Rank
Total Posts:  33
Joined  03-11-2007
Derek Allard - 21 January 2007 02:41 PM

Yeah, logging is slow, and that’s why most people recommend that you only use it for development and testing.  The only solution is to lower the threshold, or do what you’re doing and going in and periodicly deleting the contents of the log file.

Hi Derek,
When I started to work with CodeIgniter 1.5.2, I was astonished to experience that on my Windows 2000 computer at home the page render times got longer and longer in the course of a day.
People in the CodeIgniter Forum recommend to switch off the logging. Ok, I could do that, but then I do not see what’s happening in my PHP code without a debugger, which is usually not available in a productive environment. But especially on a productive system, users could do strange things and I prefer to have that recorded somewhere.

So I made some experiments which showed me, that the native PHP logging, i.e. the error_log function, that sends its messages to the standard PHP system logger - in my case the Apache error_log file - is the fastest by far - even though the log file size already was 16M bytes (!)

Here are the Results on my Windows 2000 box (I wrote 200 log entries):

- PHP Operating System:  Windows NT 5.0 build 2195
- PHP Version / SAPI:    PHP 5.1.6 / apache
- Web Server:        Apache/1.3.34 (Win32) PHP/5.1.6

CodeIgniter logging:                1.9422 sec ->  11633 bytes log file size
Native PHP logging into a specified file:  1.8453 sec ->  12359 bytes log file size
Native PHP logging into the system logger:  0.0025 sec 

You can see the benchmark running on my site:
http://roland.blochberger.us/index.php/pages/packdesc/loggingperf
The benchmark script is available here:
loggingperf_20070405.zip

Regards,
Roland

Profile
 
 
Posted: 05 April 2007 05:19 PM   [ Ignore ]   [ # 4 ]  
Administrator
Avatar
RankRankRankRankRankRank
Total Posts:  6762
Joined  03-23-2006

Thanks Roland.  In your case, I’d encourage you to use the solution that you’re most comfortable with.  Probably disabling the CI logger and using the native one.  Unfortunately, not everyone has access to PHP log files, so for them the CI logger is still a useful tool.

Thanks for the benchmarks.

 Signature 

DerekAllard.com - CodeIgniter, ExpressionEngine, and the World of Web Design
BambooInvoice - Open Source, CodeIgniter powered invoicing.

Profile
MSG
 
 
Posted: 05 April 2007 11:22 PM   [ Ignore ]   [ # 5 ]  
Grad Student
Avatar
Rank
Total Posts:  33
Joined  03-11-2007
Derek Allard - 05 April 2007 05:19 PM

... Unfortunately, not everyone has access to PHP log files, so for them the CI logger is still a useful tool…

Yes, you are right, to use the native logging to the system logger you need access to the web server’s error log. And you usually don’t have that access on a shared web space.  :-(

But the benchmark showed me - at least in the extreme case of Windows 2000 - that PHP has some deficiencies to write into a log file. Both, CI logging and native logging to a specified file need comparatively long times. Only when PHP doesn’t write the log file itself, it is fast..

Profile
 
 
   
 
 
Post Marker Legend
New Topic New posts Hot Topic Hot Topic with new posts New Poll New Poll Moved Topic Moved Topic Sticky Topic Sticky topic
Old Topic No new posts Hot Old Topic Hot Topic with no new posts Old Poll Old Poll Closed Topic Closed Topic Announcement Announcements
Theme
Change Theme
Visitor Statistics
The most visitors ever was 719, on June 06, 2008 10:16 AM
Total Registered Members: 66413 Total Logged-in Users: 22
Total Topics: 84753 Total Anonymous Users: 1
Total Replies: 454817 Total Guests: 242
Total Posts: 539570    
Members ( View Memberlist )
Newest Members:  byrooNirCalexmuellerkizerdrixcaptainredmuffquinodligtharttechsivamDjordjesammozza