eSpecialized Blog

Adding the VPS to the front end of our webserver increased responsiveness of our Drupal sites significantly.  1 second response time.. Some I cannot count to 1 second because it responds so fast.

I also put some on xcache as testing.  I am impressed with our drupal sites now.

So after nearly two weeks now.  Our database server was re-sized for our website hosting environment.

Currently psmysql43070 is guaranteed 300 MB for $15.00/month.

Our typical memory usage is 100 MB, evidently 300MB is the lowest it can go.

This gives us growing room. But there is more to this story…..

So our database server is dedicated right now. It appears our front end Web server may be the cause of our extra long webpage load times right now.
Sometimes randomly I will encounter 10-12 seconds before a webpage starts loading (first response).  This is generally unacceptable to most web users.

How our Web hosting system was setup
Webserver sits on a shared hosting environment, which is regularly over loaded ( I did not believe this to be the case)
Database server sits on a shared server environment.

How our Web hosting system is setup today
Webserver sits on a shared hosting environment, which is regularly over loaded
Database server sits on a dedicated server for us only.

Essentially its two separate server systems (computers). One shared, one dedicated.

Conclusion;
A dedicated database was not the solution. Our problem still exists.  I did see our average times decrease by 1 second.  but intermittantly I see websites take as long as before.

Courses of Action;
Move our Database server back to a shared host.   I’ve had the admins say that was not our problem before.  I did not believe them because I saw loading on the database server.
Move our Webserver to a dedicated server, and see if this provides a better speed improvement.

Found the ftp and http downloads were broken.. Not sure what broke it, but its fixed now.  One annoying thing about CKEditor is that it is all messed up, it uses the theme that Drupal is using, and because of this, I can no longer use the ckeditor, I just have to always click on “source” copy and paste HTML.. Which is a big hassle. What can you expect for free?

I also notice lots of exploit hacking on the site too..  Just one of those days eh?  Lots of phpmyadmin and setup.php hacking going on in the log files.

Tonight I found out just how important Drupal’s Page Caching feature is!?  With no concrete answers on the internet for “sizing” drupal to a server. Or how well it scales.. I would recommend running a test yourself to see how well it performs.  Be forewarned tho, you may need permission to run this against public internet servers that your website runs on. Check with your Administrator’s first.

Test server;  AMD Phenom(tm) 8650 Triple-Core Processor (2.3GHz) -> running Debian 5 in 32 bit mode. Apache2, MySql 5, 4GB ram. Drupal 6.  Each core has speed stepping on. Idle running at 1.2 Ghz. The tests were performed remotely by JMeter running on iMac.

MySQL has had little to no optimizations done to it out of the box (apt-get).

35 user test, viewing 5 pages, 10 loops. Drupal Page Cache Disabled.  With block cache enabled, Optimize CSS files enabled, Optimize JavaScript files enabled.
Samples = 1750, Deviation = 574, throughput = 1039.069 a minute, average response 1966ms, median response ms = 1979ms.  Top says the load hit 27 instant load.

Same testing, with all caching enabled (basically just re-enabled page cache);
Samples = 1750, Deviation = 394, throughput = 11,892.280 a minute, average response 123ms, median response ms = 59ms.  Top says the load hit 1.94 instant load.

Below here deals primarily with page cache disabled state;

You need to remember, if my CPU is 3 core, and my Load hits 3. This means (theoretically) full utilization. 100%.  A load of 27, means I need 24 more cores to keep up. Obviously a little misleading.. I can still type in the terminal window, and the server is responsive there giving me proc info etc.

I come up with a 97% deficiency without page caching alone. The proof is in the average response time of nearly 2 seconds.

Double checking CPU Speed Stepping; The CPU cores all Step up to 2300.00 Mhz automatically like they should during the test. “cat /proc/cpuinfo | grep MHz”

MySQL status; Threads: 1  Questions: 407233  Slow queries: 0  Opens: 2775  Flush tables: 1  Open tables: 64  Queries per second avg: 0.392
phpmyadmin runtime info; Created_tmp_disk_tables = 48.. Hmmm.  So some misconfiguration exists out of the box for mysql too..

My max conncurrent connections is set to 35.. This may cause an issue with this test.. I’ll bump it up… While running the test, I watch my Disk Wait State. It stays at 0.0% or 0.1% at the end of the test. When I changed my.cnf to have 100 and restarted mysql.  I only noticed it increase to 60 concurrent connections, when I increased the users to 100.

100 user test, page cache enabled;
Samples = 5000, Deviation = 602, throughput = 12,271.444 a minute, average response 430ms, median response 282ms.  Top says the load hit 6.21 instant load.

One more note here. I see Idle time dip to 5%.

100 users * 5 webpage views * 10 loops = 5000 webpage views in 25.3 seconds. Essentially 197 cached webpages served per second.

100 user test, page cache disabled;
idle dipped to 0.0% here.  no wait time for disk. instant load average top  94.60
100 users * 5 webpage views * 10 loops = 5000 webpage views in 290 seconds. Essentially 17 uncached webpages served per second.

I will try the php xcache stuff later.  It seems some WWW providers have this available.  We have it too, so Will test it out to see the performance increase.

Day2 (about 5 hours later, couldn’t sleep anymore);  I installed php5-xcache in Debian.  Restarted Apache and ran another test;
no wait time for disk. instant load average top  14.60
100 users * 5 webpage views * 10 loops = 5000 webpage views in 100 seconds. That is a huge improvement; 50 uncached webpages served per second using xcache.

To recap; Apache2 + Drupal6 + Mysql6 + PHP (LAMP) on this server in particular, performs well enough.  It can service 197 users per second average without breaking a sweat so long as most those pages are cached by Drupal.  If you turn off page caching, you will see 1/10th or less of the performance, and only 17 users, having a very tough time, will be able to see your website.  Most of them waiting up to 2 seconds or longer to see a response from your server for the first page request (not including graphical content)

I use OpenOffice.org as its free software for all my office daily activities.. Without it I would have to spend $500+ to do the work I do today. I recommend this product to all my friends and family as its free, and provides excel, word, powerpoint functionality which most people want on their home PC’s, but don’t usually have the $$$ to pay for it.

So go get it

http://www.openoffice.org/

I opened up the support ticket system to everyone.

Stats; Clicks 4 / Impressions 1,085
It hasn’t worked at all since Friday Noon (enrolled). I was thinking lack of traffic was due to the holiday weekend.

Ok so I’ll just keep editing the staging website, and keep the production one, production only.  But it would help if .htaccess didn’t cause me so many headaches.   the Re-Write rules although documented,  are poorly documented.  And don’t make alot of sense.  So on to Trial an error.. which only works so well with cached websites.  I hope our ISP doesn’t do caching also.

While I was at it, I added some underlines to see where people could click in the support page, and main page.  I also added a screenshot for the iPhone version. Kewl.

Seems I’m trying to fix this beasty each day.  The problem boils down to the .htaccess file. I’ll fix it, and it will work for a day, the next day I will try it again, and links will be broken.

Its hard to fix a moving target.  I’m not quite sure why when I fix a problem, it works great, then the next day it breaks..

-disappointed.

Now from documentation, the best I can tell this shouldn’t work, but its the only way it does…  Cache’s abound may be the only reason things are breaking and I don’t notice till the next day. The stats part is what I have the biggest issue with.  It will either work, or it wont with 404 errors.

<IfModule mod_rewrite.c>
RewriteEngine on
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_URI} !=/favicon.ico
#  RewriteCond %{REQUEST_URI} ^/(stats).*$ [NC]
RewriteCond %{REQUEST_URI} !^/(SGImages|auth|mn|download|updates|tickets|stats).*$ [NC]
RewriteRule ^(.*)$ index.php?q=$1 [L,QSA]
</IfModule>