Drupal fix failed cron with drush

First run:

drush --yes vset cron_semaphore 0

And then

drush cron

php5-fpm and nginx config to avoid Internal server errors

This is a setting that I've found pretty stable (on a high-end server with 16GB ram) for a busy Drupal site, after a lot of headackes with 503 Error due to poor configuration:

nano /etc/php5/fpm/pool.d/www.conf

pm = dynamic

pm.max_children = 25
pm.min_spare_servers = 2
pm.max_spare_servers = 10

pm.max_requests = 100

 request_terminate_timeout = 30s

This may be also have been useful to put the following line in php.ini to make sure that we get rid of faulty long processes:

max_execution_time = 30

Impose the same time limit in the nginx  fastcgi directives:

nano /etc/nginx/sites-available/default:

            fastcgi_connect_timeout 30;

            fastcgi_send_timeout 30;

            fastcgi_read_timeout 30;

And finally, in /etc/php5/fpm/php.in, my memory limit is:

memory_limit = 256M

Don't forget to restart both nginx and php5-fpm for the changes to take effect.

reset MySQL password

Many suggested solutions did not work. But this one does the job

First You have to  Stop mysql server:
 

service mysqld stop

Now Start mysql server in safe mode with Follwing Options:
 

mysqld_safe –user=mysql –skip-grant-tables –skip-networking &

Now you have to Login to mysql server without password:
 

mysql -u root mysql

You will get Mysql Prompt.
 

mysql> UPDATE user SET Password=PASSWORD('newrootpassword') WHERE User='root';
 
mysql> flush privileges;
 
mysql> exit
 

Restart mysql server:
 

service mysqld restart

Login to MySQL With the New Password:
 

 mysql -uroot -pnewrootpassword

Thanks UnixLab

io test of server

1 or 2U Rack space* 5000 GB bandwidth* 2 AMP / 110v power

dd if=/dev/zero of=test bs=64k count=16k conv=fdatasync

A samle taken now:

My Desktop result: 100MB/s
Dedi (under 7 load): 58.6MB/s
UK VPS (no load): 76.9 MB/s
Bursnet VPS: 118 MB/s

Django deployment on Ubuntu 10.04 using nginx and uwsgi

I spend days to figure out how to do this. The problem is that the existing docs are somehow incompelte or too geeky.  So I doccument all the necessary steps that led to a successful deployment, for futre reference. Hopefully this help other beginner uwsgi deployers too!

I used a minimal Ubuntu 10.04 (32bit) so some of the apt-gets mentioned here may not be needed in a normal ubuntu installation.

Prepare the server

apt-get update
apt-get upgrade
apt-get install nano

apt-get install --reinstall language-pack-en

apt-get install libxml2-dev build-essential python-dev python-pip

To import ppa keys you need to:

apt-get install python-software-properties 

Since uwsgi is natively supported from nginx 0.8(?) onward, you need to use the latest nginx package (which is 1.xx) instead of the archaic nginx 0.7x still in debian packages. To install the latest nginx,

sudo su -
echo "deb http://ppa.launchpad.net/nginx/stable/ubuntu lucid main" >> /etc/apt/sources.list
apt-key adv --keyserver keyserver.ubuntu.com --recv-keys C300EE8C
apt-get update 
apt-get install nginx

Now install uwsgi package:

add-apt-repository  ppa:uwsgi/release 

apt-get update

apt-get -y install uwsgi-python

Here is how I organize directories:

/ /www
/dje /proj
/static

So let's install virtualenv and make dje (Django environment)

apt-get install python-virtualenv

mkdir /www
cd /www
virtualenv dje
cd /www/dje

source bin/activate 

  pip install django

Then make a new project called  proj

python /www/dje/bin/django-admin.py startproject proj

cd /www/dje/proj

Dont' forget doing usuasl settings.py stuff and syncdb! Then

mkdir static
nano deploy.py

In the the deploy.py paste:

 import os
import sys
from os.path import abspath, dirname, join
from site import addsitedir
sys.path.insert(0, abspath(join(dirname(__file__), "../")))
from django.conf import settings
os.environ["DJANGO_SETTINGS_MODULE"] = "proj.settings"
# sys.path.insert(0, join(settings.PROJECT_ROOT, "apps"))
from django.core.handlers.wsgi import WSGIHandler
application = WSGIHandler()

You are done with the django side, now make uwsgi configs:

nano /etc/uwsgi-python/apps-available/django.xml 

 And paste in it:

<uwsgi>
    <socket>127.0.0.1:4000</socket>
    <pythonpath>/www/dje</pythonpath>
    <app mountpoint="/">
        <script>wsgihandler</script>
    </app>
</uwsgi>

Then, symlink it:

ln -s /etc/uwsgi-python/apps-available/django.xml /etc/uwsgi-python/apps-enabled/django.xml

Finally make the site's nginx config file:

nano /etc/nginx/sites-available/default

And paste in it:

upstream django {

server 127.0.0.1:4000;

}

server {

listen 80;

server_name mysite.com;

location / {

uwsgi_pass django;

include uwsgi_params;

uwsgi_param UWSGI_PYHOME /www/dje;

uwsgi_param UWSGI_SCRIPT deploy; #the name of deploy.py

# uwsgi_param SCRIPT_NAME django;

uwsgi_param UWSGI_CHDIR /www/dje/proj;

}

location ^~ /media/ {

root /www/dje/proj/static;

}

}

That's it just restart nginx and uwsgi and enjoy the combo:

service uwsgi-python restart
service nginx  restart

And to install mysql :

apt-get install mysql-server 

apt-get install python-mysqldb 

special thanks to Jason Wang and Web2py folks. 

How to complately remove php from Ubuntu server?

php_installed=`dpkg -l | grep php| awk '{print $2}' |tr "\n" " "`

# remove all php packge

sudo aptitude purge $php_installed

Pinax deployment with nginx and flup

After A LOT of searches failed attempts, at last I found this solution (kind of) working:

1. first install pinax, using the official guide:

and install the necessary packages

apt-get install python-flup nginx subversion python-mysqldb

2. Modify nginx conf

nano /etc/nginx/nginx.conf

user www-data;
worker_processes  1;

error_log  /var/log/nginx/error.log;
pid        /var/run/nginx.pid;

events {
    worker_connections  1024;
    # multi_accept on;
}

http {
    include       /etc/nginx/mime.types;

    access_log /var/log/nginx/access.log;

    sendfile        on;
    #tcp_nopush     on;

    #keepalive_timeout  0;
    keepalive_timeout  65;
    tcp_nodelay        on;

    gzip  on;
    gzip_disable "MSIE [1-6]\.(?!.*SV1)";
    gzip_comp_level  6;
    gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript;

    include /etc/nginx/conf.d/*.conf;
    include /etc/nginx/sites-enabled/*;
}

3. define  foo.com for nginx

nano /etc/nginx/sites-available/foo.com

server {
listen 80;
server_name www.foo.com foo.com;

if ($host != 'foo.com') {
rewrite ^/(.*)$ http://foo.com/$1 permanent;
}

access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log;

location / {
fastcgi_pass 127.0.0.1:7718;
fastcgi_param PATH_INFO $fastcgi_script_name;
fastcgi_param REQUEST_METHOD $request_method;
fastcgi_param QUERY_STRING $query_string;
fastcgi_param CONTENT_TYPE $content_type;
fastcgi_param CONTENT_LENGTH $content_length;
fastcgi_pass_header Authorization;
fastcgi_intercept_errors off;
fastcgi_param REMOTE_ADDR $remote_addr;
fastcgi_param REMOTE_PORT $remote_port;
fastcgi_param SERVER_ADDR $server_addr;
fastcgi_param SERVER_PORT $server_port;
fastcgi_param SERVER_NAME $server_name;
}

location /robots.txt {
alias /www/foo/media/robots.txt;
}


location /site_media/ {
expires 7d;
alias /www/foo/media/;
}

location /media/ {
alias /www/env/lib/python2.6/site-packages/django/contrib/admin/media;
}
}

Boring Symlink：

ln -s /etc/nginx/sites-available/foo.com /etc/nginx/sites-enabled/foo.com

4. Start nginx and fcgi

sudo /etc/init.d/nginx restart
python /www/foo/manage.py runfcgi host=127.0.0.1 port=7718 pidfile=/www/foo/foocom.pid maxspare=2

Some useful commands:

If you don’t know where your Python site directory is:

python -c "from distutils.sysconfig import get_python_lib; print get_python_lib()"

Thanks goes to Thinking Critically

virtualenvwrapper (and install pinax)

First:

 apt-get install python-setuptools

easy_install pip
pip install virtualenv
pip install virtualenvwrapper

Then

nano ~/.bashrc

Now copy the required path

# virtualenvwrapper
export WORKON_HOME=~/.virtualenvs
source /usr/bin/virtualenvwrapper.sh
export PIP_VIRTUALENV_BASE=$WORKON_HOME
export PIP_RESPECT_VIRTUALENV=true

source ~/.bashrc

mkdir ~/.virtualenvs

Usage examples:

mkvirtualenv --no-site-packages myenv

workon myenv

Pinax
Then you can cd to favorite directory and install pinax:

pip install Pinax

And replicate a project:

 pinax-admin setup_project -b social myproj

 Thansk:
http://blog.sidmitra.com/manage-multiple-projects-better-with-virtuale

Mysql database: Migrate and Repair

To repair, just issue this command: 

mysqlcheck -uroot -pxxxxx --auto-repair --optimize --databases your_db

Man page

Migrate mysql using file 

Usually, it is less-time consuming to migrate a database using file, instead of using mysqldump to make a *.sql file, and then restore the database. Sometimes, like when the *.sql file is not available, it is the only way to re-alleviate a website.

So to migrate the file directly:

1)Stop MySQL server; 
2) if necessary, rename the folder containing the individual table files to whatever you want the database to be named (e.g. 'drupal'); 
3) copy the folder directly into /var/lib/mysql; 
4) set permissions on the folder: as root, run 'chown -R mysql /var/lib/mysql/*'; 
5) restart MySQL server; 
6) If you've used a new mysql user, need to update the settings.php accordingly

MYSQLTuner

MySQLTuner  is a perl script for mysql optimization.

To get uset it:



  wget http://mysqltuner.com/mysqltuner.pl
  chmod +x mysqltuner.pl
  ./mysqltuner.pl

I acheived amazing performance boost (server load dropped form 10 to 1) following the suggestions of this script, i.e.:



General recommendations:
    Add skip-innodb to MySQL configuration to disable InnoDB
    Run OPTIMIZE TABLE to defragment tables for better performance
    MySQL started within last 24 hours - recommendations may be inaccurate
    Enable the slow query log to troubleshoot bad queries
    Increase table_cache gradually to avoid file descriptor limits
Variables to adjust:
    query_cache_size (> 64M)
    table_cache (> 64)







Thanks mediakey.dk for introducing this amazing script.