Running multiple django sites on a single server with gunicorn and nginx

When days are dark, friends are few, cash is low and we need to take our django sites and put them on a single server.

I am assuming you have site up and running that is using django with nginx and gunicorn. Now we need to do a few modifications to get the new site working.

First things first get the application code on the site, create your database, create your virtual environment, install your requirements, set any environment variables, migrate, collect static and create your super user. It is that easy (any issues see the guide above)

Creating 2 Gunicorn Services

We setup the nginx config using unix sockets instead of a local http port. So we used:


proxy_pass http://unix:/var/www/myfolder/mysocket.sock;

Instead of:


proxy_pass http://127.0.0.1:8000;

We then created a systemd service to start and stop guncorn for our site called /etc/systemd/system/gunicorn.service

What we need to do now is create a service for each site

Lets rename /etc/systemd/system/gunicorn.service to /etc/systemd/system/site1.gunicorn.service

Make a copy of this site and make relevant changes for the new folder location and socket etc. For a refresher of how this looks:


[Unit]
Description=gunicorn daemon
After=network.target
[Service]
User=
Group=www-data
WorkingDirectory=/var/www/
ExecStart=/var/www//env/bin/gunicorn --access-logfile - --workers 3 --bind unix:/var/www//.sock config.wsgi:application
EnvironmentFile=/var/www//.gunicorn_env
[Install]
WantedBy=multi-user.target

Creating another Nginx server block Config

Make a copy of your existing server config and the new config will look similar to the below, remember to set different log files now as we are hosting multiple sites on a single server.


server {
    listen 80;
    server_name site1.co.za www.site1.co.za;
    error_log /var/log/nginx/site1.error.log
    access_log /var/log/nginx/site1.access.log

    location = /favicon.ico {
        access_log off;
        log_not_found off;
    }   

    location /static/ {
        alias /var/www/site1/site1/staticfiles/;
    }

    location / {
        include proxy_params;
        proxy_pass http://unix:/var/www/site1/site1.sock;
    }
}

Now to test if things are working:

  1. Reload the daemon with sudo systemctl daemon-reload
  2. Check the status of your new services sudo systemctl status site1.gunicorn.service
  3. Start the services: sudo systemctl start site1.gunicorn.service

Remember to ensure your .guncorn_env file is in the project folder

Now we need to enable the nginx config:


sudo ln -s /etc/nginx/sites-available/site1 /etc/nginx/sites-enabled/

Check you don’t have any types with: nginx -t

Then reload the service: sudo service nginx reload

Now everything should be working correctly.

I would like to thank Khophi for his recent post that pretty much guided me on the things to do to make this work in his post on running multiple django projects behind an nginx proxy

Going Live with Django, MySQL on Ubuntu and server going down

Ever got to deploy day only to have errors streaming in about the site not being able to be accesses. Most likely in your logs or error emails / alerts you get something like this:

(2003, "Can't connect to MySQL server on '127.0.0.1' (111)")

What the hell does this mean, it means MySQL is down and if it was running it means it has crashed. Whoah but if you check the site now it is working…that means nothing the website is unstable as certain actions make it go down. We need to achieve system stability.

The most common reason for MySQL going down is that it is using more memory than physical memory can allocate it then uses virtual memory and crashes soon after that.

What can I do?

First double check the error logs and see if there is a configuration error causing this issue. If there is no clear warning you want to check memory usage.

Check if you are using virtual memory with vmstat


procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 1  0      0 499228  72040 358540    0    0    16    27  230  280  2  1 97  0  0

Check the amount of memory being used:


ps aux | grep mysqld

A few good commands to use during the process:

In mySQL:

show global status;

Source:

Best Practices for Configuring Optimal MySQL Memory Usage

Comparing Providers for the JSE stocks, commodities, futures and currencies

I have needed End of day and sometimes live data for my trading activities and other similar endeavours and it is hard to find a data provider. Particularly if you want stock data on the JSE as this data has become a commodity for them and they want exorbitant fees for the service.

So I have checked out what is available and I’ve found the following options:

  • quandl – A aggregator of stock data providers. It seems like they provide the platform and API and resell data from other data providers.
  • tiingo – Free end of day data.
  • eoddata – A pay-for service providing downloadable stock, commodity and other such financial data. An old looking site and no API (oh no…)
  • eodhistoricaldata – A pay for service that provides pretty much everything you need

EOD Data

Check the symbols they provide here