Author: stephen

Going Live with Django, MySQL on Ubuntu and server going down

Ever got to deploy day only to have errors streaming in about the site not being able to be accesses. Most likely in your logs or error emails / alerts you get something like this:

(2003, "Can't connect to MySQL server on '127.0.0.1' (111)")

What the hell does this mean, it means MySQL is down and if it was running it means it has crashed. Whoah but if you check the site now it is working…that means nothing the website is unstable as certain actions make it go down. We need to achieve system stability.

The most common reason for MySQL going down is that it is using more memory than physical memory can allocate it then uses virtual memory and crashes soon after that.

What can I do?

First double check the error logs and see if there is a configuration error causing this issue. If there is no clear warning you want to check memory usage.

Check if you are using virtual memory with vmstat


procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 1  0      0 499228  72040 358540    0    0    16    27  230  280  2  1 97  0  0

Check the amount of memory being used:


ps aux | grep mysqld

A few good commands to use during the process:

In mySQL:

show global status;

Source:

Best Practices for Configuring Optimal MySQL Memory Usage

Comparing Providers for the JSE stocks, commodities, futures and currencies

I have needed End of day and sometimes live data for my trading activities and other similar endeavours and it is hard to find a data provider. Particularly if you want stock data on the JSE as this data has become a commodity for them and they want exorbitant fees for the service.

So I have checked out what is available and I’ve found the following options:

  • quandl – A aggregator of stock data providers. It seems like they provide the platform and API and resell data from other data providers.
  • tiingo – Free end of day data.
  • eoddata – A pay-for service providing downloadable stock, commodity and other such financial data. An old looking site and no API (oh no…)
  • eodhistoricaldata – A pay for service that provides pretty much everything you need

EOD Data

Check the symbols they provide here

Django displaying CheckboxSelectMultiple in columns with Bootstrap

I am using django-bootstrap3 to show django forms with bootstrap. Most of the fields have their own bootstrap widgets except for:


WIDGETS_NO_REQUIRED = (
    AdminFileWidget,
    HiddenInput,
    FileInput,
    CheckboxInput,
    CheckboxSelectMultiple
)

Unfortunately that makes the CheckboxSelectMultiple render as a single line of vertical checkboxes. Not making use of space that well. So how do we change the way it is rendered?

Since django 1.11 we can now use custom renderers simply by changing the template as opposed to overriding the render method prior to django 1.11.

So let us figure out which template we need to change. Search the django source code for CheckboxSelectMultiple. We find it in django.forms.widgets.CheckboxSelectMultiple and the template it is using is template_name = 'django/forms/widgets/checkbox_select.html'.

So we override it:


class ColumnCheckboxSelectMultiple(forms.widgets.CheckboxSelectMultiple):
    template_name = 'forms/widgets/bootstrap_checkbox_select.html'

Then we create the template in:/templates/forms/widgets/bootstrap_checkbox_select.html

Unfortunately we get an error saying the TemplateDoesNotExist at /xxx/

Now the problem is it is not looking in project DIRs. This is sorted by adding the following in your settings file:


FORM_RENDERER = 'django.forms.renderers.TemplatesSetting'
and adding django.forms to INSTALLED_APPS
For more details on why you have to de this checkout this stackoverflow answer

Making the Template

So we have copied the contents of django/forms/templates/django/forms/widgets/multiple_input.html to templates/forms/widgets/bootstrap_checkbox_select.html

Whoops

django-bootstrap is actually rendering the widget or overriding how it is being rendered.

So what we just did makes no difference and even better than that, this is something that can be solved with css. We want to split these checkboxes into columns.

In essence all we need is the following form field:


    user = forms.ModelChoiceField(
        queryset=get_user_model().objects.all(),
        widget=forms.CheckboxSelectMultiple(
            attrs={
                "checked": "",
                "class": "column-checkbox"
            }
        ),
        required=False,
        empty_label=None
    )

and the following css:


.column-checkbox {
    padding-top: 20px;
}

.column-checkbox .checkbox {
    display: block;
    float: left;
    width: 25%;
}