It was noticed on a django site that made use of
django.contrib.admin that page loads were getting slow. Lots of data had been added (about 77MB worth, tables with more than 10000 records and up as well as a number of joins).
A great tool for these situations is a profiler. Django-debug-bar is a easy to use profiler built specifically for django. The tool will be used as the primary profiler in this post.
Product Change List
The site was running locally.
A request was made to the change_list url and the first port of call is the firefox
web developer tools:
The tortoise on the
http://127.0.0.1:8000/admin/catalogue/product/ request indicates the request taking longer than 500ms.
It took 890ms.
What is django debug bar showing?
Now there is a red flag. The 304 SQL queries. Remember the list should just be 1 query from a single table. It appears than a N+1 problem has been found where of the shown records – additional queries are made to other tables.
ProductAdmin like this:
class ProductAdmin(CreatedByModelAdmin): list_display = [ 'product_id', 'article', 'status', 'label', 'primary_category', 'manufacturer', 'brand', ] search_fields = ['product_id', 'label'] inlines = [PriceInline, ]
primary_category fields are from different tables. Yet there is no selecting of a related field. Let us make use of
list_select_related = True does nothing. There are still 304 queries.
So the option looked at was
list_select_related = ['article', 'primary_category', ]
This decreased the number of queries to 204.
Now what about the other 200. The problem looks to the the
__str__ function on the related models that make (N+1) queries on the model.
So adding a single related fields instead of the complex
def article_num(self, obj): return obj.article.article_num article_num.short_description = 'Article Num' def primary_category_label(self, obj): return obj.primary_category.label article_num.short_description = 'Primary Category'
That resulted in only 5 queries being made:
The 5 queries:
- Django admin user session
- Getting the admin user
- Getting the count of items (twice) – for pagination
- A single query getting products linked primary category and articles
The request now only takes 288 ms
Product Change View
The product change view is much worse. It took 19 seconds. Extremely slow.
Now a look at the debug bar:
A staggering ~13000 queries.
For the change and add views – the
list_display are not used. Instead the
fields attribute is used.
The chosen method to fix this was to change how the article and category string methods were rendered. They were made much more simpler.
That reduced the number of queries to 40 queries.
def __str__ method on a model as simple as possible.