Category: Uncategorized

Lets talk about Postgres…

Postgres queries are CaSeSensitivE unlike in MySQL. So when a simple Model.objects.get(field='hello') would get the record. In postgres you would need to use field__iexact='hello'

In fact,unique doesn’t work was different cases with standard postgres. That is why django has the CIText Mixin and postgres has the citext extension

Unfortunately this won’t help for a unique_together field. The case sensitivity will still come into play.

So you can make the field lowercase by overriding the model’s save method:


    def save(self, **kwargs):
        '''Override save to enforce project name is lowercase'''
        self.name = self.name.lower()
        super().save()

Nope you don’t need to do the above. Just ensure the field is a CICharField

Ran into another problem:


crowdminder=# create extension citext;
ERROR:  extension "citext" already exists

So citext is already there but when running a django migration I get:


django.db.utils.ProgrammingError: type "citext" does not exist
LINE 1: ...llink" ("id" serial NOT NULL PRIMARY KEY, "email" citext NUL...

FFFFFFFFFFFuck you postgres

This is flippen weird.

So I go into psql and raise hell:


drop extension citext cascade;

Then the migrate works. It’s crazy. If this shit happens on production, I will be pissed at pg.

 

Functional Tests failing on Jenkins but not on local computer

Ever struggled with a test failing on jenkins using xvfb, well there are a few things you can do to ensure that both browsers work.

Element Not Found

Sometimes there is an ElementNotFound error on the server and not on your local. Usually this is because selenium cannot see the element. It is hidden.

Usually this happens when both screens you test on are not the same size. So you want to ensure that both screens are the same size.

Solution Maintaining Consistency

Set the Window Size

In your setUp for the tests, ensure you resize the browser screen to a single size.


self.browser = webdriver.Firefox()
self.browser.set_window_size(1280, 800)

Set XVFB Screen size same as local

Set the screen size to the same as your local

jenkins-xvfb-set-screen-size

Functional Tests with Jenkins, Selenium, GeckoDriver and Firefox

I’ve being going through the django testing goat book, specifically the section on continuous integration. Running you functional tests on jenkins can be quite difficult and you’ll need to tinker around and read the logs to figure out whats what. Here are a few tips.

What is the optimal versions?

Well Jenkins should be the latest because vulnerabilities are always being discovered. As for the rest there are specific compatibility issues between them but I have found the best thing to do is use this rule of thumb: Use the latest.

And I mean the absolute stable latest not just the latest for you distribution.

In my case I was on debian 8 which comes with the iceweasel firefox alternative which was at version 52.4 so I went ahead and got geckodriver and an older version of selenium for the tests. Stupid move. Mostly because you want to be up to date with real users.

Best thing to do is head over to geckodriver releases and install the latest. Get selenium 3.7 or latest and the install the latest stable firefox from firefox releases. For me that was 56.0.2.

Make sure that all the binaries are in your path and check --versions

You can install xvfb

Where Do I look when I get error?

You can check the output in the terminal when you run your test or look at geckodriver.log which I have found to be most useful.

So here are a few errors you might see:

NS_ERROR_SOCKET_ADDRESS_IN_USE

geckodriver by default runs on port 2828 and selenium runs on 4444

So they are probably already running and you need to ensure the processes are destroyed when you tearDown in your tests.

To find the process use:

lsof -l | grep 4444

and destroy it with kill -9 <process_id>

Unable to find a matching set of capabilities

selenium.common.exceptions.WebDriverException: Message: Unable to find a matching set of capabilities

This means you are setting the wrong firefox binary, make sure firefox is installed and in the path.

binary = FirefoxBinary('/usr/bin/firefox')

Message: connection refused

selenium.common.exceptions.WebDriverException: Message: connection refused

This error happens when firefox does not have a display so ensure xvfb is installed and you are starting it in jenkins

geckodriver-selenium-firefox-jenkins-errors

Conclusion

So the headless firefox, selenium, geckodriver and jenkins setup can be tedious but once it is setup you will have a much more robust CI pipeline and ensure that you end users can function onthe site.