I’ve being going through the django testing goat book, specifically the section on continuous integration. Running you functional tests on jenkins can be quite difficult and you’ll need to tinker around and read the logs to figure out whats what. Here are a few tips.
What is the optimal versions?
Well Jenkins should be the latest because vulnerabilities are always being discovered. As for the rest there are specific compatibility issues between them but I have found the best thing to do is use this rule of thumb: Use the latest.
And I mean the absolute stable latest not just the latest for you distribution.
In my case I was on debian 8 which comes with the iceweasel firefox alternative which was at version 52.4 so I went ahead and got geckodriver and an older version of selenium for the tests. Stupid move. Mostly because you want to be up to date with real users.
Best thing to do is head over to geckodriver releases and install the latest. Get selenium 3.7 or latest and the install the latest stable firefox from firefox releases. For me that was 56.0.2.
Make sure that all the binaries are in your path and check --versions
You can install xvfb
Where Do I look when I get error?
You can check the output in the terminal when you run your test or look at geckodriver.log which I have found to be most useful.
So here are a few errors you might see:
NS_ERROR_SOCKET_ADDRESS_IN_USE
geckodriver by default runs on port 2828 and selenium runs on 4444
So they are probably already running and you need to ensure the processes are destroyed when you tearDown in your tests.
To find the process use:
lsof -l | grep 4444
and destroy it with kill -9 <process_id>
Unable to find a matching set of capabilities
selenium.common.exceptions.WebDriverException: Message: Unable to find a matching set of capabilities
This means you are setting the wrong firefox binary, make sure firefox is installed and in the path.
binary = FirefoxBinary('/usr/bin/firefox')
Message: connection refused
selenium.common.exceptions.WebDriverException: Message: connection refused
This error happens when firefox does not have a display so ensure xvfb is installed and you are starting it in jenkins
Conclusion
So the headless firefox, selenium, geckodriver and jenkins setup can be tedious but once it is setup you will have a much more robust CI pipeline and ensure that you end users can function onthe site.