Earlier today, I stumbled upon CircleCI1 and I was curious about how it compared to continuous integration products I've used in the past (Jenkins2 and Bamboo3). So far I'm impressed.
Testing a Node.js project
After connecting CircleCI to my GitHub account, the only repo I owned that had tests was my Express middleware4 (described in the last few posts), so I clicked the "build" button and waited to see what other information it wanted. It turns out the answer is: none. It saw the package.json so it assumed it was a Node.js project, ran npm install, then npm test, and worked perfectly5. So, my first experience with CircleCI was very impressive.
Testing a C project
Next up was building a C project. It was kind of annoying that I couldn't set it up using an organization repo, since I'm not an admin. It turns out that this project doesn't need to be private anyway, so I just moved it to my personal account6.
After doing that, I was hoping it would notice that it's an Autotools project and just work, but it didn't. Luckily the circle.yml was really simple:
dependencies:
pre:
- sudo apt-get install autoconf-archive automake-1.14 autoconf2.59 build-essential check
test:
pre:
- ./autogen.sh
- make
override:
- make check
Unfortunately, it took me another hour to get it working, since CircleCI uses Ubuntu 12.04 by default, and my project apparently required a somewhat modern Autotools. In the end, I gave up on complete backwards compatibility, since I wanted to leave the "serial-tests" option in place, and I couldn't get autoreconf to work right. Luckily, I was able to install the updated packages I needed (see the apt-get line), but if they had just used Ubuntu 14.04, it would have worked without any effort.
Also, I'm surprised that they don't detect and configure Autotools projects. The logic shouldn't be too complicated:
autogen.sh->./autogen.shconfigure.ac->autoreconf --install --verboseMakefile.am->automake --add-missing
Then if there's a Makefile, run make, then read the Makefile looking for a target named test or check and run the first one you find.
You could do this with CMake too, by looking for a CMakeLists.txt, running cmake --build . followed by ctest ..
Testing a Python project
I also tried setting up a Python project7, which was also a little complicated since I wasn't familiar with modern Python testing tools. Apparently, pytest8 makes it a lot easier to write tests, and tox9 lets you test against multiple versions of Python. I ended up only testing against Python 3.4 though, since making my API accept and return both bytes and str was too complicated.
CircleCI didn't automatically run py.test, but I'm not sure how it could without looking at the tests and detecting that they used it. It did automatically detect the tox.ini though, so my circle.yml just tells it to use the right version of Python:
machine:
python:
version: 3.4.2
Conclusion
I'm definitely going to continue using CircleCI, and I suspect C and C++ projects will be easier in the future, now that I know how to install newer versions. I do think the CircleCI developers should make this easier though, since it wouldn't take much to make it just as magical as Node.js.
https://circleci.com/ - "Autonomous validation for the AI era - CircleCI"
https://jenkins-ci.org/ - "Jenkins"
https://www.atlassian.com/software/bamboo - "Bamboo: Continuous Integration & Deployment | Atlassian"
https://github.com/brendanlong/express-prefer-wait - "GitHub - brendanlong/express-prefer-wait: Express middleware for handling HTTP 'Prefer: wait' header for static files"
https://circleci.com/gh/brendanlong/express-prefer-wait/1
https://github.com/brendanlong/dash-ts-validator - "GitHub - brendanlong/dash-ts-validator: The MPEG-TS conformance tests for MPEG-DASH."
https://github.com/brendanlong/compression - "GitHub - brendanlong/compression: This is a collection of compression algorithms that I implemented for my own entertainment"
http://pytest.org/ - "pytest documentation"
https://tox.readthedocs.org/en/latest/