Earlier today, I stumbled upon CircleCI and I was curious about how it compared to continuous integration products I've used in the past (Jenkins and Bamboo). So far I'm impressed.

Testing a Node.js project

After connecting CircleCI to my GitHub account, the only repo I owned that had tests was my Express middleware (described in the last few posts), so I clicked the "build" button and waited to see what other information it wanted. It turns out the answer is: none. It saw the package.json so it assumed it was a Node.js project, ran npm install, then npm test, and worked perfectly. So, my first experience with CircleCI was very impressive.

Testing a C project

Next up was building a C project. It was kind of annoying that I couldn't set it up using an organization repo, since I'm not an admin. It turns out that this project doesn't need to be private anyway, so I just moved it to my personal account.

After doing that, I was hoping it would notice that it's an Autotools project and just work, but it didn't. Luckily the circle.yml was really simple:

        - sudo apt-get install autoconf-archive automake-1.14 autoconf2.59 build-essential check
        - ./autogen.sh
        - make
        - make check

Unfortunately, it took me another hour to get it working, since CircleCI uses Ubuntu 12.04 by default, and my project apparently required a somewhat modern Autotools. In the end, I gave up on complete backwards compatibility, since I wanted to leave the "serial-tests" option in place, and I couldn't get autoreconf to work right. Luckily, I was able to install the updated packages I needed (see the apt-get line), but if they had just used Ubuntu 14.04, it would have worked without any effort.

Also, I'm surprised that they don't detect and configure Autotools projects. The logic shouldn't be too complicated:

  • autogen.sh -> ./autogen.sh
  • configure.ac -> autoreconf --install --verbose
  • Makefile.am -> automake --add-missing

Then if there's a Makefile, run make, then read the Makefile looking for a target named test or check and run the first one you find.

You could do this with CMake too, but looking for a CMakeLists.txt, running cmake --build . followed by ctest ..

Testing a Python project

I also tried setting up a Python project, which was also a little complicated since I wasn't familiar with modern Python testing tools. Apparently, pytest makes it a lot easier to write tests, and tox lets you test against multiple versions of Python. I ended up only testing against Python 3.4 though, since making my API accept and return both bytes and str was too complicated.

CircleCI didn't automatically run py.test, but I'm not sure how it could without looking at the tests and detecting that they used it. It did automatically detect the tox.ini though, so my circle.yml just tells it to use the right version of Python:

        version: 3.4.2


I'm definitely going to continue using CircleCI, and I suspect C and C++ projects will be easier in the future, now that I know how to install newer versions. I do think the CircleCI developers should make this easier though, since it wouldn't take much to make it just as magical as Node.js.