Thursday, November 12, 2015

Continuous Integration tests for your Emacs package

I recently contributed some Emacs Lisp projects to MELPA. The process was pretty simple. I forked MELPA and created a file that pointed to my repo. Then after pushing to my fork, I submitted a pull request.

But I had a few issues where my code was emitting compiler warnings. Every time I pushed code to my emacs package repository with byte-compile warnings, I would have someone tell me to fix these warnings. So I got fed up with this and created some tests that verify no compiler warnings exist.

 I linked this together with Travis CI so that whenever I push my changes, I run a full test suite for my Emacs Lisp and also make sure no warnings exist. To setup Travis CI, I created this .travis.yml dotfile:

.travis.yml basically tells Travis CI to install a particular version of Emacs, then run "make test". Thats it. Here is what the makefile looks like:

The Makefile test target essentially runs all the tests under the test/ directory while using the .cask directory as a sandbox, which will have all the necessary emacs lisp packages installed for your project. Now all you have to do is put tests under a test/ directory in your git project. Here is the test I use to verify my package builds with no warnings:

Now all you have to do is run make test under your emacs package repository and you can ensure you have no warnings that other people might run into when running package-install. Since the makefile test target is dynamic, any test-file.el you put under the tests directory will now be run in Travis CI.

If you want to run them manually you can just run "make test/test-file.el". If you want to be really fancy, you can setup make test to be run as part of your git pre-commit hooks.

Hope this helps!


  1. Hello Joe ! :)

    You faced the same thing I faced few months ago.
    To help automate emacs package, I created a cask extension to setup all this CI (with also EVM, travis, ecukes, coveralls)

    Have a look on github.
    Any feedback would be much appreciated :)

    1. Thanks for sharing. I wanted to post this to show the state of things. I've seen a lot of different repos with their own solutions and I am interested in making it easier.

  2. this could be an alternative to checking the actual text of the compile buffer:

    cask exec emacs -batch -Q -L . -eval "(progn (setq byte-compile-error-on-warn t) (batch-byte-compile))" clj-refactor.el


    1. This worked for me. I created a makefile target:

      $(CASK) exec $(EMACS) -batch -Q -L . -eval "(progn (setq byte-compile-error-on-warn t) (batch-byte-compile))" helm-git-grep.el

      Then I made my main target depend on this. Cheers!

  3. Doh! I forgot to mention setting up a Cask file and a .gitignore file for .cask directory.