Continuous quality checking

Damien Cassou

EmacsConf 2019

Introduction

  1. Quality?
  2. Local setup
  3. Remote setup

Quality?

Quality

  • What?
  • Why?
  • When?
  • How?

Quality: What?

  • quality of behavior
  • quality of code

Quality: Why?

  • let tools find bugs
  • ease contributions
    • standard style
    • quick feedback
  • allow user overrides

Quality: When?

  • to read the code later
  • to submit on melpa
  • to show your skills

Quality: How?

  • naming
  • docstrings
  • small functions
  • automated tests
  • package meta-data

Quality: How?

Avoid Warnings

  • can pinpoint potential bugs
  • won't work if lost in an ocean
  • target 0 warnings
  • even if checkdoc can be annoying
  • warnings ⇒ same as errors

Quality: How? References

Local setup

Local setup: Why?

You want

  • to find bugs fast
  • to share clean code

Local setup: packages

Local setup: ert tests

(require 'ert)

(ert-deftest libmpdel-test-artist ()
  (let* ((artist (libmpdel--artist-create :name "The Artist"))
         (album (libmpdel--album-create :name "The Album"
                                        :artist artist))
         (song (libmpdel--song-create :name "The song"
                                      :album album)))
    (should (equal artist (libmpdel-artist artist)))
    (should (equal artist (libmpdel-artist album)))
    (should (equal artist (libmpdel-artist song)))))

M-x ert-run-tests-interactively

Local setup: buttercup tests

(require 'buttercup)

(describe "beginend in a dired buffer"
  (it "ignores . and .. at the beginning"
    (beginend-dired-test--with-buffer '("." ".." "dir1" "dir2")
      (beginend-dired-test-begin "dir1")
      (beginend-dired-test-end "dir2"))))

M-x buttercup-run-at-point

Local setup: flycheck

Use flycheck to run

  • the byte compiler,
  • checkdoc, and
  • package-lint

while you type

Local setup: flycheck

Use the load-path of the current session: 

(setq flycheck-emacs-lisp-load-path 'inherit)

Local setup: flycheck. When?

Only on your code!

Don't: 

(add-hook 'prog-mode-hook #'flycheck-mode)

Local setup: flycheck. When?

Only on your code!

Do (in a .dir-locals.el file): 

((prog-mode
  (eval flycheck-mode)))

Or (in each file): 

; Local Variables:
; eval: (flycheck-mode)
; End:

Remote setup

Remote setup: Why?

You want

  • contributors to get quick feedback
  • to merge only clean code

Remote setup: Meta-tools

Remote setup: Makefile

ELPA_DEPENDENCIES       = package-lint
ELPA_ARCHIVES           = melpa-stable gnu
TEST_ERT_FILES          = $(wildcard test/*.el)
LINT_CHECKDOC_FILES     = $(wildcard *.el) ${TEST_ERT_FILES}
LINT_PACKAGE_LINT_FILES = ${LINT_CHECKDOC_FILES}
LINT_COMPILE_FILES      = ${LINT_CHECKDOC_FILES}

makel.mk:
    curl \
      --fail --silent --show-error --insecure --location \
      --retry 9 --retry-delay 9 \
      -O https://…/makel.mk \

# Include makel.mk if present
-include makel.mk

Remote setup: Emacs versions on CI

Remote setup: github+travis

.travis.yml 

language: nix

install:
  - bash <(curl https://…/purcell/nix-emacs-ci/…)

env:
  - EMACS_CI=emacs-25-3
  - EMACS_CI=emacs-26-3

script:
  - emacs --version
  - make ci-dependencies
  - make check

Remote setup: gitlab

.gitlab-ci.yml 

image: ubuntu:latest

.test_job_template: &test_job_definition
  script:
    - emacs --version
    - make ci-dependencies
    - make check

test:emacs:25.3:
  image: silex/emacs:25.3-dev
  <<: *test_job_definition

test:emacs:26:
  image: silex/emacs:26-dev
  <<: *test_job_definition

Remote setup: gitea+drone

.drone.yml 

kind: pipeline
name: default
steps:
- name: check
  image: ubuntu:latest
  commands:
  - add-apt-repository ppa:kelleyk/emacs -y
  - apt-get update
  - apt-get install -y emacs26
  - emacs --version
  - make ci-dependencies
  - make check

Conclusion

Tools

Examples

Thank you everyone for your amazing work!