Flexible, Composable, and Incremental Compiler Pipelines

Project Description

Background

Software build scripts are important for software development productivity by executing build tasks incrementally, i.e. only execute tasks when they are affected by changes to dependent modules. However, build scripts are often non-optimal (requiring too many tasks to be executed) or unsound (not performing some tasks, resulting in the need for a clean build). Compiler pipelines are essentially build scripts internal to a compiler, orchestrating the tasks needed to compile a program (parse, type check, analyze, optimize, generate code, etc.). Yet, compiler pipelines are typically developed from scratch and incrementality is an afterthought that is hard to add.

PIE: Interactive Software Pipelines

We are working on a uniform approach to support the high-level definition of software pipelines that are guaranteed to have optimal and sound incremental execution. The PIE DSL provides a language for defining tasks as functions and pipelines as function compositions. The PIE compiler generates incremental build programs from these pipelines. The compiler targets the PIE API, which is supported by the PIE runtime. The PIE runtime algorithm extends the pluto algorithm with a bottom-up evaluation phase that avoids inspecting the entire dependency graph on re-builds.

PIE in Spoofax

We expect PIE to be used as the framework used to conduct all builds in Spoofax:

• the build of a Spoofax language project
• the composition of compiler pipelines for languages developed with Spoofax
• the composition of interactive builders in IDEs for Spoofax languages
• the builds of the Spoofax meta-languages and the workbench itself

This should make all these builders optimally incremental, resulting in a live development experience, i.e. significantly reduce build times for all stages of (meta) development. (A truly live experience also requires incremental execution within individual tasks; we are working on incrementalizing specific analyses, such as parsing and type checking, in parallel to this project.)

Furthermore, PIE provides a more flexible framework for composition of compiler pipelines than the current fixed Spoofax Core pipeline in combination with Stratego builders.

Team

• Eelco Visser
• Gabriël Konat (postdoc)

Related Publications