The term "project" in this post corresponds to the term "application" in package manager glossary
Some ideas on basic julia workflow are given here. For a larger project, structuring of the project code is essential for many reasons, some of them are:
Different project scripts and Pluto notebooks may share some parts of the code, and you want to avoid repeating yourself.
Modularization of the project code should facilitate maintainability and extendability.
May be some of the code slowly turns into a package.
You want to have unit testing available in the project.
The whole project directory still shall be shareable with collaborators in a reproducible way.
Name this e.g.
Pkg.generate or PkgTemplates.jl. Unlinke a published Julia package, this project-package will remain not be registered in a Julia package registry, its code will be shared directly via git repository urls. Developing the project in such a project-package has the following advantages:
Straightforward way to share the code in the
src folder among different project scripts and notebooks based on one shared environment via
Straightforward availability of Julia's test and documentation functionality for the project.
Reproducibility due to dependencies recorded in the project environment (files
For this purpose, optionally you can have a folder
packages which contains other sub-packages which potentially can evolve into standalone, even registered packages. As relative paths are recorded in Manifest.toml, these are made available within the project via
Pkg.develop(path="packages/MySubPackage"). This way, the whole project tree including sub-packages will stay relocateable.
This allows for easy start of low key package development. At a later stage,
MySubPackage could be registered as a Julia package while still residing in the project tree, or even removed from the project tree without affecting scripts depending on it – once registered, the package can be added in to the project environment via
Pkg.add instead of
When working with the project, always run julia from the package root with the package environment activated:
Project specific Pluto notebooks would reside in a
notebooks subdirectory and call
Pkg.activate(joinpath(@__DIR__,"..")) in their respective Pkg cell to activate the
MyProject environment. As a consequence, Pluto's in-built package manager will be disabled and the project specific notebooks will share the
MyProject environment and cannot be shared independent from the
MyProject tree (If independent sharing is desired, common project code can be collected into a package residing in
packages and registered in a registry; registering
MyProject itself as a package is not recommended. – More about this in another post).
You may use DrWatson.jl for managing subdirectory access, simulation results and version tagging. By explicitely activating the project environment at the start of Julia or in the notebook Pkg cell, you can avoid
@quickactivate and avoid putting
using DrWatson into script files and notebooks for the sole purpose of activating the common environment. See also this discussion.
A sample project tree could e.g. look like this
MyProject ├── LICENSE ├── Manifest.toml ├── notebooks │ └── demo-notebook.jl ├── packages │ └── MySubPackage │ ├── Manifest.toml │ ├── Project.toml │ ├── src │ │ └── MySubPackage.jl │ └── test │ └── runtests.jl ├── papers ├── Project.toml ├── README.md ├── scripts │ └── demo-script.jl ├── src │ └── MyProject.jl └── test └── runtests.jl
You can download the Julia script genproject.jl and generate this structure a subdirectory on your computer via
julia --compile=min genproject.jl name_of_your_project
Feel free to adapt the generated directory tree to your needs and don't forget to make a git repository out of it as early as possible.
Script functionality should be developed in functions, avoiding global variables
Create a project specific environment as a project-package
Use the REPL and Revise.jl, start julia in the activated project environment
Place shared code among project specific scripts or notebooks either in the
src subdirectory as part of the project-package, or in a sub-package in a subdirectory of the project-package
Separate namespaces of scripts by using modules
Activate the shared project environment of the project-package in project specific Pluto notebooks.
Write tests and examples
By taking advantage of Julia's best-in-class package management facilities, the proposed approach goes a long way in the direction of maintaining sustainable research software. From a talk by A. Zeller:
Have a repo ✓
Anyone can build ✓
Have tests ✓
Be open for extensions ✓
Have examples ✓
2023-04-24: Intermediate headers
2021-12-02: The term "project" in this post corresponds to the term "application" in package manager glossary.
2021-11-15: Initial version