Marginalia

Julia: Basic Workflow Recomendations

  1. Never leave Julia and write code in functions
    1. Avoid untyped global variables
    2. Load code into the REPL via include()
    3. Think about wrapping code into modules
    4. Load modules into the REPL via using
  2. Use Revise.jl to have modified code reloading automatically
    1. Load code into the REPL via Revise.includet()
    2. Loading modules into the REPL via using allows to track all included files
  3. Record project dependencies in reproducible environments
    1. Global environment
    2. Local environments
    3. Enviroment stacking
    4. Shared ("@") Environments
    5. Further info

These workflow hints have been developed from my own experience and are essentially an illustration of the workflow tips found in the Julia documentation.

Never leave Julia and write code in functions

Many available Julia examples and the mindset influenced by Matlab or Python suggest that code is written in scripts where computations are performed in the global context. E.g. a script MyScript.jl would look like:

using Package1
using Package2

# computations here
...

and executed like

$ julia MyScript.jl

However, for Julia this is a bad idea for at least two reasons:

Avoid untyped global variables

Develop any code in functions. E.g. MyScript.jl could look like:

using Package1
using Package2

function main(; kwarg1=1, kwarg2=2)
 # action here 
end

Load code into the REPL via include()

Always invoke the code from within a running julia instance. In this case you encounter the Read-Eval-Print-Loop (REPL) of Julia. You don't need to leave julia for restarting modified code (except in the case when you re-define a constant or a struct). Just reload the code by repeating the include statement:

$ julia
julia> include("MyScript.jl")
julia> main(kwarg1=5)

Think about wrapping code into modules

The previous example can be enhanced by wrapping the code of the script into a module. This has the advantage that you can load different scripts into the same session without name clashes.

Module MyScript

using Package1
using PackageN

function main(; kwarg1=1, kwarg2=2)
 # action here 
end

end
$ julia
julia> include("MyScript.jl")
julia> MyScript.main(kwarg1=5)

Load modules into the REPL via using

Alternatively, load code via using. Once you have mastered modules and ensured that the file name corresponds to the module name, you can load code via using. In order to allow for this, you need to ensure to have the directory containing MyScript.jl (e.g. the current directory pwd()) on the LOAD_PATH:

$ julia
julia> push!(LOAD_PATH,pwd())
julia> using MyScript
julia> MyScript.main(kwarg1=5)

LOAD_PATH can also be passed to Julia as an environment variable defined before the invocation of julia. In order to allow this to work, the directory of MyScript.jlshould not constitute an environment, i.e. it should contain neither a Project.toml nor a Manifest.toml file.

Use Revise.jl to have modified code reloading automatically

In the previous examples, re-loading the code after modifications required to re-run the include statement. The package Revise.jl exports a function includet which triggers automatic recompilation if the source code of the script file or of packages used therein has been modified.

In order to set this up, after invoking Pkg.add("Revise") once, place the following into the Julia startup file .julia/config/startup.jl in your home directory:

using Revise

Load code into the REPL via Revise.includet()

You would then run:

$ julia -i
julia> includet("MyScript.jl")
julia> MyScript.main(kwarg1=5)

After having modified MyScript.jl, just another invocation of MyScript.main() would see the changes. See also the corresponding hints in the Julia documentation.

Loading modules into the REPL via using allows to track all included files

If MyScript.jl itself uses include() to load more code, Revise.jl is unable to track changes in the included code if MyScript.jl has been loaded via includet. This problem however is mitigated by loading MyScript.jl via using. This is true as well for modules and packages under development loaded into the script via using or import. In particular, this way, Revise.jl also works in Pluto notebooks when the inbuilt package manager has been disabled.

Record project dependencies in reproducible environments

In julia, an environment determines which code is loaded via import X and using X. An environment can be described using a Project.toml file in a certain directory. This file contains a list of packages available for loading via import or using.

Global environment

By default, packages added to the Julia installation are recorded in the default global environment:

$ julia
julia> using Pkg
julia> Pkg.add("Package1")

This results in corresponding entries in .julia/environments/vx.y/Project.toml and .julia/environments/vx.y/Manifest.toml (where x.y stands for your installed Julia version, e.g. 1.10). If no other measures are taken, all your julia code will look into this environment in order find the a package to be loaded via using or import.

Sharing this global environment between all your different projects is risky because of possible conflicts in package version requirements. In addition, relying on the global environment makes it hard to share your code with others, as you would have to find a way to communicate the names of the packages (with versions) which they need to install to run your code in a reproducible way.

Local environments

Local environments provide a remedy.

Assume that an application is Julia code residing in a given directory MyApp, uses one or several other Julia packages and is not intended to be invoked from other packages or applications. Set up an environment in the project directory in the following way:

$ cd MyApp
$ julia --project=.
julia> using Pkg
julia> Pkg.add("Package1")
julia> Pkg.add("Package2")
$ exit()

This creates an environment in MyApp directory described by the two files MyApp/Project.toml and MyApp/Manifest.toml. The Project.toml file lists the packages added to the environment. In addition, the Manifest.toml file a holds the information about the exact versions of all Julia packages used by the project.

After setting up the environment like this, you can perform

$ cd MyApp
$ julia --project=.

and work in the environment. All packages added to Julia in this case are recorded in MyApp instead of .julia/environments/vx.y/.

Project.toml should be checked into version control along with the source code. If you took care about adding all necessary dependencies to the local environment, after checking out your code, another project collaborator can easily install all dependencies via

$ cd MyApp
$ julia --project=.
$ julia> using Pkg
$ julia> Pkg.instantiate()

If Manifest.toml is distributed and checked into version control along with Project.toml, instantiate will install the exact same package versions as recorded in the manifest.

Enviroment stacking

After activating a local environment, packages in the global environment still will be visible to your project. You can use this to keep available utilities your project should not depend upon, but which are useful during development. This could e.g. be BenchmarkTools, JuliaFormatter etc.

Shared ("@") Environments

Since Julia 1.7 it is possible to easily work with different more or less global environments:

$ julia --project=@myenv

calls Julia and activates the environment .julia/environments/myenv

Further info

Update history

© Jürgen Fuhrmann. License: CC-BY-SA 4.0. Last modified: November 14, 2024. Website built with Franklin.jl and the Julia programming language. Upon finding factual errors, typos etc. raise an issue.