Reproducible Environments with Singularity
Steffen Ridderbusch
JuliaCon 2020
Imagine…
We need
- the same code and parameters
- the same packages
-
build-in package manager Pkg
-
the same environment and binaries ?
Containers 
Docker 
Docker 
Singularity 
Base Image
as small as possible
| Demo > singularity pull –arch amd64 library://crown421/default/juliabase:1.4.2 |
| INFO: Container is trusted - run ‘singularity key list’ to list your trusted keys |
| INFO: Download complete: juliabase_1.4.2.sif |
| Demo > du -sh juliabase_1.4.2.sif |
| 138M juliabase_1.4.2.sif |
| Demo > singularity shell –no-home juliabase_1.4.2.sif |
| Singularity> julia |
| _ |
| |
| () | () (_) | |
| _ _ | | __ _ | Type “?” for help, “]?” for Pkg help. |
| | | | | | | |/ _| | |
| | | |_| | | | (_| | | Version 1.4.2 (2020-05-23) |
| _/ |\__'_|_|_|\__'_| | Official https://julialang.org/ release |
| |__/ | |
| (@v1.4) pkg> add Distances |
| Cloning default registries into/home/steffen/Code/Demo/containerhome/.julia |
| Cloning registry from "https://github.com/JuliaRegistries/General.git" |
| Added registryGeneralto/home/steffen/Code/Demo/containerhome/.julia/registries/General |
| Resolving package versions... |
| Installed Distances ─ v0.9.0 |
| Updating/home/steffen/Code/Demo/containerhome/.julia/environments/v1.4/Project.toml |
| [b4f34e82] + Distances v0.9.0 |
| Updating/home/steffen/Code/Demo/containerhome/.julia/environments/v1.4/Manifest.toml` |
| [b4f34e82] + Distances v0.9.0 |
| [8f399da3] + Libdl |
| [37e2e46d] + LinearAlgebra |
| [9a3f8284] + Random |
| [9e88b42a] + Serialization |
| [2f01184e] + SparseArrays |
| [10745b16] + Statistics |
| julia> using Distances |
| [ Info: Precompiling Distances [b4f34e82-e78d-54a5-968a-f98e89d6e8f7] |
| julia> euclidean(rand(3), rand(3)) |
| 0.6122474761590728 |
| julia> exit() |
| Singularity> exit |
| exit |
| Demo > tree -a -L 3 |
| . |
| ├── containerhome |
| │ └── .julia |
| │ ├── compiled |
| │ ├── environments |
| │ ├── logs |
| │ ├── packages |
| │ └── registries |
| └── juliabase_1.4.2.sif |
| 7 directories, 1 file |
| Demo > |
| Demo > singularity pull –arch amd64 library://crown421/default/juliabase:1.4.2 |
| INFO: Container is trusted - run ‘singularity key list’ to list your trusted keys |
| INFO: Download complete: juliabase_1.4.2.sif |
| Demo > du -sh juliabase_1.4.2.sif |
| 138M juliabase_1.4.2.sif |
| Demo > singularity shell –no-home juliabase_1.4.2.sif |
| Singularity> julia |
| _ |
| |
| () | () (_) | |
| _ _ | | __ _ | Type “?” for help, “]?” for Pkg help. |
| | | | | | | |/ _| | |
| | | |_| | | | (_| | | Version 1.4.2 (2020-05-23) |
| _/ |\__'_|_|_|\__'_| | Official https://julialang.org/ release |
| |__/ | |
| (@v1.4) pkg> add Distances |
| Cloning default registries into/home/steffen/Code/Demo/containerhome/.julia |
| Cloning registry from "https://github.com/JuliaRegistries/General.git" |
| Added registryGeneralto/home/steffen/Code/Demo/containerhome/.julia/registries/General |
| Resolving package versions... |
| Installed Distances ─ v0.9.0 |
| Updating/home/steffen/Code/Demo/containerhome/.julia/environments/v1.4/Project.toml |
| [b4f34e82] + Distances v0.9.0 |
| Updating/home/steffen/Code/Demo/containerhome/.julia/environments/v1.4/Manifest.toml` |
| [b4f34e82] + Distances v0.9.0 |
| [8f399da3] + Libdl |
| [37e2e46d] + LinearAlgebra |
| [9a3f8284] + Random |
| [9e88b42a] + Serialization |
| [2f01184e] + SparseArrays |
| [10745b16] + Statistics |
| julia> using Distances |
| [ Info: Precompiling Distances [b4f34e82-e78d-54a5-968a-f98e89d6e8f7] |
| julia> euclidean(rand(3), rand(3)) |
| 0.6122474761590728 |
| julia> exit() |
| Singularity> exit |
| exit |
| Demo > tree -a -L 3 |
| . |
| ├── containerhome |
| │ └── .julia |
| │ ├── compiled |
| │ ├── environments |
| │ ├── logs |
| │ ├── packages |
| │ └── registries |
| └── juliabase_1.4.2.sif |
| 7 directories, 1 file |
| Demo > |
| Demo > singularity pull –arch amd64 library://crown421/default/juliabase:1.4.2 |
| INFO: Container is trusted - run ‘singularity key list’ to list your trusted keys |
| INFO: Download complete: juliabase_1.4.2.sif |
| Demo > du -sh juliabase_1.4.2.sif |
| 138M juliabase_1.4.2.sif |
| Demo > singularity shell –no-home juliabase_1.4.2.sif |
| Singularity> julia |
| _ |
| |
| () | () (_) | |
| _ _ | | __ _ | Type “?” for help, “]?” for Pkg help. |
| | | | | | | |/ _| | |
| | | |_| | | | (_| | | Version 1.4.2 (2020-05-23) |
| _/ |\__'_|_|_|\__'_| | Official https://julialang.org/ release |
| |__/ | |
| (@v1.4) pkg> add Distances |
| Cloning default registries into/home/steffen/Code/Demo/containerhome/.julia |
| Cloning registry from "https://github.com/JuliaRegistries/General.git" |
| Added registryGeneralto/home/steffen/Code/Demo/containerhome/.julia/registries/General |
| Resolving package versions... |
| Installed Distances ─ v0.9.0 |
| Updating/home/steffen/Code/Demo/containerhome/.julia/environments/v1.4/Project.toml |
| [b4f34e82] + Distances v0.9.0 |
| Updating/home/steffen/Code/Demo/containerhome/.julia/environments/v1.4/Manifest.toml` |
| [b4f34e82] + Distances v0.9.0 |
| [8f399da3] + Libdl |
| [37e2e46d] + LinearAlgebra |
| [9a3f8284] + Random |
| [9e88b42a] + Serialization |
| [2f01184e] + SparseArrays |
| [10745b16] + Statistics |
| julia> using Distances |
| [ Info: Precompiling Distances [b4f34e82-e78d-54a5-968a-f98e89d6e8f7] |
| julia> euclidean(rand(3), rand(3)) |
| 0.6122474761590728 |
| julia> exit() |
| Singularity> exit |
| exit |
| Demo > tree -a -L 3 |
| . |
| ├── containerhome |
| │ └── .julia |
| │ ├── compiled |
| │ ├── environments |
| │ ├── logs |
| │ ├── packages |
| │ └── registries |
| └── juliabase_1.4.2.sif |
| 7 directories, 1 file |
| Demo > |
| Demo > singularity pull –arch amd64 library://crown421/default/juliabase:1.4.2 |
| INFO: Container is trusted - run ‘singularity key list’ to list your trusted keys |
| INFO: Download complete: juliabase_1.4.2.sif |
| Demo > du -sh juliabase_1.4.2.sif |
| 138M juliabase_1.4.2.sif |
| Demo > singularity shell –no-home juliabase_1.4.2.sif |
| Singularity> julia |
| _ |
| |
| () | () (_) | |
| _ _ | | __ _ | Type “?” for help, “]?” for Pkg help. |
| | | | | | | |/ _| | |
| | | |_| | | | (_| | | Version 1.4.2 (2020-05-23) |
| _/ |\__'_|_|_|\__'_| | Official https://julialang.org/ release |
| |__/ | |
| (@v1.4) pkg> add Distances |
| Cloning default registries into/home/steffen/Code/Demo/containerhome/.julia |
| Cloning registry from "https://github.com/JuliaRegistries/General.git" |
| Added registryGeneralto/home/steffen/Code/Demo/containerhome/.julia/registries/General |
| Resolving package versions... |
| Installed Distances ─ v0.9.0 |
| Updating/home/steffen/Code/Demo/containerhome/.julia/environments/v1.4/Project.toml |
| [b4f34e82] + Distances v0.9.0 |
| Updating/home/steffen/Code/Demo/containerhome/.julia/environments/v1.4/Manifest.toml` |
| [b4f34e82] + Distances v0.9.0 |
| [8f399da3] + Libdl |
| [37e2e46d] + LinearAlgebra |
| [9a3f8284] + Random |
| [9e88b42a] + Serialization |
| [2f01184e] + SparseArrays |
| [10745b16] + Statistics |
| julia> using Distances |
| [ Info: Precompiling Distances [b4f34e82-e78d-54a5-968a-f98e89d6e8f7] |
| julia> euclidean(rand(3), rand(3)) |
| 0.6122474761590728 |
| julia> exit() |
| Singularity> exit |
| exit |
| Demo > tree -a -L 3 |
| . |
| ├── containerhome |
| │ └── .julia |
| │ ├── compiled |
| │ ├── environments |
| │ ├── logs |
| │ ├── packages |
| │ └── registries |
| └── juliabase_1.4.2.sif |
| 7 directories, 1 file |
| Demo > |
| Demo > singularity pull –arch amd64 library://crown421/default/juliabase:1.4.2 |
| INFO: Container is trusted - run ‘singularity key list’ to list your trusted keys |
| INFO: Download complete: juliabase_1.4.2.sif |
| Demo > du -sh juliabase_1.4.2.sif |
| 138M juliabase_1.4.2.sif |
| Demo > singularity shell –no-home juliabase_1.4.2.sif |
| Singularity> julia |
| _ |
| |
| () | () (_) | |
| _ _ | | __ _ | Type “?” for help, “]?” for Pkg help. |
| | | | | | | |/ _| | |
| | | |_| | | | (_| | | Version 1.4.2 (2020-05-23) |
| _/ |\__'_|_|_|\__'_| | Official https://julialang.org/ release |
| |__/ | |
| (@v1.4) pkg> add Distances |
| Cloning default registries into/home/steffen/Code/Demo/containerhome/.julia |
| Cloning registry from "https://github.com/JuliaRegistries/General.git" |
| Added registryGeneralto/home/steffen/Code/Demo/containerhome/.julia/registries/General |
| Resolving package versions... |
| Installed Distances ─ v0.9.0 |
| Updating/home/steffen/Code/Demo/containerhome/.julia/environments/v1.4/Project.toml |
| [b4f34e82] + Distances v0.9.0 |
| Updating/home/steffen/Code/Demo/containerhome/.julia/environments/v1.4/Manifest.toml` |
| [b4f34e82] + Distances v0.9.0 |
| [8f399da3] + Libdl |
| [37e2e46d] + LinearAlgebra |
| [9a3f8284] + Random |
| [9e88b42a] + Serialization |
| [2f01184e] + SparseArrays |
| [10745b16] + Statistics |
| julia> using Distances |
| [ Info: Precompiling Distances [b4f34e82-e78d-54a5-968a-f98e89d6e8f7] |
| julia> euclidean(rand(3), rand(3)) |
| 0.6122474761590728 |
| julia> exit() |
| Singularity> exit |
| exit |
| Demo > tree -a -L 3 |
| . |
| ├── containerhome |
| │ └── .julia |
| │ ├── compiled |
| │ ├── environments |
| │ ├── logs |
| │ ├── packages |
| │ └── registries |
| └── juliabase_1.4.2.sif |
| 7 directories, 1 file |
| Demo > |
| Demo > singularity pull –arch amd64 library://crown421/default/juliabase:1.4.2 |
| INFO: Container is trusted - run ‘singularity key list’ to list your trusted keys |
| INFO: Download complete: juliabase_1.4.2.sif |
| Demo > du -sh juliabase_1.4.2.sif |
| 138M juliabase_1.4.2.sif |
| Demo > singularity shell –no-home juliabase_1.4.2.sif |
| Singularity> julia |
| _ |
| |
| () | () (_) | |
| _ _ | | __ _ | Type “?” for help, “]?” for Pkg help. |
| | | | | | | |/ _| | |
| | | |_| | | | (_| | | Version 1.4.2 (2020-05-23) |
| _/ |\__'_|_|_|\__'_| | Official https://julialang.org/ release |
| |__/ | |
| (@v1.4) pkg> add Distances |
| Cloning default registries into/home/steffen/Code/Demo/containerhome/.julia |
| Cloning registry from "https://github.com/JuliaRegistries/General.git" |
| Added registryGeneralto/home/steffen/Code/Demo/containerhome/.julia/registries/General |
| Resolving package versions... |
| Installed Distances ─ v0.9.0 |
| Updating/home/steffen/Code/Demo/containerhome/.julia/environments/v1.4/Project.toml |
| [b4f34e82] + Distances v0.9.0 |
| Updating/home/steffen/Code/Demo/containerhome/.julia/environments/v1.4/Manifest.toml` |
| [b4f34e82] + Distances v0.9.0 |
| [8f399da3] + Libdl |
| [37e2e46d] + LinearAlgebra |
| [9a3f8284] + Random |
| [9e88b42a] + Serialization |
| [2f01184e] + SparseArrays |
| [10745b16] + Statistics |
| julia> using Distances |
| [ Info: Precompiling Distances [b4f34e82-e78d-54a5-968a-f98e89d6e8f7] |
| julia> euclidean(rand(3), rand(3)) |
| 0.6122474761590728 |
| julia> exit() |
| Singularity> exit |
| exit |
| Demo > tree -a -L 3 |
| . |
| ├── containerhome |
| │ └── .julia |
| │ ├── compiled |
| │ ├── environments |
| │ ├── logs |
| │ ├── packages |
| │ └── registries |
| └── juliabase_1.4.2.sif |
| 7 directories, 1 file |
| Demo > |
| Demo > singularity pull –arch amd64 library://crown421/default/juliabase:1.4.2 |
| INFO: Container is trusted - run ‘singularity key list’ to list your trusted keys |
| INFO: Download complete: juliabase_1.4.2.sif |
| Demo > du -sh juliabase_1.4.2.sif |
| 138M juliabase_1.4.2.sif |
| Demo > singularity shell –no-home juliabase_1.4.2.sif |
| Singularity> julia |
| _ |
| |
| () | () (_) | |
| _ _ | | __ _ | Type “?” for help, “]?” for Pkg help. |
| | | | | | | |/ _| | |
| | | |_| | | | (_| | | Version 1.4.2 (2020-05-23) |
| _/ |\__'_|_|_|\__'_| | Official https://julialang.org/ release |
| |__/ | |
| (@v1.4) pkg> add Distances |
| Cloning default registries into/home/steffen/Code/Demo/containerhome/.julia |
| Cloning registry from "https://github.com/JuliaRegistries/General.git" |
| Added registryGeneralto/home/steffen/Code/Demo/containerhome/.julia/registries/General |
| Resolving package versions... |
| Installed Distances ─ v0.9.0 |
| Updating/home/steffen/Code/Demo/containerhome/.julia/environments/v1.4/Project.toml |
| [b4f34e82] + Distances v0.9.0 |
| Updating/home/steffen/Code/Demo/containerhome/.julia/environments/v1.4/Manifest.toml` |
| [b4f34e82] + Distances v0.9.0 |
| [8f399da3] + Libdl |
| [37e2e46d] + LinearAlgebra |
| [9a3f8284] + Random |
| [9e88b42a] + Serialization |
| [2f01184e] + SparseArrays |
| [10745b16] + Statistics |
| julia> using Distances |
| [ Info: Precompiling Distances [b4f34e82-e78d-54a5-968a-f98e89d6e8f7] |
| julia> euclidean(rand(3), rand(3)) |
| 0.6122474761590728 |
| julia> exit() |
| Singularity> exit |
| exit |
| Demo > tree -a -L 3 |
| . |
| ├── containerhome |
| │ └── .julia |
| │ ├── compiled |
| │ ├── environments |
| │ ├── logs |
| │ ├── packages |
| │ └── registries |
| └── juliabase_1.4.2.sif |
| 7 directories, 1 file |
| Demo > |
available at cloud.sylabs.io/library/crown421/default/juliabase
container a project
| ContaineredExample > master > tree -L 3 |
| . |
| ├── data |
| │ ├── exp_pro |
| │ ├── exp_raw |
| │ └── sims |
| ├── Manifest.toml |
| ├── notebooks |
| ├── old_result.bson |
| ├── papers |
| ├── plots |
| ├── Project.toml |
| ├── README.md |
| ├── _research |
| ├── scripts |
| │ └── intro.jl |
| └── src |
| └── ExampleModule |
| ├── Manifest.toml |
| ├── Project.toml |
| └── src |
|
|
| 12 directories, 8 files |
| ContaineredExample > master > julia –project |
| _ |
| |
| () | () (_) | |
| _ _ | | __ _ | Type “?” for help, “]?” for Pkg help. |
| | | | | | | |/ _| | |
| | | |_| | | | (_| | | Version 1.4.1 (2020-04-14) |
| _/ |\__'_|_|_|\__'_| | Official https://julialang.org/ release |
| |__/ | |
| </br> |
| julia> using Singularity |
| julia> generate_deffile() |
| julia> buildsif() |
| runningsudo singularity build –force container/projectcontainer.sif container/Singularity.pack |
| [sudo] password for steffen: |
| WARNING: 'nodev' mount option set on /tmp, it could be a source of failure during build process |
| INFO: Starting build... |
| INFO: Running setup scriptlet |
| ++ pwd |
| + dir=/home/steffen/Code/ContaineredExample |
| + git clone file:///home/steffen/Code/ContaineredExample /tmp/rootfs-ee0425d8-c83c-11ea-9229-eee13580103b/Project |
| Cloning into '/tmp/rootfs-ee0425d8-c83c-11ea-9229-eee13580103b/Project'... |
| remote: Enumerating objects: 49, done. |
| remote: Counting objects: 100% (49/49), done. |
| remote: Compressing objects: 100% (39/39), done. |
| remote: Total 49 (delta 12), reused 0 (delta 0), pack-reused 0 |
| Receiving objects: 100% (49/49), 8.31 KiB | 2.08 MiB/s, done. |
| Resolving deltas: 100% (12/12), done. |
| INFO: Running post scriptlet |
| + export JULIA_DEPOT_PATH=/user/.julia |
| + export PATH=/opt/julia/bin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin |
| + cd Project |
| + git checkout master -- src/ scripts/ Project.toml Manifest.toml |
| + julia --project -e using Pkg; Pkg.rm.(Any[]) |
| + julia --project -e using Pkg; Pkg.instantiate() |
| Cloning default registries into/user/.julia |
| Cloning registry from "https://github.com/JuliaRegistries/General.git" |
| Added registryGeneralto/user/.julia/registries/General |
| Installed Requires ── v1.0.1 |
| Installed UnPack ──── v1.0.1 |
| Installed Distances ─ v0.9.0 |
| Installed DrWatson ── v1.14.4 |
| Installed FileIO ──── v1.3.0 |
| Installed BSON ────── v0.2.6 |
| + julia --project -e using Pkg; Pkg.precompile() |
| Precompiling project... |
| + chmod -R a+rX /user/.julia |
| + chmod -R a+rX /Project/scripts |
| INFO: Adding runscript |
| INFO: Creating SIF file... |
| INFO: Build complete: container/projectcontainer.sif |
| Process(sudo singularity build –force container/projectcontainer.sif container/Singularity.pack`, ProcessExited(0)) |
| julia> exit() |
|
|
| ContaineredExample > git status |
| On branch master |
| Your branch is ahead of ‘origin/master’ by 3 commits. |
| (use “git push” to publish your local commits) |
| Changes not staged for commit: |
| (use “git add …” to update what will be committed) |
| (use “git restore …” to discard changes in working directory) |
| modified: .gitignore |
| Untracked files: |
| (use “git add …” to include in what will be committed) |
| container/ |
| no changes added to commit (use “git add” and/or “git commit -a”) |
|
|
| ContaineredExample > cd container |
| container > tree |
| . |
| ├── pack.log |
| ├── projectcontainer.sif |
| └── Singularity.pack |
|
|
| 0 directories, 3 files |
| container > du -sh projectcontainer.sif |
| 221M projectcontainer.sif |
| container > singularity run –bind=../data:/Project/data projectcontainer.sif intro.jl |
| container > cd .. |
| ContaineredExample > tree data container |
| data |
| ├── exp_pro |
| ├── exp_raw |
| └── sims |
| └── compute_x0=7.bson |
| container |
| ├── intro.jl-2020-07-17T160055.log |
| ├── pack.log |
| ├── projectcontainer.sif |
| └── Singularity.pack |
|
|
| 3 directories, 5 files |
| |
| ContaineredExample > master > tree -L 3 |
| . |
| ├── data |
| │ ├── exp_pro |
| │ ├── exp_raw |
| │ └── sims |
| ├── Manifest.toml |
| ├── notebooks |
| ├── old_result.bson |
| ├── papers |
| ├── plots |
| ├── Project.toml |
| ├── README.md |
| ├── _research |
| ├── scripts |
| │ └── intro.jl |
| └── src |
| └── ExampleModule |
| ├── Manifest.toml |
| ├── Project.toml |
| └── src |
|
|
| 12 directories, 8 files |
| ContaineredExample > master > julia –project |
| _ |
| |
| () | () (_) | |
| _ _ | | __ _ | Type “?” for help, “]?” for Pkg help. |
| | | | | | | |/ _| | |
| | | |_| | | | (_| | | Version 1.4.1 (2020-04-14) |
| _/ |\__'_|_|_|\__'_| | Official https://julialang.org/ release |
| |__/ | |
| </br> |
| julia> using Singularity |
| julia> generate_deffile() |
| julia> buildsif() |
| runningsudo singularity build –force container/projectcontainer.sif container/Singularity.pack |
| [sudo] password for steffen: |
| WARNING: 'nodev' mount option set on /tmp, it could be a source of failure during build process |
| INFO: Starting build... |
| INFO: Running setup scriptlet |
| ++ pwd |
| + dir=/home/steffen/Code/ContaineredExample |
| + git clone file:///home/steffen/Code/ContaineredExample /tmp/rootfs-ee0425d8-c83c-11ea-9229-eee13580103b/Project |
| Cloning into '/tmp/rootfs-ee0425d8-c83c-11ea-9229-eee13580103b/Project'... |
| remote: Enumerating objects: 49, done. |
| remote: Counting objects: 100% (49/49), done. |
| remote: Compressing objects: 100% (39/39), done. |
| remote: Total 49 (delta 12), reused 0 (delta 0), pack-reused 0 |
| Receiving objects: 100% (49/49), 8.31 KiB | 2.08 MiB/s, done. |
| Resolving deltas: 100% (12/12), done. |
| INFO: Running post scriptlet |
| + export JULIA_DEPOT_PATH=/user/.julia |
| + export PATH=/opt/julia/bin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin |
| + cd Project |
| + git checkout master -- src/ scripts/ Project.toml Manifest.toml |
| + julia --project -e using Pkg; Pkg.rm.(Any[]) |
| + julia --project -e using Pkg; Pkg.instantiate() |
| Cloning default registries into/user/.julia |
| Cloning registry from "https://github.com/JuliaRegistries/General.git" |
| Added registryGeneralto/user/.julia/registries/General |
| Installed Requires ── v1.0.1 |
| Installed UnPack ──── v1.0.1 |
| Installed Distances ─ v0.9.0 |
| Installed DrWatson ── v1.14.4 |
| Installed FileIO ──── v1.3.0 |
| Installed BSON ────── v0.2.6 |
| + julia --project -e using Pkg; Pkg.precompile() |
| Precompiling project... |
| + chmod -R a+rX /user/.julia |
| + chmod -R a+rX /Project/scripts |
| INFO: Adding runscript |
| INFO: Creating SIF file... |
| INFO: Build complete: container/projectcontainer.sif |
| Process(sudo singularity build –force container/projectcontainer.sif container/Singularity.pack`, ProcessExited(0)) |
| julia> exit() |
|
|
| ContaineredExample > git status |
| On branch master |
| Your branch is ahead of ‘origin/master’ by 3 commits. |
| (use “git push” to publish your local commits) |
| Changes not staged for commit: |
| (use “git add …” to update what will be committed) |
| (use “git restore …” to discard changes in working directory) |
| modified: .gitignore |
| Untracked files: |
| (use “git add …” to include in what will be committed) |
| container/ |
| no changes added to commit (use “git add” and/or “git commit -a”) |
|
|
| ContaineredExample > cd container |
| container > tree |
| . |
| ├── pack.log |
| ├── projectcontainer.sif |
| └── Singularity.pack |
|
|
| 0 directories, 3 files |
| container > du -sh projectcontainer.sif |
| 221M projectcontainer.sif |
| container > singularity run –bind=../data:/Project/data projectcontainer.sif intro.jl |
| container > cd .. |
| ContaineredExample > tree data container |
| data |
| ├── exp_pro |
| ├── exp_raw |
| └── sims |
| └── compute_x0=7.bson |
| container |
| ├── intro.jl-2020-07-17T160055.log |
| ├── pack.log |
| ├── projectcontainer.sif |
| └── Singularity.pack |
|
|
| 3 directories, 5 files |
| |
| ContaineredExample > master > tree -L 3 |
| . |
| ├── data |
| │ ├── exp_pro |
| │ ├── exp_raw |
| │ └── sims |
| ├── Manifest.toml |
| ├── notebooks |
| ├── old_result.bson |
| ├── papers |
| ├── plots |
| ├── Project.toml |
| ├── README.md |
| ├── _research |
| ├── scripts |
| │ └── intro.jl |
| └── src |
| └── ExampleModule |
| ├── Manifest.toml |
| ├── Project.toml |
| └── src |
|
|
| 12 directories, 8 files |
| ContaineredExample > master > julia –project |
| _ |
| |
| () | () (_) | |
| _ _ | | __ _ | Type “?” for help, “]?” for Pkg help. |
| | | | | | | |/ _| | |
| | | |_| | | | (_| | | Version 1.4.1 (2020-04-14) |
| _/ |\__'_|_|_|\__'_| | Official https://julialang.org/ release |
| |__/ | |
| </br> |
| julia> using Singularity |
| julia> generate_deffile() |
| julia> buildsif() |
| runningsudo singularity build –force container/projectcontainer.sif container/Singularity.pack |
| [sudo] password for steffen: |
| WARNING: 'nodev' mount option set on /tmp, it could be a source of failure during build process |
| INFO: Starting build... |
| INFO: Running setup scriptlet |
| ++ pwd |
| + dir=/home/steffen/Code/ContaineredExample |
| + git clone file:///home/steffen/Code/ContaineredExample /tmp/rootfs-ee0425d8-c83c-11ea-9229-eee13580103b/Project |
| Cloning into '/tmp/rootfs-ee0425d8-c83c-11ea-9229-eee13580103b/Project'... |
| remote: Enumerating objects: 49, done. |
| remote: Counting objects: 100% (49/49), done. |
| remote: Compressing objects: 100% (39/39), done. |
| remote: Total 49 (delta 12), reused 0 (delta 0), pack-reused 0 |
| Receiving objects: 100% (49/49), 8.31 KiB | 2.08 MiB/s, done. |
| Resolving deltas: 100% (12/12), done. |
| INFO: Running post scriptlet |
| + export JULIA_DEPOT_PATH=/user/.julia |
| + export PATH=/opt/julia/bin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin |
| + cd Project |
| + git checkout master -- src/ scripts/ Project.toml Manifest.toml |
| + julia --project -e using Pkg; Pkg.rm.(Any[]) |
| + julia --project -e using Pkg; Pkg.instantiate() |
| Cloning default registries into/user/.julia |
| Cloning registry from "https://github.com/JuliaRegistries/General.git" |
| Added registryGeneralto/user/.julia/registries/General |
| Installed Requires ── v1.0.1 |
| Installed UnPack ──── v1.0.1 |
| Installed Distances ─ v0.9.0 |
| Installed DrWatson ── v1.14.4 |
| Installed FileIO ──── v1.3.0 |
| Installed BSON ────── v0.2.6 |
| + julia --project -e using Pkg; Pkg.precompile() |
| Precompiling project... |
| + chmod -R a+rX /user/.julia |
| + chmod -R a+rX /Project/scripts |
| INFO: Adding runscript |
| INFO: Creating SIF file... |
| INFO: Build complete: container/projectcontainer.sif |
| Process(sudo singularity build –force container/projectcontainer.sif container/Singularity.pack`, ProcessExited(0)) |
| julia> exit() |
|
|
| ContaineredExample > git status |
| On branch master |
| Your branch is ahead of ‘origin/master’ by 3 commits. |
| (use “git push” to publish your local commits) |
| Changes not staged for commit: |
| (use “git add …” to update what will be committed) |
| (use “git restore …” to discard changes in working directory) |
| modified: .gitignore |
| Untracked files: |
| (use “git add …” to include in what will be committed) |
| container/ |
| no changes added to commit (use “git add” and/or “git commit -a”) |
|
|
| ContaineredExample > cd container |
| container > tree |
| . |
| ├── pack.log |
| ├── projectcontainer.sif |
| └── Singularity.pack |
|
|
| 0 directories, 3 files |
| container > du -sh projectcontainer.sif |
| 221M projectcontainer.sif |
| container > singularity run –bind=../data:/Project/data projectcontainer.sif intro.jl |
| container > cd .. |
| ContaineredExample > tree data container |
| data |
| ├── exp_pro |
| ├── exp_raw |
| └── sims |
| └── compute_x0=7.bson |
| container |
| ├── intro.jl-2020-07-17T160055.log |
| ├── pack.log |
| ├── projectcontainer.sif |
| └── Singularity.pack |
|
|
| 3 directories, 5 files |
| |
| ContaineredExample > master > tree -L 3 |
| . |
| ├── data |
| │ ├── exp_pro |
| │ ├── exp_raw |
| │ └── sims |
| ├── Manifest.toml |
| ├── notebooks |
| ├── old_result.bson |
| ├── papers |
| ├── plots |
| ├── Project.toml |
| ├── README.md |
| ├── _research |
| ├── scripts |
| │ └── intro.jl |
| └── src |
| └── ExampleModule |
| ├── Manifest.toml |
| ├── Project.toml |
| └── src |
|
|
| 12 directories, 8 files |
| ContaineredExample > master > julia –project |
| _ |
| |
| () | () (_) | |
| _ _ | | __ _ | Type “?” for help, “]?” for Pkg help. |
| | | | | | | |/ _| | |
| | | |_| | | | (_| | | Version 1.4.1 (2020-04-14) |
| _/ |\__'_|_|_|\__'_| | Official https://julialang.org/ release |
| |__/ | |
| </br> |
| julia> using Singularity |
| julia> generate_deffile() |
| julia> buildsif() |
| runningsudo singularity build –force container/projectcontainer.sif container/Singularity.pack |
| [sudo] password for steffen: |
| WARNING: 'nodev' mount option set on /tmp, it could be a source of failure during build process |
| INFO: Starting build... |
| INFO: Running setup scriptlet |
| ++ pwd |
| + dir=/home/steffen/Code/ContaineredExample |
| + git clone file:///home/steffen/Code/ContaineredExample /tmp/rootfs-ee0425d8-c83c-11ea-9229-eee13580103b/Project |
| Cloning into '/tmp/rootfs-ee0425d8-c83c-11ea-9229-eee13580103b/Project'... |
| remote: Enumerating objects: 49, done. |
| remote: Counting objects: 100% (49/49), done. |
| remote: Compressing objects: 100% (39/39), done. |
| remote: Total 49 (delta 12), reused 0 (delta 0), pack-reused 0 |
| Receiving objects: 100% (49/49), 8.31 KiB | 2.08 MiB/s, done. |
| Resolving deltas: 100% (12/12), done. |
| INFO: Running post scriptlet |
| + export JULIA_DEPOT_PATH=/user/.julia |
| + export PATH=/opt/julia/bin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin |
| + cd Project |
| + git checkout master -- src/ scripts/ Project.toml Manifest.toml |
| + julia --project -e using Pkg; Pkg.rm.(Any[]) |
| + julia --project -e using Pkg; Pkg.instantiate() |
| Cloning default registries into/user/.julia |
| Cloning registry from "https://github.com/JuliaRegistries/General.git" |
| Added registryGeneralto/user/.julia/registries/General |
| Installed Requires ── v1.0.1 |
| Installed UnPack ──── v1.0.1 |
| Installed Distances ─ v0.9.0 |
| Installed DrWatson ── v1.14.4 |
| Installed FileIO ──── v1.3.0 |
| Installed BSON ────── v0.2.6 |
| + julia --project -e using Pkg; Pkg.precompile() |
| Precompiling project... |
| + chmod -R a+rX /user/.julia |
| + chmod -R a+rX /Project/scripts |
| INFO: Adding runscript |
| INFO: Creating SIF file... |
| INFO: Build complete: container/projectcontainer.sif |
| Process(sudo singularity build –force container/projectcontainer.sif container/Singularity.pack`, ProcessExited(0)) |
| julia> exit() |
|
|
| ContaineredExample > git status |
| On branch master |
| Your branch is ahead of ‘origin/master’ by 3 commits. |
| (use “git push” to publish your local commits) |
| Changes not staged for commit: |
| (use “git add …” to update what will be committed) |
| (use “git restore …” to discard changes in working directory) |
| modified: .gitignore |
| Untracked files: |
| (use “git add …” to include in what will be committed) |
| container/ |
| no changes added to commit (use “git add” and/or “git commit -a”) |
|
|
| ContaineredExample > cd container |
| container > tree |
| . |
| ├── pack.log |
| ├── projectcontainer.sif |
| └── Singularity.pack |
|
|
| 0 directories, 3 files |
| container > du -sh projectcontainer.sif |
| 221M projectcontainer.sif |
| container > singularity run –bind=../data:/Project/data projectcontainer.sif intro.jl |
| container > cd .. |
| ContaineredExample > tree data container |
| data |
| ├── exp_pro |
| ├── exp_raw |
| └── sims |
| └── compute_x0=7.bson |
| container |
| ├── intro.jl-2020-07-17T160055.log |
| ├── pack.log |
| ├── projectcontainer.sif |
| └── Singularity.pack |
|
|
| 3 directories, 5 files |
| |
example project at github.com/Crown421/ContaineredExample
Reproduce results
| ContaineredExample > julia –project |
| _ |
| |
| () | () (_) | |
| _ _ | | __ _ | Type “?” for help, “]?” for Pkg help. |
| | | | | | | |/ | | |
| | | |_| | | | (_| | | Version 1.4.1 (2020-04-14) |
| _/ |\__'_|_|_|\__'_| | Official https://julialang.org/ release |
| |__/ | |
| </br> |
| julia> using DrWatson |
| </br> |
| julia> load("data/sims/compute_x0=7.bson") |
| Dict{Symbol,Any} with 5 entries: |
| :result => 42 |
| :params => (x0 = 7,) |
| :gitcommit => "v0.2-3-g83b59df" |
| :script => "scripts/intro.jl#12" |
| :gitpatch => "" |
| </br> |
| julia> load("old_result.bson") |
| Dict{Symbol,Any} with 5 entries: |
| :result => 43 |
| :params => (x0 = 7,) |
| :gitcommit => "v0.2-1-g185718b" |
| :script => "scripts/intro.jl#12" |
| :gitpatch => "" |
| </br> |
| julia> using Singularity |
| julia> recreatedata("old_result.bson") |
| julia> buildsif() |
| runningsudo singularity build –force container/projectcontainer.sif container/Singularity.pack |
| [sudo] password for steffen: |
| WARNING: 'nodev' mount option set on /tmp, it could be a source of failure during build process |
| INFO: Starting build... |
| INFO: Running setup scriptlet |
| ++ pwd |
| + dir=/home/steffen/Code/ContaineredExample |
| + git clone file:///home/steffen/Code/ContaineredExample /tmp/rootfs-0d3b9d22-c846-11ea-9d0d-eee13580103b/Project |
| Cloning into '/tmp/rootfs-0d3b9d22-c846-11ea-9d0d-eee13580103b/Project'... |
| remote: Enumerating objects: 49, done. |
| remote: Counting objects: 100% (49/49), done. |
| remote: Compressing objects: 100% (39/39), done. |
| remote: Total 49 (delta 12), reused 0 (delta 0), pack-reused 0 |
| Receiving objects: 100% (49/49), 8.31 KiB | 8.31 MiB/s, done. |
| Resolving deltas: 100% (12/12), done. |
| INFO: Running post scriptlet |
| + export JULIA_DEPOT_PATH=/user/.julia |
| + export PATH=/opt/julia/bin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin |
| + cd Project |
| + git checkout v0.2-1-g185718b -- src/ scripts/ Project.toml Manifest.toml |
| + julia --project -e using Pkg; Pkg.rm.(Any[]) |
| + julia --project -e using Pkg; Pkg.instantiate() |
| Cloning default registries into/user/.julia |
| Cloning registry from "https://github.com/JuliaRegistries/General.git" |
| Added registryGeneralto/user/.julia/registries/General |
| Installed Requires ── v1.0.1 |
| Installed UnPack ──── v1.0.1 |
| Installed Distances ─ v0.9.0 |
| Installed BSON ────── v0.2.6 |
| Installed FileIO ──── v1.3.0 |
| Installed DrWatson ── v1.14.4 |
| + julia --project -e using Pkg; Pkg.precompile() |
| Precompiling project... |
| + chmod -R a+rX /user/.julia |
| + chmod -R a+rX /Project/scripts |
| INFO: Adding runscript |
| INFO: Creating SIF file... |
| INFO: Build complete: container/projectcontainer.sif |
| Process(sudo singularity build –force container/projectcontainer.sif container/Singularity.pack, ProcessExited(0)) |
| </br> |
| julia> exit() |
| ContaineredExample > cd container |
| container > singularity run --bind=.:/Project/data projectcontainer.sif |
| ┌ Warning: The Git repository is dirty! Appending _dirty to the commit ID |
| └ @ DrWatson ~/.julia/packages/DrWatson/LGFKJ/src/saving_tools.jl:60 |
| container > julia --project |
| _ |
| _ _ _(_)_ | Documentation: https://docs.julialang.org |
| (_) | (_) (_) | |
| _ _ _| |_ __ _ | Type "?" for help, "]?" for Pkg help. |
| | | | | | | |/ _ | | |
| | | || | | | (_| | | Version 1.4.1 (2020-04-14) |
| |
| |__/ | |
|
|
| julia> using DrWatson |
| julia> load(“sims/compute_x0=7.bson”) |
| Dict{Symbol,Any} with 5 entries: |
| :result => 43 |
| :params => (x0 = 7,) |
| :gitcommit => “v0.2-3-g83b59df_dirty” |
| :script => “scripts/intro.jl |
| :gitpatch => “diff –git a/src/ExampleModule/src/ExampleModule.jl … |
| ContaineredExample > julia –project |
| _ |
| |
| () | () (_) | |
| _ _ | | __ _ | Type “?” for help, “]?” for Pkg help. |
| | | | | | | |/ | | |
| | | |_| | | | (_| | | Version 1.4.1 (2020-04-14) |
| _/ |\__'_|_|_|\__'_| | Official https://julialang.org/ release |
| |__/ | |
| </br> |
| julia> using DrWatson |
| </br> |
| julia> load("data/sims/compute_x0=7.bson") |
| Dict{Symbol,Any} with 5 entries: |
| :result => 42 |
| :params => (x0 = 7,) |
| :gitcommit => "v0.2-3-g83b59df" |
| :script => "scripts/intro.jl#12" |
| :gitpatch => "" |
| </br> |
| julia> load("old_result.bson") |
| Dict{Symbol,Any} with 5 entries: |
| :result => 43 |
| :params => (x0 = 7,) |
| :gitcommit => "v0.2-1-g185718b" |
| :script => "scripts/intro.jl#12" |
| :gitpatch => "" |
| </br> |
| julia> using Singularity |
| julia> recreatedata("old_result.bson") |
| julia> buildsif() |
| runningsudo singularity build –force container/projectcontainer.sif container/Singularity.pack |
| [sudo] password for steffen: |
| WARNING: 'nodev' mount option set on /tmp, it could be a source of failure during build process |
| INFO: Starting build... |
| INFO: Running setup scriptlet |
| ++ pwd |
| + dir=/home/steffen/Code/ContaineredExample |
| + git clone file:///home/steffen/Code/ContaineredExample /tmp/rootfs-0d3b9d22-c846-11ea-9d0d-eee13580103b/Project |
| Cloning into '/tmp/rootfs-0d3b9d22-c846-11ea-9d0d-eee13580103b/Project'... |
| remote: Enumerating objects: 49, done. |
| remote: Counting objects: 100% (49/49), done. |
| remote: Compressing objects: 100% (39/39), done. |
| remote: Total 49 (delta 12), reused 0 (delta 0), pack-reused 0 |
| Receiving objects: 100% (49/49), 8.31 KiB | 8.31 MiB/s, done. |
| Resolving deltas: 100% (12/12), done. |
| INFO: Running post scriptlet |
| + export JULIA_DEPOT_PATH=/user/.julia |
| + export PATH=/opt/julia/bin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin |
| + cd Project |
| + git checkout v0.2-1-g185718b -- src/ scripts/ Project.toml Manifest.toml |
| + julia --project -e using Pkg; Pkg.rm.(Any[]) |
| + julia --project -e using Pkg; Pkg.instantiate() |
| Cloning default registries into/user/.julia |
| Cloning registry from "https://github.com/JuliaRegistries/General.git" |
| Added registryGeneralto/user/.julia/registries/General |
| Installed Requires ── v1.0.1 |
| Installed UnPack ──── v1.0.1 |
| Installed Distances ─ v0.9.0 |
| Installed BSON ────── v0.2.6 |
| Installed FileIO ──── v1.3.0 |
| Installed DrWatson ── v1.14.4 |
| + julia --project -e using Pkg; Pkg.precompile() |
| Precompiling project... |
| + chmod -R a+rX /user/.julia |
| + chmod -R a+rX /Project/scripts |
| INFO: Adding runscript |
| INFO: Creating SIF file... |
| INFO: Build complete: container/projectcontainer.sif |
| Process(sudo singularity build –force container/projectcontainer.sif container/Singularity.pack, ProcessExited(0)) |
| </br> |
| julia> exit() |
| ContaineredExample > cd container |
| container > singularity run --bind=.:/Project/data projectcontainer.sif |
| ┌ Warning: The Git repository is dirty! Appending _dirty to the commit ID |
| └ @ DrWatson ~/.julia/packages/DrWatson/LGFKJ/src/saving_tools.jl:60 |
| container > julia --project |
| _ |
| _ _ _(_)_ | Documentation: https://docs.julialang.org |
| (_) | (_) (_) | |
| _ _ _| |_ __ _ | Type "?" for help, "]?" for Pkg help. |
| | | | | | | |/ _ | | |
| | | || | | | (_| | | Version 1.4.1 (2020-04-14) |
| |
| |__/ | |
|
|
| julia> using DrWatson |
| julia> load(“sims/compute_x0=7.bson”) |
| Dict{Symbol,Any} with 5 entries: |
| :result => 43 |
| :params => (x0 = 7,) |
| :gitcommit => “v0.2-3-g83b59df_dirty” |
| :script => “scripts/intro.jl |
| :gitpatch => “diff –git a/src/ExampleModule/src/ExampleModule.jl … |
| ContaineredExample > julia –project |
| _ |
| |
| () | () (_) | |
| _ _ | | __ _ | Type “?” for help, “]?” for Pkg help. |
| | | | | | | |/ | | |
| | | |_| | | | (_| | | Version 1.4.1 (2020-04-14) |
| _/ |\__'_|_|_|\__'_| | Official https://julialang.org/ release |
| |__/ | |
| </br> |
| julia> using DrWatson |
| </br> |
| julia> load("data/sims/compute_x0=7.bson") |
| Dict{Symbol,Any} with 5 entries: |
| :result => 42 |
| :params => (x0 = 7,) |
| :gitcommit => "v0.2-3-g83b59df" |
| :script => "scripts/intro.jl#12" |
| :gitpatch => "" |
| </br> |
| julia> load("old_result.bson") |
| Dict{Symbol,Any} with 5 entries: |
| :result => 43 |
| :params => (x0 = 7,) |
| :gitcommit => "v0.2-1-g185718b" |
| :script => "scripts/intro.jl#12" |
| :gitpatch => "" |
| </br> |
| julia> using Singularity |
| julia> recreatedata("old_result.bson") |
| julia> buildsif() |
| runningsudo singularity build –force container/projectcontainer.sif container/Singularity.pack |
| [sudo] password for steffen: |
| WARNING: 'nodev' mount option set on /tmp, it could be a source of failure during build process |
| INFO: Starting build... |
| INFO: Running setup scriptlet |
| ++ pwd |
| + dir=/home/steffen/Code/ContaineredExample |
| + git clone file:///home/steffen/Code/ContaineredExample /tmp/rootfs-0d3b9d22-c846-11ea-9d0d-eee13580103b/Project |
| Cloning into '/tmp/rootfs-0d3b9d22-c846-11ea-9d0d-eee13580103b/Project'... |
| remote: Enumerating objects: 49, done. |
| remote: Counting objects: 100% (49/49), done. |
| remote: Compressing objects: 100% (39/39), done. |
| remote: Total 49 (delta 12), reused 0 (delta 0), pack-reused 0 |
| Receiving objects: 100% (49/49), 8.31 KiB | 8.31 MiB/s, done. |
| Resolving deltas: 100% (12/12), done. |
| INFO: Running post scriptlet |
| + export JULIA_DEPOT_PATH=/user/.julia |
| + export PATH=/opt/julia/bin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin |
| + cd Project |
| + git checkout v0.2-1-g185718b -- src/ scripts/ Project.toml Manifest.toml |
| + julia --project -e using Pkg; Pkg.rm.(Any[]) |
| + julia --project -e using Pkg; Pkg.instantiate() |
| Cloning default registries into/user/.julia |
| Cloning registry from "https://github.com/JuliaRegistries/General.git" |
| Added registryGeneralto/user/.julia/registries/General |
| Installed Requires ── v1.0.1 |
| Installed UnPack ──── v1.0.1 |
| Installed Distances ─ v0.9.0 |
| Installed BSON ────── v0.2.6 |
| Installed FileIO ──── v1.3.0 |
| Installed DrWatson ── v1.14.4 |
| + julia --project -e using Pkg; Pkg.precompile() |
| Precompiling project... |
| + chmod -R a+rX /user/.julia |
| + chmod -R a+rX /Project/scripts |
| INFO: Adding runscript |
| INFO: Creating SIF file... |
| INFO: Build complete: container/projectcontainer.sif |
| Process(sudo singularity build –force container/projectcontainer.sif container/Singularity.pack, ProcessExited(0)) |
| </br> |
| julia> exit() |
| ContaineredExample > cd container |
| container > singularity run --bind=.:/Project/data projectcontainer.sif |
| ┌ Warning: The Git repository is dirty! Appending _dirty to the commit ID |
| └ @ DrWatson ~/.julia/packages/DrWatson/LGFKJ/src/saving_tools.jl:60 |
| container > julia --project |
| _ |
| _ _ _(_)_ | Documentation: https://docs.julialang.org |
| (_) | (_) (_) | |
| _ _ _| |_ __ _ | Type "?" for help, "]?" for Pkg help. |
| | | | | | | |/ _ | | |
| | | || | | | (_| | | Version 1.4.1 (2020-04-14) |
| |
| |__/ | |
|
|
| julia> using DrWatson |
| julia> load(“sims/compute_x0=7.bson”) |
| Dict{Symbol,Any} with 5 entries: |
| :result => 43 |
| :params => (x0 = 7,) |
| :gitcommit => “v0.2-3-g83b59df_dirty” |
| :script => “scripts/intro.jl |
| :gitpatch => “diff –git a/src/ExampleModule/src/ExampleModule.jl … |
| ContaineredExample > julia –project |
| _ |
| |
| () | () (_) | |
| _ _ | | __ _ | Type “?” for help, “]?” for Pkg help. |
| | | | | | | |/ | | |
| | | |_| | | | (_| | | Version 1.4.1 (2020-04-14) |
| _/ |\__'_|_|_|\__'_| | Official https://julialang.org/ release |
| |__/ | |
| </br> |
| julia> using DrWatson |
| </br> |
| julia> load("data/sims/compute_x0=7.bson") |
| Dict{Symbol,Any} with 5 entries: |
| :result => 42 |
| :params => (x0 = 7,) |
| :gitcommit => "v0.2-3-g83b59df" |
| :script => "scripts/intro.jl#12" |
| :gitpatch => "" |
| </br> |
| julia> load("old_result.bson") |
| Dict{Symbol,Any} with 5 entries: |
| :result => 43 |
| :params => (x0 = 7,) |
| :gitcommit => "v0.2-1-g185718b" |
| :script => "scripts/intro.jl#12" |
| :gitpatch => "" |
| </br> |
| julia> using Singularity |
| julia> recreatedata("old_result.bson") |
| julia> buildsif() |
| runningsudo singularity build –force container/projectcontainer.sif container/Singularity.pack |
| [sudo] password for steffen: |
| WARNING: 'nodev' mount option set on /tmp, it could be a source of failure during build process |
| INFO: Starting build... |
| INFO: Running setup scriptlet |
| ++ pwd |
| + dir=/home/steffen/Code/ContaineredExample |
| + git clone file:///home/steffen/Code/ContaineredExample /tmp/rootfs-0d3b9d22-c846-11ea-9d0d-eee13580103b/Project |
| Cloning into '/tmp/rootfs-0d3b9d22-c846-11ea-9d0d-eee13580103b/Project'... |
| remote: Enumerating objects: 49, done. |
| remote: Counting objects: 100% (49/49), done. |
| remote: Compressing objects: 100% (39/39), done. |
| remote: Total 49 (delta 12), reused 0 (delta 0), pack-reused 0 |
| Receiving objects: 100% (49/49), 8.31 KiB | 8.31 MiB/s, done. |
| Resolving deltas: 100% (12/12), done. |
| INFO: Running post scriptlet |
| + export JULIA_DEPOT_PATH=/user/.julia |
| + export PATH=/opt/julia/bin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin |
| + cd Project |
| + git checkout v0.2-1-g185718b -- src/ scripts/ Project.toml Manifest.toml |
| + julia --project -e using Pkg; Pkg.rm.(Any[]) |
| + julia --project -e using Pkg; Pkg.instantiate() |
| Cloning default registries into/user/.julia |
| Cloning registry from "https://github.com/JuliaRegistries/General.git" |
| Added registryGeneralto/user/.julia/registries/General |
| Installed Requires ── v1.0.1 |
| Installed UnPack ──── v1.0.1 |
| Installed Distances ─ v0.9.0 |
| Installed BSON ────── v0.2.6 |
| Installed FileIO ──── v1.3.0 |
| Installed DrWatson ── v1.14.4 |
| + julia --project -e using Pkg; Pkg.precompile() |
| Precompiling project... |
| + chmod -R a+rX /user/.julia |
| + chmod -R a+rX /Project/scripts |
| INFO: Adding runscript |
| INFO: Creating SIF file... |
| INFO: Build complete: container/projectcontainer.sif |
| Process(sudo singularity build –force container/projectcontainer.sif container/Singularity.pack, ProcessExited(0)) |
| </br> |
| julia> exit() |
| ContaineredExample > cd container |
| container > singularity run --bind=.:/Project/data projectcontainer.sif |
| ┌ Warning: The Git repository is dirty! Appending _dirty to the commit ID |
| └ @ DrWatson ~/.julia/packages/DrWatson/LGFKJ/src/saving_tools.jl:60 |
| container > julia --project |
| _ |
| _ _ _(_)_ | Documentation: https://docs.julialang.org |
| (_) | (_) (_) | |
| _ _ _| |_ __ _ | Type "?" for help, "]?" for Pkg help. |
| | | | | | | |/ _ | | |
| | | || | | | (_| | | Version 1.4.1 (2020-04-14) |
| |
| |__/ | |
|
|
| julia> using DrWatson |
| julia> load(“sims/compute_x0=7.bson”) |
| Dict{Symbol,Any} with 5 entries: |
| :result => 43 |
| :params => (x0 = 7,) |
| :gitcommit => “v0.2-3-g83b59df_dirty” |
| :script => “scripts/intro.jl |
| :gitpatch => “diff –git a/src/ExampleModule/src/ExampleModule.jl … |
| ContaineredExample > julia –project |
| _ |
| |
| () | () (_) | |
| _ _ | | __ _ | Type “?” for help, “]?” for Pkg help. |
| | | | | | | |/ | | |
| | | |_| | | | (_| | | Version 1.4.1 (2020-04-14) |
| _/ |\__'_|_|_|\__'_| | Official https://julialang.org/ release |
| |__/ | |
| </br> |
| julia> using DrWatson |
| </br> |
| julia> load("data/sims/compute_x0=7.bson") |
| Dict{Symbol,Any} with 5 entries: |
| :result => 42 |
| :params => (x0 = 7,) |
| :gitcommit => "v0.2-3-g83b59df" |
| :script => "scripts/intro.jl#12" |
| :gitpatch => "" |
| </br> |
| julia> load("old_result.bson") |
| Dict{Symbol,Any} with 5 entries: |
| :result => 43 |
| :params => (x0 = 7,) |
| :gitcommit => "v0.2-1-g185718b" |
| :script => "scripts/intro.jl#12" |
| :gitpatch => "" |
| </br> |
| julia> using Singularity |
| julia> recreatedata("old_result.bson") |
| julia> buildsif() |
| runningsudo singularity build –force container/projectcontainer.sif container/Singularity.pack |
| [sudo] password for steffen: |
| WARNING: 'nodev' mount option set on /tmp, it could be a source of failure during build process |
| INFO: Starting build... |
| INFO: Running setup scriptlet |
| ++ pwd |
| + dir=/home/steffen/Code/ContaineredExample |
| + git clone file:///home/steffen/Code/ContaineredExample /tmp/rootfs-0d3b9d22-c846-11ea-9d0d-eee13580103b/Project |
| Cloning into '/tmp/rootfs-0d3b9d22-c846-11ea-9d0d-eee13580103b/Project'... |
| remote: Enumerating objects: 49, done. |
| remote: Counting objects: 100% (49/49), done. |
| remote: Compressing objects: 100% (39/39), done. |
| remote: Total 49 (delta 12), reused 0 (delta 0), pack-reused 0 |
| Receiving objects: 100% (49/49), 8.31 KiB | 8.31 MiB/s, done. |
| Resolving deltas: 100% (12/12), done. |
| INFO: Running post scriptlet |
| + export JULIA_DEPOT_PATH=/user/.julia |
| + export PATH=/opt/julia/bin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin |
| + cd Project |
| + git checkout v0.2-1-g185718b -- src/ scripts/ Project.toml Manifest.toml |
| + julia --project -e using Pkg; Pkg.rm.(Any[]) |
| + julia --project -e using Pkg; Pkg.instantiate() |
| Cloning default registries into/user/.julia |
| Cloning registry from "https://github.com/JuliaRegistries/General.git" |
| Added registryGeneralto/user/.julia/registries/General |
| Installed Requires ── v1.0.1 |
| Installed UnPack ──── v1.0.1 |
| Installed Distances ─ v0.9.0 |
| Installed BSON ────── v0.2.6 |
| Installed FileIO ──── v1.3.0 |
| Installed DrWatson ── v1.14.4 |
| + julia --project -e using Pkg; Pkg.precompile() |
| Precompiling project... |
| + chmod -R a+rX /user/.julia |
| + chmod -R a+rX /Project/scripts |
| INFO: Adding runscript |
| INFO: Creating SIF file... |
| INFO: Build complete: container/projectcontainer.sif |
| Process(sudo singularity build –force container/projectcontainer.sif container/Singularity.pack, ProcessExited(0)) |
| </br> |
| julia> exit() |
| ContaineredExample > cd container |
| container > singularity run --bind=.:/Project/data projectcontainer.sif |
| ┌ Warning: The Git repository is dirty! Appending _dirty to the commit ID |
| └ @ DrWatson ~/.julia/packages/DrWatson/LGFKJ/src/saving_tools.jl:60 |
| container > julia --project |
| _ |
| _ _ _(_)_ | Documentation: https://docs.julialang.org |
| (_) | (_) (_) | |
| _ _ _| |_ __ _ | Type "?" for help, "]?" for Pkg help. |
| | | | | | | |/ _ | | |
| | | || | | | (_| | | Version 1.4.1 (2020-04-14) |
| |
| |__/ | |
|
|
| julia> using DrWatson |
| julia> load(“sims/compute_x0=7.bson”) |
| Dict{Symbol,Any} with 5 entries: |
| :result => 43 |
| :params => (x0 = 7,) |
| :gitcommit => “v0.2-3-g83b59df_dirty” |
| :script => “scripts/intro.jl |
| :gitpatch => “diff –git a/src/ExampleModule/src/ExampleModule.jl … |
Outlook 
- soon
containers (basically working)
- experimental:
- shallow clones in container
- next
- improved robustness and tests
- support for multiple def files
(See JuliaCon Talk )
