It is recommended to split a huge Rust project into crates and manage them in a workspace. I’m currently working on a project which consists of about 60 crates. It works well so far until I try to publish these crates to crates.io.
I will list the problems I have met and solutions or workarounds I have adopted.
Crates.io has some requirements on Cargo.toml files:
- Fields like license, authors, description, homepage, and repository are required.
- Local crates via
pathmust specify the version as well so the dependencies can be resolved via crates.io.
cargo publish --dry-run will verify whether the file Cargo.toml is valid.
cargo publish --all
Cargo does not support publishing all local packages yet. It means that I have to cd into each crate and publish them separately.
There are two major issues here.
The first, if a crate
foo depends on
bar, a version of
foo's requirement must be available in crates.io before publishing
foo. In the project I’m working on, all crates share the same version, and they lock the local dependencies using the exact version such as
bar = "= 0.0.1". If I want to publish these crates, I must topologically sort them by dependencies first and publish them in that order. The newly published crate has a delay of up to 10 seconds before it is searchable. In my publish script, I’ll retry several times when
cargo complaints that it fails to resolve a dependency.
Second, it is not atomic to publish the whole workspace. If I successfully published
bar 0.0.1 and later fails to publish
foo 0.0.1 because of a bug in
bar 0.0.1, I have to yank
bar 0.0.1, bump the version, and re-publish both
dev-dependencies may introduce cyclic dependencies. Take the example again that the crate
foo depends on
bar. This time the test cases of
bar depend on
foo. Cargo can resolve these cyclic dependencies because it does not need the
dev-dependencies to build both
bar, so it can build
foo, and then
bar test cases in order.
cargo publish requires both dependencies and
dev-dependencies are available in crates.io, now cyclic dependencies will cause problems.
It’s tedious to arrange the crates to resolve the cyclic dependencies, so I adopted this workaround to remove all the
dev-dependencies before publish and run
cargo publish --allow-dirty to ignore the dirty git working directory.
It Is Very Slow to Publish
Finally, it is really slow to publish 60 crates in a big project. It seems that they will not share the target directory and each crate uses its own directory inside
target/publish. If the workspace has three crates,
foo depends on
bar depends on
baz, cargo will take quadratic time to publish all the crates:
bar, then publish
foo, then publish
target/publish also takes a lot of disk space. After I published 60 crates, the folder occupied about 80G storage. If the publish host has limited disk space, the script must clean up the folder regularly.