mldonkey: opam, dune and ocaml 5

One of the things mldonkey is missing is ocaml 5 support, so I decided to try to port it.

In the docker image I currently provide on Docker Hub, I typically use older distros, so that I can use the ocaml compiler provided in the repos. This is changing with the latest updates.

Opam

The first step I took towards ocaml 5 was to move to opam, the ocaml package manager. That gave me more freedom in the choice of the version of the ocaml compiler.

In the docker image git repo, you can now see that opam is used to select the ocaml compiler version and to install the dependencies.

Dune

With the new build scripts, I tried to compile mldonkey with the ocaml 5 compiler. With the existing build system however (configure+make), I faced a few weird errors which seemed a bit long to fix.

Another thing that mldonkey seemed to miss was a migration to dune, the ocaml build system. That could give some benefits, so I thought I could take that path first.

After some days, I successfully compiled mldonkey with dune. This is an approximate list of the changes I made:

• configure script is not used anymore: dune expects dependencies and native libs to be installed.
• Configuration is only approximate and computed using a python script.
• No conditional build of dependencies: all the dependencies are now mandatory.
• I ported most modules, but not all. Some are so outdated that may not even be used by anyone anymore.
• Some external dependencies were copy-pasted into the mldonkey repo: I removed that code from the mldonkey repo and I referenced those through opam (ocurl, camlp-streams, bitstring).
• Initialization and link in dune works differently from configure+make. This forced me to change the initialization of mldonkey considerably.
• During the build, some ml files are generated on the fly: in most cases I ported this behavior, in a few cases I wasn’t able to do it because of changes in the bitstring module. In that case, I had to generate the code and store the generated code in the repo.
• Dropped support for any OS except Linux.

These changes are available in this mldonkey branch: https://github.com/carlonluca/mldonkey/tree/dune.

After this, I moved to the ocaml 5 port itself.

Ocaml 5

After migrating to dune, porting to ocaml 5 seemed much simpler. In a few hours, I got a working version. The build succeeds up to ocaml version 5.3.0, which is the most recent I could find in opam. The ocaml5 branch is here: https://github.com/carlonluca/mldonkey/tree/ocaml5.

These are the changes:

• I removed the xml-light code and I referenced the latest version of this project using opam.
• I updated the syntax of the entire C/ocaml code.
• I removed the syslog code and I referenced the latest version of the syslog module from opam.
• I added camomile to properly handle UTF-8.
• I was also able to update ocurl to the latest version.

Benefits

I can see some benefits from these changes:

• using opam allows to use whatever ocaml version I want to use during the build process. I do not need to use the distro-provided version anymore.
• Using opam/dune allows to add dependencies very quickly. This was cumbersome with configure + make.
• Using dune allows to build with multiple cores: building with parallel jobs was not working well with configure + make.
• Porting to ocaml 5 keeps mldonkey up to date with latest technologies, allows to build it on recent distros and benefits from ocaml improvements.

Does It Still Work?

Difficult to say. I do not use most of the modules. I am currently using it for my needs. You can provide feedback/patches if you want.

My recent projects of modern clients for the web and Linux/Android should still properly work with this new version.

How do I Get It?

At the moment, I do not provide binaries of the updated version. I, however, publish a docker image for mldonkey here: https://hub.docker.com/r/carlonluca/mldonkey. The relevant tags are dune and ocaml5. The ocaml 5 is the one you want to try if you want mldonkey built against ocaml 5. The dune tag is mldonkey running on ocaml 4.14.2, but built with the newer build system.

Please note that these two images already includes all the latest developments I uploaded in my mldonkey fork here: https://github.com/carlonluca/mldonkey. For example, these images already include curl to interface with HTTPS webservers and trackers.

Building

After these changes, mldonkey cannot be built using configure and make anymore. You’ll need opam, dune and the ocaml compiler. For a Ubuntu based OS, this is how I build it:

$ # Install some deps
$ sudo apt-get install ca-certificates \
    libcurl4-gnutls-dev \
    libz-dev \
    libbz2-dev \
    libmagic-dev \
    libnatpmp-dev \
    libupnp-dev \
    libgd-dev \
    libminiupnpc-dev \
    librsvg2-dev \
    libc6-dev \
    libcrypto++-dev \
    zlib1g-dev \
    m4 \
    opam \
    build-essential \
    ca-certificates \
    python-is-python3

$ # Download mldonkey source code
$ cd mldonkey

$ # Generate the configuration
$ python autoconf.py

$ # Init context
$ opam init --disable-sandboxing --bare --yes --jobs=$(nproc)
$ eval $(opam env)
$ opam switch create --yes --jobs=$(nproc) 5.3.0
$ eval $(opam env --switch=5.3.0)

$ # Install dependencies
$ opam install . --deps-only --yes --jobs=$(nproc)

$ # Build mldonkey
$ opam exec -- dune build --profile release

Upstreaming Changes

Not for the moment. The upstream project is moving slowly and has diverged much from my fork. Do not expect changes to be upstreamed soon.

Source Code

• My fork of mldonkey: https://github.com/carlonluca/mldonkey.
  • Branch dune: mldonkey port to dune, still for ocaml 4.14.
  • Branch ocaml5: mldonkey port to dune and supporting ocaml 5.
• Docker image for mldonkey: https://github.com/carlonluca/docker-mldonkey
  • Branch dune: mldonkey built with dune and ocaml 4.14.
  • Branch ocaml5: mldonkey built with dune and ocaml 5.

Final Note

Please note that I am not an ocaml developer, so do not expect a stable port here. Use this fork at your own risk and be prepared to troubles. If you test the new updates, feedback is welcome.

Have fun! 😉