Native app development with sttp and Scala Toolkit

Intro

The Scala ecosystem is undergoing exciting changes. As a language for the JVM, it offers numerous benefits, but doesn’t fit cases where one needs fast startup or a low memory footprint. It would be great to develop in this remarkable language and build lightweight VM-free programs. That’s where Scala Native is making significant progress, with growing support from the libraries. In this short example, we'll explore how to write a native program that uses the powerful sttp-client library for HTTP communication and Cats Effect for a rich effect system, all achievable with a simple setup in scala-cli. We’ll use Scala 3.3.0, scala-cli 1.0.2, and Scala Native 0.4.14.

Warmup: A JVM version

The Scala Toolkit documentation provides good examples. Let’s base our warmup code on one of them:

//> using toolkit latest

import sttp.client4.quick.*
import sttp.client4.Response

@main def sttpExample: Unit =
  val response: Response[String] = quickRequest
    .get(uri"http://icanhazip.com")
    .send()

  println(response.body)

As you can see, there’s no need to specify any explicit library dependency. The sttp client is included in scala-toolkit out of the box, just as some other powerful Scala tools. They include ujson and upickle for JSON parsing and (de)serialization, os for filesystem operations, and MUnit for convenient testing.

The //> using toolkit latest special clause is all that scala-cli needs to include these libraries in your app.

Save the file as sttp-jvm.scala and it’s ready to run with scala-cli sttp-jvm.scala.

Setting up the native tooling

What we’re after is the possibility of building Scala programs that use scala-toolkit (especially sttp) into native images, which are blazing fast and free of the burden of the JVM. Native builds may be difficult to set up because they often depend on libraries outside the JVM ecosystem. The Scala Native documentation contains instructions on configuring the system, especially clang and runtime dependencies.

These dependencies should be enough to write in Scala, but for sttp, you additionally need libcurl and libidn2 native libraries. They may be already present, depending on the OS/distribution. Try to run the examples and return here if your system complains about missing dependencies.

As a Nix user, I have prepared a shell configuration based on the Scala Native Nix shell. It provides all the dependencies for mu Ubuntu setup:

let
  pkgs = import <nixpkgs> { };
  stdenv = pkgs.stdenv;
in
rec {
  clangEnv = stdenv.mkDerivation rec {
    name = "clang-env";
    shellHook = ''
    alias cls=clear
    '';
    LLVM_BIN = pkgs.clang + "/bin";
    buildInputs = with pkgs; [
    scala-cli

    # for scala-native
    stdenv
    sbt
    openjdk
    boehmgc
    libunwind
    clang
    zlib

    # for sttp
    libidn2
    curl
    ];
    LD_LIBRARY_PATH = "${pkgs.libidn2.out}/lib:${pkgs.curl.out}/lib";
  };
}

It’s the default scala-native shell extended with two mentioned libraries. The important path is LD_LIBRARY_PATH, which has to be specified to make the libraries available for dynamic linking when running the shell with nix-shell scala-native.nix -A clangEnv. You’ll find the latest shell version in the GH project repository.

If you’re not into Nix, follow your operating system setup on the Scala Native website, and add libidn2 and curl dependencies using apt-get or another preferred package manager.

The simplest native example: Curl backend

Let’s get straight into the first example:

//> using toolkit latest

import sttp.client4.*
import sttp.client4.curl.*
import sttp.client4.Response

@main def sttpExample: Unit =
  val backend = CurlBackend()
  val response: Response[String] = quickRequest
    .get(uri"http://icanhazip.com")
    .send(backend)

  println(response.body)

Notice the import sttp.client4.curl.* import. This package is included in sttp-core, a part of scala-toolkit, but only in the artifact dedicated to Scala Native (core_native0.4). Running scala-cli on this script for JVM would fail if executed in a standard way for the JVM. Thanks to this import, we can use CurlBackend, which is the only backend supported for Scala Native at the moment. Our scala-cli execution needs a --native flag then, so let’s run the build with

scala-cli --power package --native sttp.scala -o sttp-curl -f

The --power option allows using extended features like package, for building various kinds of output formats, like JAR, native executable, docker image, etc.
The -f option forces overwriting the file if it already exists.

And voila. You now have a native executable, ./sttp-curl, which uses sttp with and native libraries to send a request and process a response. Explore the documentation of sttp client and upickle to learn how to do more sophisticated operations and work with request/response serialization and error handling.

Adding Cats Effect 3

If you’re a more advanced Scala user, you might want to write native programs using effects and powerful composable abstractions provided by libraries like Cats Effect. Fortunately, CE3 is available for Scala Native as well, so you may leverage it without too much effort:

//> using toolkit latest
//> using dep org.typelevel::cats-effect_native0.4:3.5.1  

import cats.effect.{ IO, IOApp }
import sttp.client4.*
import sttp.client4.curl.*
import sttp.client4.Response

object sttp4Native extends IOApp.Simple:
    val backend = CurlBackend()
    val sendRequest: IO[Response[String]] =
    IO.blocking(
        quickRequest
        .get(uri"http://icanhazip.com")
        .send(backend)
    )

    val run = IO.println("Sending...") >>
    sendRequest.flatMap(response =>
        IO.println(response.body)
    )

A Cats-Effect-based backend is also planned as an sttp module, so this should become even more seamless in the near future.

Building with sbt

Using scala-cli for simple apps is a great and easy new way to approach the ecosystem. If you’d like to build a native app using good old sbt, there’s a plugin you can add to project/plugins.sbt:

addSbtPlugin("org.scala-native" % "sbt-scala-native" % "0.4.14")

Then, make sure to specify the dependencies properly, as well as additional compiler/linker options in your build.sbt file:

scalaVersion := "3.3.0"

enablePlugins(ScalaNativePlugin)

// set to Debug for compilation details (Info is default)
logLevel := Level.Info

// import to add Scala Native options
import scala.scalanative.build._

// defaults set with common options shown
nativeConfig ~= { c =>
  c.withLTO(LTO.none) // thin
    .withMode(Mode.releaseFast) // releaseFast
    .withGC(GC.commix) // commix
}

libraryDependencies ++= List(
  "com.softwaremill.sttp.client4" %%% "core" % "4.0.0-M2",
  "org.typelevel" %%% "cats-effect" % "3.5.1"
)

Notice the atypical %%% operator, which is specific for native dependencies. The sbt plugin will translate it to ”com.softwaremill.sttp.client4” %% “core_native0.4”, as that’s the actual name of the artifact.

The core module of sttp provides the CurlBackend, I also add cats-effect to demonstrate how to integrate other libraries. For more information about how to tweak specific scala-native settings using nativeConfig, check the official documentation.

The entire showcase project is available on GitHub.

If you run compile followed by nativeLink in the console, you will be asked which main object to use. Select the plain or Cats Effect versions, and wait for the binary to be built. You’ll find it in target/scala-3.3.0/sttp-native-sbt-out.

Wrapping up

As we can see, Scala Native, although still quite experimental, is already very useful and accessible. This progress makes the future of Scala really promising. It also makes the ecosystem friendly for newcomers, as building native apps with scala-cli and scala-toolkit makes the experience impressively straightforward.

GitHub repo for scala-cli examples: https://github.com/softwaremill/sttp-native-cli
GitHub repo for sbt examples: https://github.com/softwaremill/sttp-native-sbt