Building Partially Static Libraries with Cabal

10-minute read
Programming
ghc • cabal • haskell • programming

One of the things that took me way to long to figure out in haskell land is how to build libraries that play well with other languages. Surely its easy? I mean, haskell has a lovely C interop and cabal, since version 2.0, got given the foreign-library stanza. But the libraries I was generating were awful. They had hundreds of shared libraries dynamically linked in all of which were compiler specific!

I shall give you an example. I have created an empty haskell library using cabal. It has one dependency, attoparsec. Seems reasonable enough. My cabal file is using a foreign-library stanza. Let’s ldd the .so output:

$ ldd testlib.so.1.0.0
	linux-vdso.so.1 (0x00007ffe83b15000)
	libHSrts-ghc9.2.1.so => /home/james/.ghcup/ghc/9.2.1-fPIC/lib64/ghc-9.2.1/rts/libHSrts-ghc9.2.1.so (0x00007f9e05ee0000)
	libffi.so.7 => /home/james/.ghcup/ghc/9.2.1-fPIC/lib64/ghc-9.2.1/rts/libffi.so.7 (0x00007f9e05cd1000)
	libHSattoparsec-0.14.3-e6461e438bc9e9bfa7ba26dd6e05f2876ea3ba961055dc727c8c9ca7654dc8e7-ghc9.2.1.so => /home/james/.cabal/store/ghc-9.2.1/attoparsec-0.14.3-e6461e438bc9e9bfa7ba26dd6e05f2876ea3ba961055dc727c8c9ca7654dc8e7/lib/libHSattoparsec-0.14.3-e6461e438bc9e9bfa7ba26dd6e05f2876ea3ba961055dc727c8c9ca7654dc8e7-ghc9.2.1.so (0x00007f9e05922000)
	libHSscientific-0.3.7.0-0ef75f1ad39504acbad57cab997eace92bdef38cb831f001e340f18e7da20c68-ghc9.2.1.so => /home/james/.cabal/store/ghc-9.2.1/scientific-0.3.7.0-0ef75f1ad39504acbad57cab997eace92bdef38cb831f001e340f18e7da20c68/lib/libHSscientific-0.3.7.0-0ef75f1ad39504acbad57cab997eace92bdef38cb831f001e340f18e7da20c68-ghc9.2.1.so (0x00007f9e056c1000)
	libHSprimitive-0.7.3.0-822d0e7468a94ea528f485ef32dda62bbc66567a948587839674a2abe7ccb63d-ghc9.2.1.so => /home/james/.cabal/store/ghc-9.2.1/primitive-0.7.3.0-822d0e7468a94ea528f485ef32dda62bbc66567a948587839674a2abe7ccb63d/lib/libHSprimitive-0.7.3.0-822d0e7468a94ea528f485ef32dda62bbc66567a948587839674a2abe7ccb63d-ghc9.2.1.so (0x00007f9e053d7000)
	libHStransformers-0.5.6.2-ghc9.2.1.so => /home/james/.ghcup/ghc/9.2.1-fPIC/lib64/ghc-9.2.1/transformers-0.5.6.2/libHStransformers-0.5.6.2-ghc9.2.1.so (0x00007f9e05098000)
	libHSinteger-logarithms-1.0.3.1-63f7c97ea42cdc7e0b4d9c809bfacd02a6fcd9e3fb976c9a733dc7fdd13494ed-ghc9.2.1.so => /home/james/.cabal/store/ghc-9.2.1/integer-logarithms-1.0.3.1-63f7c97ea42cdc7e0b4d9c809bfacd02a6fcd9e3fb976c9a733dc7fdd13494ed/lib/libHSinteger-logarithms-1.0.3.1-63f7c97ea42cdc7e0b4d9c809bfacd02a6fcd9e3fb976c9a733dc7fdd13494ed-ghc9.2.1.so (0x00007f9e04e82000)
	libHShashable-1.4.0.2-4cf6895fe59b6aa51ba08d8dd0aaa74d226fe36f771525bde9bcbb4df07cfcbd-ghc9.2.1.so => /home/james/.cabal/store/ghc-9.2.1/hashable-1.4.0.2-4cf6895fe59b6aa51ba08d8dd0aaa74d226fe36f771525bde9bcbb4df07cfcbd/lib/libHShashable-1.4.0.2-4cf6895fe59b6aa51ba08d8dd0aaa74d226fe36f771525bde9bcbb4df07cfcbd-ghc9.2.1.so (0x00007f9e04c30000)
	libHStext-1.2.5.0-ghc9.2.1.so => /home/james/.ghcup/ghc/9.2.1-fPIC/lib64/ghc-9.2.1/text-1.2.5.0/libHStext-1.2.5.0-ghc9.2.1.so (0x00007f9e04852000)
	libHStemplate-haskell-2.18.0.0-ghc9.2.1.so => /home/james/.ghcup/ghc/9.2.1-fPIC/lib64/ghc-9.2.1/template-haskell-2.18.0.0/libHStemplate-haskell-2.18.0.0-ghc9.2.1.so (0x00007f9e042ac000)
	libHSpretty-1.1.3.6-ghc9.2.1.so => /home/james/.ghcup/ghc/9.2.1-fPIC/lib64/ghc-9.2.1/pretty-1.1.3.6/libHSpretty-1.1.3.6-ghc9.2.1.so (0x00007f9e0403e000)
	libHSghc-boot-th-9.2.1-ghc9.2.1.so => /home/james/.ghcup/ghc/9.2.1-fPIC/lib64/ghc-9.2.1/ghc-boot-th-9.2.1/libHSghc-boot-th-9.2.1-ghc9.2.1.so (0x00007f9e03df8000)
	libHSbinary-0.8.9.0-ghc9.2.1.so => /home/james/.ghcup/ghc/9.2.1-fPIC/lib64/ghc-9.2.1/binary-0.8.9.0/libHSbinary-0.8.9.0-ghc9.2.1.so (0x00007f9e03b2e000)
	libHScontainers-0.6.5.1-ghc9.2.1.so => /home/james/.ghcup/ghc/9.2.1-fPIC/lib64/ghc-9.2.1/containers-0.6.5.1/libHScontainers-0.6.5.1-ghc9.2.1.so (0x00007f9e03628000)
	libHSbytestring-0.11.1.0-ghc9.2.1.so => /home/james/.ghcup/ghc/9.2.1-fPIC/lib64/ghc-9.2.1/bytestring-0.11.1.0/libHSbytestring-0.11.1.0-ghc9.2.1.so (0x00007f9e03340000)
	libHSdeepseq-1.4.6.0-ghc9.2.1.so => /home/james/.ghcup/ghc/9.2.1-fPIC/lib64/ghc-9.2.1/deepseq-1.4.6.0/libHSdeepseq-1.4.6.0-ghc9.2.1.so (0x00007f9e03125000)
	libHSarray-0.5.4.0-ghc9.2.1.so => /home/james/.ghcup/ghc/9.2.1-fPIC/lib64/ghc-9.2.1/array-0.5.4.0/libHSarray-0.5.4.0-ghc9.2.1.so (0x00007f9e02ea5000)
	libHSbase-4.16.0.0-ghc9.2.1.so => /home/james/.ghcup/ghc/9.2.1-fPIC/lib64/ghc-9.2.1/base-4.16.0.0/libHSbase-4.16.0.0-ghc9.2.1.so (0x00007f9e0225b000)
	libHSghc-bignum-1.2-ghc9.2.1.so => /home/james/.ghcup/ghc/9.2.1-fPIC/lib64/ghc-9.2.1/ghc-bignum-1.2/libHSghc-bignum-1.2-ghc9.2.1.so (0x00007f9e02008000)
	libHSghc-prim-0.8.0-ghc9.2.1.so => /home/james/.ghcup/ghc/9.2.1-fPIC/lib64/ghc-9.2.1/ghc-prim-0.8.0/libHSghc-prim-0.8.0-ghc9.2.1.so (0x00007f9e01913000)
	libgmp.so.10 => /usr/lib64/libgmp.so.10 (0x00007f9e0167d000)
	libc.so.6 => /lib64/libc.so.6 (0x00007f9e012a8000)
	libm.so.6 => /lib64/libm.so.6 (0x00007f9e00f67000)
	librt.so.1 => /lib64/librt.so.1 (0x00007f9e00d5f000)
	libdl.so.2 => /lib64/libdl.so.2 (0x00007f9e00b5b000)
	libnuma.so.1 => /usr/lib64/libnuma.so.1 (0x00007f9e0094f000)
	libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f9e0072f000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f9e06375000)

Oh my god! My eyes are burning! How am I meant to distribute this to customers!? Worse still, at work we build RPMs for customers, I would have to get them to add the haskell development repositiory just to run half the code!

What I really wanted was an executable with haskell dependencies (and RTS) statically linked and the lower level c libs dynamically linked. I have finally managed. In this tutorial, I will give you the details on how!

GHC linking options

GHC has a three main linking related flags. They are:

  • -dynamic
  • -static
  • -shared

-dynamic means “dynamically link to haskell things”, -static means “statically link to haskell things”, and -shared means “build me a shared object.” So from this, it would appear that by passing -shared to ghc without -dynamic should create us exactly what we are looking for? Well yes and no. If you are lucky (by that I mean running a packaged version of haskell on a -fPIC only linux distro or mac os) that will work. But there is no way to get cabal to actually do that. You could invoke ghc --make manually, but you would be a complete masochist. Just look at the number of loooong ghc commands cabal runs by building a project with cabal build -v.

But don’t lose hope. I lied when I said that there was no way to get cabal to pass -shared without -dynamic. You can do it in a foreign-library stanza by adding options: standalone. But I’ll quote the cabal manual to show you why this isn’t possible:

Options for building the foreign library, typically specific to the specified type of foreign library. Currently we only support standalone here. A standalone dynamic library is one that does not have any dependencies on other (Haskell) shared libraries; without the standalone option the generated library would have dependencies on the Haskell runtime library (libHSrts), the base library (libHSbase), etc. Currently, standalone must be used on Windows and must not be used on any other platform.

You can test this out. Regrettably the desired behaviour is only available on windows…

Why is it like this?

The reason why is because of position independent static libraries. Because static libraries were designed to be included in the executable, there was no need to make them position independent. But shared objects have to be so it is impossible to use the static haskell libraries to build a shared object. This is different on windows… I don’t know why… but it just is.

But surely then, by passing -fPIC to ghc (which we can do from cabal), we can make all of the static libraries position independent and then, like a rebel, enable the standalone option, even though we are not windows? Again! Yes and no. The issue here is that if your gcc wasn’t compiled with the fPICflag set by default, then the static rts will not be position independent. The same goes for the GHC base libraries.

Solution

So! First thing is first. There is a high chance that the GHC and GCC in your package manager has had the -fPIC flag baked in. I know that a lot of distributions have a policy that it should be enabled by default these days. If that is the case then you, my friend, are done! It isn’t very flexible, but you can just, in spite of the cabal documentation, enable the standalone option and it will work!

But what if you want a different version of GHC? Well then we better go about building our own. I like to use ghcup to manage multiple versions of GHC on a single system, and thankfully, it has the ability to easily build your own versions and have them managed by the ghcup command. This is what I am going to be doing in the rest of this article. So if you don’t have ghcup already, grab it now!

Building GHC with ghcup

Setting up the directories

First we are going to make a new folder (we can call it ghc) and inside it create another called patches. ghcup doesn’t actually need you to grab the ghc source in order to build a compiler (it fetches the source without your help) but we are going to need to create some patches for it to apply first. You can then share your patches with your friends and save them the effort of actually doing it themselves…

The next step is to clone the ghc git repositiory into your folder and cd in. We will then choose an appropriate branch and create a link to our patches folder called ‘patches’. To summarise:

mkdir -p ghc/patches
cd ghc
git clone https://gitlab.haskell.org/ghc/ghc.git
cd ghc
git checkout ghc-9.2.1-release
ln -s ../patches patches

Making the patch

Next, we need to actually patch ghc. We are going to use a tool called quilt for this. If you have ever made a package for a linux distribution, you should be very familiar with quilt

quilt new fpic-default.patch

This creates a new patch in our patches directory. We can then use quilt to edit files. Our edits will appear in the patch. If you want to make more changes to GHC, you will want to put them in other patches. Luckily, this is what quilt is for. If you ever want to create a new patch, you can use the quilt new command. You can then use quilt push and quilt pop to apply and un-apply patches in the order you created them! Very handy! quilt push -a and quilt pop -a will apply and remove all of your patches respectively.

Alas! I digress. We want to patch GHC to add -fPIC, by default, to all invocations of the compiler. Lets work out what file to patch:

~/ghc/ghc $ grep -R 'PIC' ./compiler
...
./compiler/GHC/Driver/Session.hs:default_PIC platform =
./compiler/GHC/Driver/Session.hs:    -- Darwin always requires PIC.  Especially on more recent macOS releases
./compiler/GHC/Driver/Session.hs:    (OSDarwin,  ArchX86_64)  -> [Opt_PIC]
./compiler/GHC/Driver/Session.hs:    -- For AArch64, we need to always have PIC enabled.  The relocation model
./compiler/GHC/Driver/Session.hs:    -- This requires PIC on AArch64, and ExternalDynamicRefs on Linux as on top
./compiler/GHC/Driver/Session.hs:    -- be built with -fPIC.
./compiler/GHC/Driver/Session.hs:    (OSDarwin,  ArchAArch64) -> [Opt_PIC]
./compiler/GHC/Driver/Session.hs:    (OSLinux,   ArchAArch64) -> [Opt_PIC, Opt_ExternalDynamicRefs]
./compiler/GHC/Driver/Session.hs:    (OSLinux,   ArchARM {})  -> [Opt_PIC, Opt_ExternalDynamicRefs]
./compiler/GHC/Driver/Session.hs:    (OSOpenBSD, ArchX86_64)  -> [Opt_PIC] -- Due to PIE support in
./compiler/GHC/Driver/Session.hs:                                         -- always generate PIC. See
...

The reason I used grep is because where this is defined is in a completely different file in ghc 8.10.7 and ghc 9.2.1. So use grep if in doubt! Anyway, looking at this it seems to be the case that -fPIC is implied on many platforms but not linux x86_64! So lets patch this.

quilt edit compiler/GHC/Driver/Session.hs

Will open up the correct file after copying it to a location. This copy is so that the diffs can be calculated. If you don’t want to use vim, you can simply run

quilt add compiler/GHC/Driver/Session.hs

and it will do the copy without opening vim. You can then edit it in whatever text editor you please.

Now that the file is open in an editor, we need to find this default_PIC function and add:

(OSLinux, ArchX86_64) -> [Opt_PIC]

to it. Very good! We now need to tell quilt to sync our changes to the patch:

quilt refresh

Building GHC

To built GHC, we need one more file, build.mk. You can find a sample in ghc/mk/build.mk.sample copy this somewhere safe (I would recommend the outer-most ghc directory) and call it build.mk. Open it up and take a look. There are lots of options. But essentially, you should uncomment/add:

GhcLibHcOpts += -fPIC
GhcRtsHcOpts += -fPIC
GhcRtsCcOpts += -fPIC
BuildFlavour = quick

Now I know that we just forced ghc to add -fPIC by default. Why am I enforcing it twice? The reason is paranoia. You are right, it might be fine, not baking in those flags. But what if your gcc toolchain isn’t -fPIC by default? How do I know that it will build the runtime library properly? I don’t! And I can’t be bothered to test otherwise! If you want to see if these flags are actually needed, try it and email me.

We are now ready to build ghc. The command to run from your outer ghc directory is:

ghcup compile ghc -j8 -v 9.2.1 -b 8.10.7 -p ${PWD}/patches -c ${PWD}/build.mk -o 9.2.1-fPIC -- --with-system-libffi

I shall go through each part:

  • ghcup - invoke the ghcup program
  • compile - we want to build something
  • ghc - we want to build ghc
  • -j8 - we want to use 8 cpus/threads. You can change that to the number of cpus/threads on your system
  • -v 9.2.1 - we want to build ghc version 9.2.1
  • -b 8.10.7 - we want to use ghc 8.10.7 as the bootstrap compiler. This has to be installed already. You can use ghcup tui to get it
  • -p ${PWD}/patches - an absolute path to a directory of patch files
  • -c ${PWD}/buikd.mk - an absolute path to a build.mk file
  • -o 9.2.1-fpic - so that you can which installed compiler is -fPIC by default, we will call the compiler this
  • -- - after this mark, arguments are passed to the configure script
  • --with-system-libffi - GHC contains a version of libffi. If we want to give our shared objects to another person we can’t link to that one, so instead we link to the system one

When built, you are done! Use ghcup tui to select your new compiler and then you can ignore the cabal docs! Add the standalone flag to your foreign library! No one can stop you now! You have been freed!

So lets test this by building the same shared library as before and running ldd:

$ ldd testlib.so.1.0.0
	linux-vdso.so.1 (0x00007fff545fe000)
	libffi.so.7 => /usr/lib64/libffi.so.7 (0x00007fe82842a000)
	libgmp.so.10 => /usr/lib64/libgmp.so.10 (0x00007fe828194000)
	libc.so.6 => /lib64/libc.so.6 (0x00007fe827dbf000)
	libm.so.6 => /lib64/libm.so.6 (0x00007fe827a7e000)
	/lib64/ld-linux-x86-64.so.2 (0x00007fe8293bb000)

Updates

Mac OS

It seems to work on mac os with the default ghcup compiler. I’m less familiar with .dylib files but I got a minimal example up and running. It seems as though you have to pass -lffi after -lyoulib to get it to work though. Easy enough! Next step will be to build a cross compiler to iOS (I don’t have an M1 mac) and see if it works there. I’m not sure what iOS lets you link to by default to it will be intresting to see!