Building Haskell for Omni OS

6-minute read
Programming
cabal • ghc • haskell

OpenSolaris is a fascinating operating system, developed by sun just before it was bought by Oracle. It didn’t last very long, the codebase was close sourced by Oracle soon after, but it’s short life has left an impact. Features that it was famous for were

  • Zones: Lightweight containers as a first class citizen (They can even run linux!)
  • SMF: Service management that doesn’t suck
  • Crossbow: Software defined networks for zones
  • DTrace: Needs no introduction

The Illumos Project has forked the OpenSolaris kernel and a few distributions have popped up using it, one of which is OmniOS, a server oriented flavor.

Download Haskell

For those who don’t want to DIY, you can download:

Building it yourself

Instead of blowing the dust of a really old solaris build of GHC and building newer and newer versions, the easiest way to bring haskell to a new platform is to cross compile. The way this works is that we build a haskell cross compiler (using our C cross compiler) and then cross compile haskell itself. This is a simple, but long journey. We then have to bootstrap cabal on our OmniOS system.

Sysroot

The easiest way to build a cross compiler is to use a sysroot. This is a subset of the root file system of your target OS that contains all of the libraries needed to link against. A nice speed up is that the illumos project provides a bare bones one for us! So lets grab that:

NOTE
In all the code examples, we expect $SYSROOT to be set to a path leading to the sysroot.

URL=https://github.com/illumos/sysroot/releases/download/20181213-de6af22ae73b-v1/illumos-sysroot-i386-20181213-de6af22ae73b-v1.tar.gz
rm -fr $SYSROOT
mkdir -p $SYSROOT
wget $URL -O $SYSROOT/sysroot.tar.gz
pushd $SYSROOT
tar xvf sysroot.tar.gz
ln -s lib lib64
popd
export PATH=$PATH:$SYSROOT/bin:$SYSROOT/usr/bin

Binutils

Binutils provides us with a linker and some other utilities that our compilers will use to build executables for OmniOS

BINUTILS_NAME=binutils-2.39
BINUTILS_SOURCE_ARCHIVE=$BINUTILS_NAME.tar.bz2
wget http://ftpmirror.gnu.org/binutils/$BINUTILS_SOURCE_ARCHIVE
tar xjfv $BINUTILS_SOURCE_ARCHIVE

BINUTILS_BUILD_DIR=binutils-build
mkdir $BINUTILS_BUILD_DIR

pushd $BINUTILS_BUILD_DIR
../$BINUTILS_NAME/configure      \
  --with-sysroot=$SYSROOT        \
  --target=x86_64-pc-solaris2.11 \
  --prefix=$SYSROOT              \
  --disable-nls                  \
  --disable-multilib             \
  --with-gnu-as                  \
  --with-gnu-ld                  \
  --disable-libssp
make -j4 all
make install
popd

GCC

We can now build gcc, which is part of the GNU Compiler Collection or… GCC

GCC_NAME=gcc-11.2.0
GCC_SOURCE_ARCHIVE=$GCC_NAME.tar.xz
wget https://ftp.gnu.org/gnu/gcc/gcc-11.2.0/$GCC_SOURCE_ARCHIVE

tar xfv $GCC_SOURCE_ARCHIVE
GCC_BUILD_DIR=gcc-build
mkdir $GCC_BUILD_DIR

pushd $GCC_BUILD_DIR
../$GCC_NAME/configure           \
  --target=x86_64-pc-solaris2.11 \
  --prefix=$SYSROOT              \
  --enable-languages=c           \
  --disable-bootstrap            \
  --disable-libssp               \
  --disable-libgomp              \
  --disable-libmudflap           \
  --disable-multilib             \
  --disable-decimal-float        \
  --disable-libffi               \
  --disable-libmudflap           \
  --disable-libquadmath          \
  --disable-libssp               \
  --disable-nls                  \
  --with-gnu-as                  \
  --with-gnu-ld                  \
  --with-sysroot=$SYSROOT
make -j4
make install
popd

At this point we should have in $SYSROOT/bin all of the compilers and utilities we need to build haskell. They will be named something like x86_64-pc-solaris2.11-gcc for example.

GHC

This is where it starts to get tricky as we have to patch the haskell runtime to support illumos. You can download my patches here, which are assumed to be applied with quilt.

git clone https://github.com/ghc/ghc

pushd ghc
git checkout ghc-8.10.7-release
git submodule init
git submodule update

./boot
./configure                      \
  --target=x86_64-pc-solaris2.11 \
  --prefix=$SYSROOT              \
  --disable-ld-override          \
  --with-intree-gmp

cp ../patches.tar.xz .
tar xf patches.tar.xz
quilt push -a

cp mk/build.mk.sample mk/build.mk
vim mk/build.mk

We want to uncomment the line #BuildFlavour = quick and add #HADDOCK_DOCS = NO underneath then save with :wq

make -j4
make install
make binary-dist

We now have a binary distribution of GHC!

Cabal

We want to be on our OmniOS machine for this part. We install GHC using our bindist from the last stage. We can install it by unpacking it, and in the unpacked folder, running:

./configure --prefix=/usr --disable-ld-override
pfexec make install

We now want to grab cabal and build it!

git clone https://github.com/haskell/cabal
cd cabal
git checkout Cabal-v3.6.2.0
./bootstrap/bootstrap.py -d bootstrap/linux-8.10.7.json -w /usr/bin/ghc

Uh oh! It failed! And sadly in a dependency that it downloaded!

Now run the bootstrap script again, but this time as soon as it says Configuring Network quickly press CTRL-z, to suspend the task in the terminal.

We then have to modify the network package before it starts building to fix it.

Configuring network-3.1.2.2...
^Z
[1]+  Stopped                 ./bootstrap/bootstrap.py -d bootstrap/linux-8.10.7.json -w /usr/bin/ghc
james@omnios:~/gits/cabal$ █

The first file we are going to change is one that marshells the posix c struct for a socket message to haskell

vim _build/unpacked/network-3.1.2.2/Network/Socket/Posix/MsgHdr.hsc

We are going to change the Storable instance in the file to:

instance Storable (MsgHdr sa) where
  sizeOf    _ = (#const sizeof(struct msghdr))
  alignment _ = alignment (0 :: CInt)

  peek p = do
    name       <- (#peek struct msghdr, msg_name)       p
    nameLen    <- (#peek struct msghdr, msg_namelen)    p
    iov        <- (#peek struct msghdr, msg_iov)        p
    iovLen     <- (#peek struct msghdr, msg_iovlen)     p
    return $ MsgHdr name nameLen iov iovLen nullPtr 0 0

  poke p mh = do
    -- We need to zero the msg_control, msg_controllen, and msg_flags
    -- fields, but they only exist on some platforms (e.g. not on
    -- Solaris).  Instead of using CPP, we zero the entire struct.
    zeroMemory p (#const sizeof(struct msghdr))
    (#poke struct msghdr, msg_name)       p (msgName       mh)
    (#poke struct msghdr, msg_namelen)    p (msgNameLen    mh)
    (#poke struct msghdr, msg_iov)        p (msgIov        mh)
    (#poke struct msghdr, msg_iovlen)     p (msgIovLen     mh)

Next, we are going to change a c file that seems to do something about message authentification… I don’t really know.

vim _build/unpacked/network-3.1.2.2/cbits/cmsg.c

Look down at lines 78-69. We want to change all of there returns to default values instead of function calls (null and 0).

struct cmsghdr *cmsg_firsthdr(struct msghdr *mhdr) {
  return NULL;
}

struct cmsghdr *cmsg_nxthdr(struct msghdr *mhdr, struct cmsghdr *cmsg) {
  return NULL;
}

unsigned char *cmsg_data(struct cmsghdr *cmsg) {
  return NULL;
}

int cmsg_space(int l) {
  return 0;
}

int cmsg_len(int l) {
  return 0;
}

I’m quite happy with the haskell code change, but I would be lying if I said I knew what I had done with the c change. If anyone has any idea, please get in contact so we can get feature parity and support for illumos distributions in the network package!

next we run the fg command, and our code should all built just fine! We have built cabal!

james@omnios:~/gits/cabal$ fg
./bootstrap/bootstrap.py -d bootstrap/linux-8.10.7.json -w /usr/local/bin/ghc
configure: WARNING: unrecognized options: --with-compiler, --with-hc-pkg
checking build system type... x86_64-pc-solaris2.11
checking host system type... x86_64-pc-solaris2.11
checking for gcc... /usr/bin/gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
...

Work to do

  • Fix the Network package. Already fixed in newer version
  • Get my RTS patch looked at by someone who understands the illumos threading library or haskell RTS.
  • Submit the RTS patch upstream
  • Add GHC and cabal to an IPS repository

Another thing to try is to look at passing -D_XOPEN_SOURCE=1 -D_XOPEN_SOURCE_EXTENDED=1 to gcc during the build of ghc and patching GHC to set it as a default compiler flag. This might get rid of our networking issues. If you want to have a try, check out this blog post where we did exactly this to build GHC with fPic by default on linux.