Docker

The easiest way to run Crail is to use Docker images. We provide two preconfigured Docker images:

  1. TCP/DRAM: apache/crail

  2. RDMA/DRAM: apache/crail_rdma

If you want to run other or more complex configurations you can use either image as a basis and provide your own configuration files. Refer to Own configurations for details.

  1. and 2. share the following configuration parameters:

Property

Default Value

Description

NAMENODE_HOST

localhost

Namenode hostname/ip to bind to

NAMENODE_PORT

9060

Namenode port to listen on

INTERFACE

eth0

Datanode network interface to listen on

DATAPATH

/dev/hugepages/data

Datanode hugepage path to data

STORAGELIMIT

1073741824

Datanode Size (Bytes) of DRAM to provide

CACHEPATH

/dev/hugepages/cache

Client/Datanode hugepage path to buffer cache

CACHELIMIT

0

Size (Bytes) of hugepage buffer cache

These properties can be specified as environment variables when starting a Docker image with -e <property>=<value>.

TCP image

To run Crail you first need to start the namenode. For example

docker run -it --network host -e NAMENODE_HOST=host02 -e INTERFACE=eth5 apache/crail namenode

starts a TCP namenode using Docker’s host network configuration on host02 on interface eth5. The TCP tier allows for other network configurations. Refer to https://docs.docker.com/network/ for details.

Once the namenode has started start a Crail TCP storage tier (preferably on a different node). For example

docker run -it --network host -e NAMENODE_HOST=host02 -e INTERFACE=eth5 apache/crail datanode

starts a TCP datanode with 1GB of storage (default) listening on eth5. It is recommended to put the data directory on a hugetlb mount. You can do this by passing an mounted hugetlb directory on the host as a volume to Docker. For example

docker run -it --network host -e NAMENODE_HOST=host02 -e INTERFACE=eth5 -v /dev/hugepages:/dev/hugepages apache/crail datanode

passes the hugetlb mount /dev/hugepages to the container.

RDMA image

To run Crail/RDMA/DRAM you can start the namenode as follows:

docker run -it --network host -e NAMENODE_HOST=host02 -e INTERFACE=eth5 --cap-add=IPC_LOCK --device=/dev/infiniband/uverbs0 --device=/dev/infiniband/rdma_cm -v /dev/hugepages:/dev/hugepages apache/crail_rdma namenode

This starts a namenode on host02 using the host’s network on eth5. Note that the uverbs device needs to match the interface.

To run a RDMA storage tier:

docker run -it --network host -e NAMENODE_HOST=host02 -e INTERFACE=eth5 --cap-add=IPC_LOCK --device=/dev/infiniband/uverbs0 --device=/dev/infiniband/rdma_cm -v /dev/hugepages:/dev/hugepages apache/crail_rdma datanode

Note: The RDMA docker image provides default RDMA provider libraries from Ubuntu 18.04. They might not be compatible with your host’s RDMA stack. To install your own RDMA libraries use -v or create your own Docker image from crail_rdma.

Own configurations

If you want to run more complex configurations that are not covered by the options above you have two options:

  1. Create your own Docker image by creating a Dockerfile and using the crail images as a source. You can then change the configuration by e.g. copying your own config into the image

  2. Pass your config as a volume with -v <local_path>:<docker_path>