Cloud / Programming / technology / 🌱 Seedling

Running LanceDB with Node.js Express API in Docker Containers

2 minutes
🌱 Seedling
Running LanceDB with Node.js Express API in Docker Containers

Running LanceDB alongside a Node.js Express API in a Dockerized environment should be straightforward, but there’s a hiccup—particularly when you are aiming for a production setting and using one of the Nodejs Alpine images. One of the challenges we face in such a production environment is a compatibility issue between the Node.js Alpine image and the npm package vectordb. While each of these components is robust on its own, they don’t always play well together when you’re trying to run them in a container.

Node.js Alpine and vectordb

The main challenge comes from using the Nodejs Alpine image. Although the Alpine image is minimal and fast, it sometimes lacks the capability to execute LanceDB’s vectordb compiled binaries. This is a significant drawback when integrating LanceDB into your Node.js application while using the Alpine image.

The error we encountered was similar to the following:

Error: vectordb: failed to load native library.
You may need to run `npm install @lancedb/vectordb-linux-x64-musl`.

Solution

To resolve this, we switched from using nodejs-alpine to node:18.17.0-bookworm-slim, as it contains the libraries needed for vectordb to operate smoothly.

Here’s a sample Dockerfile configuration that I often use to get everything up and running.

# This image has the minimum dependencies to run our
# in-memory vector database, LanceDB.
FROM node:18.17.0-bookworm-slim

# Install system dependencies
RUN apt-get update && apt-get install -y \
  python3 \
  python3-pip \
  make \
  g++ \
  && rm -rf /var/lib/apt/lists/*


# Set environment variables
ENV NODE_ENV production


# Set the working directory
WORKDIR /app

# Copy dependency files and install dependencies
COPY package.json yarn.lock ./
RUN yarn install

# Copy the rest of the code
COPY . .

# Build the application
RUN yarn build

# Expose the port the app will run on
EXPOSE 8080

# Command to run the application
CMD ["yarn", "start"]

While nodejs-alpine is an excellent choice for running lightweight Node.js applications, its limitations become apparent when you require specific native libraries. We found node:18.17.0-bookworm-slim to be a better fit for running vectordb along with our Node.js Express API.

Continue Reading