Building the IBKR C++ API Client Library, Protobuf Edition

A while back I wrote about building the IBKR C++ API client library on MacOS and Linux/aarch64. Since then, Interactive Brokers has added a Google Protocol Buffers dependency to the C++ API, which means the build steps in the original post no longer work as-is. While I was at it I also wanted to clean up a couple of things that had been bothering me — specifically the Linux assumptions baked into the shipped makefile and the need to set DYLD_LIBRARY_PATH at runtime. So this is a follow-up covering what changed.

I tested this against TWS API version 1046.01 on Apple Silicon, with Homebrew protobuf 34 and abseil 20260107 (the abseil dep is pulled in transitively by protobuf). The Intel Decimal Floating-Point Math Library build hasn’t changed; if you don’t already have a working libbid.dylib, the original post walks through that. The only new packages we need are protobuf and pkg-config from Homebrew, and we’ll see in a moment why pkg-config is so useful here:

~ $ brew install protobuf pkg-config

Putting libbid in Place

The libbid.dylib produced by the Intel build needs to land in IBJts/source/cppclient/client/lib, just like before. The new wrinkle is that the dylib’s install_name is a bare libbid.dylib, which means the dynamic linker only finds it if it’s already on a known search path. Rather than carry DYLD_LIBRARY_PATH everywhere, I retag the dylib with an @rpath install_name, and we’ll let the consumers add the right rpath when they link against it:

~/Downloads/twsapi_macunix/IBJts/source/cppclient/client $ mkdir lib
~/Downloads/twsapi_macunix/IBJts/source/cppclient/client $ cp ~/Downloads/IntelRDFPMathLib20U2/LIBRARY/libbid.dylib lib/
~/Downloads/twsapi_macunix/IBJts/source/cppclient/client $ install_name_tool -id @rpath/libbid.dylib lib/libbid.dylib
~/Downloads/twsapi_macunix/IBJts/source/cppclient/client $ otool -D lib/libbid.dylib
lib/libbid.dylib:
@rpath/libbid.dylib

Building the IBKR C++ API Client Library

The shipped makefile in IBJts/source/cppclient/client now compiles the protobuf-generated .cc files in protobufUnix/ alongside the existing .cpp sources, but it’s still pretty Linux-flavored and a few things need to change for MacOS:

1. It targets .so and uses g++, but on MacOS we want .dylib and Apple clang++. Protobuf 22+ also no longer compiles under -std=c++11, so we need at least -std=c++17.
2. The shipped recipe links libbid with -l:libbid.so, which is a Linux-only ld extension. The MacOS linker doesn’t accept the -l:filename form — we just use plain -lbid and let it pick up libbid.dylib from -L lib.
3. Protobuf 34 pulls in dozens of Abseil symbols at link time. Listing all of those -labsl_* flags by hand would be miserable and brittle, so I let pkg-config --cflags --libs protobuf resolve them. On my machine that expands to roughly 80 link flags; not something you want to maintain by hand.
4. I baked -install_name @rpath/libTwsSocketClient.dylib and -Wl,-rpath,@loader_path into the dylib so it finds libbid.dylib sitting next to it without DYLD_LIBRARY_PATH.

Here’s the resulting makefile:

CXX=clang++
CXXFLAGS=-pthread -Wall -Wno-switch -Wno-unused-function -std=c++17 -fPIC
ROOT_DIR=.
BASE_SRC_DIR=${ROOT_DIR}
PROTOBUF_DIR=${BASE_SRC_DIR}/protobufUnix
PROTOBUF_CFLAGS=$(shell pkg-config --cflags protobuf)
PROTOBUF_LIBS=$(shell pkg-config --libs protobuf)
INCLUDES=-I${ROOT_DIR} -I${PROTOBUF_DIR} $(PROTOBUF_CFLAGS)
LIB_DIR=lib
TARGET=libTwsSocketClient.dylib

LDFLAGS=-dynamiclib -install_name @rpath/$(TARGET) \
        -L$(LIB_DIR) -lbid \
        $(PROTOBUF_LIBS) \
        -Wl,-rpath,@loader_path

$(TARGET):
	$(CXX) $(CXXFLAGS) $(INCLUDES) $(BASE_SRC_DIR)/*.cpp ${PROTOBUF_DIR}/*.cc $(LDFLAGS) -o $(TARGET)

clean:
	rm -f $(TARGET) *.o

With that in place, make builds the dylib without complaint:

~/Downloads/twsapi_macunix/IBJts/source/cppclient/client $ make
~/Downloads/twsapi_macunix/IBJts/source/cppclient/client $ ls -l libTwsSocketClient.dylib
-rwxr-xr-x  1 dlewis  staff  5086472 May  9 13:48 libTwsSocketClient.dylib
~/Downloads/twsapi_macunix/IBJts/source/cppclient/client $ otool -L libTwsSocketClient.dylib | head
libTwsSocketClient.dylib:
	@rpath/libTwsSocketClient.dylib (compatibility version 0.0.0, current version 0.0.0)
	@rpath/libbid.dylib (compatibility version 0.0.0, current version 0.0.0)
	/opt/homebrew/opt/protobuf/lib/libprotobuf.34.1.0.dylib (compatibility version 0.0.0, current version 34.1.0)
	/opt/homebrew/opt/abseil/lib/libabsl_log_internal_check_op.2601.0.0.dylib (compatibility version 2601.0.0, current version 0.0.0)
	/opt/homebrew/opt/abseil/lib/libabsl_die_if_null.2601.0.0.dylib (compatibility version 2601.0.0, current version 0.0.0)

The @rpath/libTwsSocketClient.dylib and @rpath/libbid.dylib references are exactly what we want — at link time the consumer adds an rpath, and at load time the dynamic linker will find them sitting next to whatever links them.

As before, copy the resulting library next to libbid.dylib:

~/Downloads/twsapi_macunix/IBJts/source/cppclient/client $ cp libTwsSocketClient.dylib lib/
~/Downloads/twsapi_macunix/IBJts/source/cppclient/client $ ls -l lib/
total 17456
-rwxr-xr-x  1 dlewis  staff  5086472 May  9 13:48 libTwsSocketClient.dylib
-rwxr-xr-x  1 dlewis  staff  5019408 May  9 13:40 libbid.dylib

Building the C++ Sample Application

The sample makefile in IBJts/samples/Cpp/TestCppClient needs the same MacOS treatment, plus one more change that took me a while to figure out. Here’s the version I ended up with:

CXX=clang++
CXXFLAGS=-pthread -Wall -Wno-switch -Wno-unused-function -std=c++17
ROOT_DIR=../../../source/cppclient
BASE_SRC_DIR=${ROOT_DIR}/client
PROTOBUF_DIR=$(BASE_SRC_DIR)/protobufUnix
PROTOBUF_CFLAGS=$(shell pkg-config --cflags protobuf)
PROTOBUF_LIBS=$(shell pkg-config --libs protobuf)
INCLUDES=-I${BASE_SRC_DIR} -I${ROOT_DIR} -I${PROTOBUF_DIR} $(PROTOBUF_CFLAGS)
SOURCE_DIR=${BASE_SRC_DIR}
SOURCE_LIB=libTwsSocketClient.dylib
LIB_DIR=$(SOURCE_DIR)/lib
LIB_NAME_DYLIB=libbid.dylib
TARGET=TestCppClient

$(TARGET)Static:
	$(CXX) $(CXXFLAGS) $(INCLUDES) $(BASE_SRC_DIR)/*.cpp ${PROTOBUF_DIR}/*.cc ./*.cpp \
	    -L$(LIB_DIR) -lbid $(PROTOBUF_LIBS) \
	    -Wl,-rpath,@executable_path/$(LIB_DIR) \
	    -o $(TARGET)Static

$(TARGET)Dynamic:
	$(CXX) $(CXXFLAGS) $(INCLUDES) ./*.cpp \
	    -L$(LIB_DIR) -lbid -lTwsSocketClient $(PROTOBUF_LIBS) \
	    -Wl,-rpath,@executable_path/$(LIB_DIR) \
	    -o $(TARGET)Dynamic

clean:
	rm -f $(TARGET) $(TARGET)Static $(TARGET)Dynamic *.o

The @executable_path/$(LIB_DIR) rpath lets the binary find both dylibs at runtime relative to where it sits in the source tree, so we no longer need DYLD_LIBRARY_PATH at all.

The change worth flagging is the Dynamic recipe: notice that, unlike the shipped version, it does not compile ${PROTOBUF_DIR}/*.cc. That’s because those translation units are already inside libTwsSocketClient.dylib — and if you also link them directly into the binary, both copies register their protobuf descriptors during static initialization, and protobuf aborts the process before main even runs:

~/Downloads/twsapi_macunix/IBJts/samples/Cpp/TestCppClient $ ./TestCppClientDynamic
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
E0000 00:00:1778352731.947974 9124957 descriptor_database.cc:683] File already exists in database: AccountDataEnd.proto
F0000 00:00:1778352731.948062 9124957 descriptor.cc:2526] Check failed: GeneratedDatabase()->Add(encoded_file_descriptor, size)
*** Check failure stack trace: ***
    @        0x1025821a0  absl::lts_20260107::log_internal::LogMessage::SendToLog()
    @        0x102582140  absl::lts_20260107::log_internal::LogMessage::Flush()
    @        0x1030ff2ec  google::protobuf::DescriptorPool::InternalAddGeneratedFile()
    @        0x103179448  google::protobuf::internal::AddDescriptors()

Dropping ${PROTOBUF_DIR}/*.cc from the Dynamic recipe (but leaving it in Static, since that recipe doesn’t link the dylib) fixes it. With that out of the way, the build is uneventful:

~/Downloads/twsapi_macunix/IBJts/samples/Cpp/TestCppClient $ make TestCppClientDynamic
~/Downloads/twsapi_macunix/IBJts/samples/Cpp/TestCppClient $ otool -l TestCppClientDynamic | grep -A2 LC_RPATH
          cmd LC_RPATH
      cmdsize 72
         path @executable_path/../../../source/cppclient/client/lib (offset 12)

Pointing it at a paper IB Gateway on 127.0.0.1:4002 (or paper TWS on 7497):

~/Downloads/twsapi_macunix/IBJts/samples/Cpp/TestCppClient $ ./TestCppClientDynamic 127.0.0.1 4002
Start of C++ Socket Client Test 0
Attempt 1 of 50
Connecting to 127.0.0.1:4002 clientId:0
Connected to 127.0.0.1:4002 clientId:0 serverVersion: 223
Account List: DUH782952
Next Valid Id: 64
Error. Id: -1, Time: Sat May  9 13:55:18 2026, Code: 2104, Msg: Market data farm connection is OK:usfarm
Error. Id: -1, Time: Sat May  9 13:55:18 2026, Code: 2107, Msg: HMDS data farm connection is inactive but should be available upon demand.ushmds
Error. Id: -1, Time: Sat May  9 13:55:18 2026, Code: 2158, Msg: Sec-def data farm connection is OK:secdefnj
Error. Id: -1, Time: Sat May  9 13:55:19 2026, Code: 2106, Msg: HMDS data farm connection is OK:ushmds

Success! The protobuf double-registration trap was the one that cost me the most time, so hopefully writing it down here saves someone else the same afternoon. Otherwise the changes since the original post are mostly mechanical: bump to C++17, swap in pkg-config for the abseil link soup, and use @rpath / @loader_path / @executable_path so nothing depends on DYLD_LIBRARY_PATH at runtime.