support OpenMP with system macOS toolchain

[kevinushey]

With R 4.0.0, R (as well as packages on CRAN) will be built with the system toolchain (Apple Clang), which unfortunately mean OpenMP support is not provided natively by the compiler. Fortunately, it is still possible to use OpenMP with the system toolchain if you:

1. Instruct clang to enable OpenMP with -Xpreprocessor -fopenmp,
2. Explicitly provide include paths to libomp during compilation,
3. Explicitly provide library paths to libomp during link.

Unfortunately, data.table fails to detect this case as it invokes the compiler with just -fopenmp here:

data.table/configure

Line 64 in b1b1832

printf "#include <omp.h>\nint main () { return omp_get_num_threads(); }" | ${CC} ${CFLAGS} -fopenmp -xc - >/dev/null 2>&1 || R_NO_OPENMP=1;

It would be helpful for users on macOS if support for OpenMP on macOS could be detected + used.

Sorry, I recognize this is kind of a pain but figured it was worth filing since OpenMP is somewhat vital to data.table's performance.

See also:

http://r-sig-mac.29524.n8.nabble.com/R-SIG-Mac-Apple-Clang-does-support-OpenMP-if-libomp-is-available-td458.html#a461

(+ some other recent discussion on R-SIG-Mac)

[MichaelChirico]

I think Jim Hester raised this same issue, let me see...

I see, it was on the orgs page... good to have this in the issue tracker

https://github.com/orgs/Rdatatable/teams/project-members/discussions/34

[krivard]

I am pasting my working version of Makevars here with edits from the version in the install docs marked, so that google may find it and help a future traveller:

# Added: -Xpreprocessor
CC=/usr/bin/clang -Xpreprocessor -fopenmp

# Added: -Xpreprocessor -lomp -I/usr/local/include
CXX=/usr/bin/clang++ -Xpreprocessor -fopenmp -lomp -I/usr/local/include
# -O3 should be faster than -O2 (default) level optimisation ..
CFLAGS=-g -O3 -Wall -pedantic -std=gnu99 -mtune=native -pipe
CXXFLAGS=-g -O3 -Wall -pedantic -std=c++11 -mtune=native -pipe

# Added: -lomp
LDFLAGS=-L/usr/local/opt/gettext/lib -L/usr/lib -Wl,-rpath,/usr/lib -lomp
CPPFLAGS=-I/usr/local/opt/gettext/include -I/usr/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include

macOS Mojave, 10.14.6

with thanks to these two threads on StackOverflow

[dbrookeUAB]

Since @krivard last post was super helpful for me to get things working, here is my working version of Makevars

# 2020-11-15 Added:  -Xpreprocessor -fopenmp 
CC=/usr/local/Cellar/llvm/11.0.0/bin/clang -Xpreprocessor -fopenmp
# 2020-11-15 Added:   -Xpreprocessor -fopenmp -lomp -I/usr/local/include
CXX=/usr/local/Cellar/llvm/11.0.0/bin/clang++  -Xpreprocessor -fopenmp -lomp -I/usr/local/include
CXX11=/usr/local/Cellar/llvm/11.0.0/bin/clang++
CXX14=/usr/local/Cellar/llvm/11.0.0/bin/clang++
CXX17=/usr/local/Cellar/llvm/11.0.0/bin/clang++
CXX1X=/usr/local/Cellar/llvm/11.0.0/bin/clang++
# 2020-11-15 Added:  CFLAGS
CFLAGS=-g -O3 -Wall -pedantic -std=gnu99 -mtune=native -pipe
# 2020-11-15 Added:  CXXFLAGS
CXXFLAGS=-g -O3 -Wall -pedantic -std=c++11 -mtune=native -pipe
LDFLAGS=-L/usr/local/Cellar/llvm/11.0.0/lib
SHLIB_OPENMP_CFLAGS= -fopenmp
SHLIB_OPENMP_CXXFLAGS= -fopenmp

Also Mac Mojave 10.14.6

Please reply if you find anything about this that could be improved or if you find something redundant. I just change lines on a whim because I'm too lazy to spend the time really learning the ins and outs of the Makevars file.

UPDATE
I was able to get macOS clang to compile with OpenMP for my MacBook Pro (18,2 M1 Max) running macOS Ventura 13.3 with only this for Makevars.

CC=clang
CFLAGS += -Xclang -fopenmp
LDFLAGS += -lomp

I did have to download the libomp. Specifically, LLVM 14.0.6.

Voilà!

> install.packages('data.table', type = 'source')
> library(data.table)
data.table 1.14.8 using 5 threads (see ?getDTthreads).  Latest news: r-datatable.com

PS...
CC=clang -Wno-nullability-completeness was nice.

[jangorecki]

@kevinushey could you please check if 1.13.2 resolves the problem for you? There was a change in #4735

4. User-supplied PKG_LIBS and PKG_CFLAGS are now retained and the suggestion in https://mac.r-project.org/openmp/; i.e.,
   PKG_CPPFLAGS='-Xclang -fopenmp' PKG_LIBS=-lomp R CMD INSTALL data.table_<ver>.tar.gz
   has a better chance of working on Mac.

[kevinushey]

Looks like this works for me as well. It sounds like the issue is ultimately resolved, then? Unless data.table wants to try and automagically enable OpenMP on macOS.