Make file not working on Mac OSX

[rebrancher]

Using Lua 5.1 (include path is -I/opt/homebrew/include/lua5.1, library path is -llua5.1)
c++ -O3 -Wall -Wno-unknown-pragmas -Wno-sign-compare -std=c++14 -pthread -fPIE -DTM_VERSION=v2.4.0  -o src/external/streamvbyte_decode.o -c src/external/streamvbyte_decode.cc -I/opt/homebrew/include -isystem ./include -I./src -I/opt/homebrew/include/lua5.1
In file included from src/external/streamvbyte_decode.cc:11:
src/external/streamvbyte_arm_decode.c:12:31: error: expected ')'
                                        const uint8_t * restrict *dataPtrPtr) {
                                                                 ^
src/external/streamvbyte_arm_decode.c:11:37: note: to match this '('
static inline decode_t  _decode_neon(const uint8_t key,
                                    ^
src/external/streamvbyte_arm_decode.c:18:37: error: use of undeclared identifier 'dataPtrPtr'
  uint8x16_t compressed = vld1q_u8(*dataPtrPtr);
                                    ^
src/external/streamvbyte_arm_decode.c:34:4: error: use of undeclared identifier 'dataPtrPtr'
  *dataPtrPtr += len;
   ^
src/external/streamvbyte_arm_decode.c:38:63: error: expected ')'
static void streamvbyte_decode_quad( const uint8_t * restrict *dataPtrPtr, uint8_t key, uint32_t * restrict out ) {
                                                              ^
src/external/streamvbyte_arm_decode.c:38:36: note: to match this '('
static void streamvbyte_decode_quad( const uint8_t * restrict *dataPtrPtr, uint8_t key, uint32_t * restrict out ) {
                                   ^
src/external/streamvbyte_arm_decode.c:39:32: error: use of undeclared identifier 'key'
  decode_t data =_decode_neon( key, dataPtrPtr );
                               ^
src/external/streamvbyte_arm_decode.c:39:37: error: use of undeclared identifier 'dataPtrPtr'
  decode_t data =_decode_neon( key, dataPtrPtr );
                                    ^
src/external/streamvbyte_arm_decode.c:41:24: error: use of undeclared identifier 'out'
  vst1q_u8((uint8_t *) out, data);
                       ^
src/external/streamvbyte_arm_decode.c:50:5: error: no matching function for call to 'streamvbyte_decode_quad'
    streamvbyte_decode_quad( &dataPtr, keyPtr[i], out + 4*i );
    ^~~~~~~~~~~~~~~~~~~~~~~
src/external/streamvbyte_arm_decode.c:38:13: note: candidate function not viable: requires single argument 'restrict', but 3 arguments were provided
static void streamvbyte_decode_quad( const uint8_t * restrict *dataPtrPtr, uint8_t key, uint32_t * restrict out ) {

This is the error I get after installing dependencies on Mac OSX. Does anyone know what could be wrong?

[systemed]

Yes, there's some ARM Mac build issues at the moment. The bottom line at #536 suggests a solution:

For now I added #define restrict __restrict to work around it.

(I'm assuming this is at the top of src/external/streamvbyte_arm_decode.c)

Intel Macs are fine. I'm hoping to have a chance to test it out on an ARM Mac in the next few days but it won't be immediate I'm afraid.

[rebrancher]

Thanks for the info - no problem. Have a happy new year

[FunMiles]

Yes, there's some ARM Mac build issues at the moment. The bottom line at #536 suggests a solution:

For now I added #define restrict __restrict to work around it.

(I'm assuming this is at the top of src/external/streamvbyte_arm_decode.c)

Intel Macs are fine. I'm hoping to have a chance to test it out on an ARM Mac in the next few days but it won't be immediate I'm afraid.

At the root, it is because a 'c' file is included in a 'cc' file, causing the offending lines of code to be compiled as C++ rather than C. I wonder if the restrict has a meaningful impact on the code speed. Nothing is written to memory in the routine except for the last line which increments the pointer in memory. That writing cannot affect the other reading since len = lengthTable[key]; is needed before the increment is done. Unless I am missing something, it seems that the restrict keyword should have absolutely no impact since just by reading the code, one (the compiler included) can see that no other pointer is modifying what is being pointed to.

I'll be happy to be corrected if I missed something.

[FunMiles]

@cldellow success:

build % time cmake --build . -j 32    
[24/35] Building CXX object CMakeFiles/tilemaker.dir/src/osm_store.cpp.o
/Users/michel/Dev/OpenSource/GIS/Test/tilemaker/src/osm_store.cpp:52:19: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]
                mp.emplace_back(move(poly));
                                ^
                                std::
/Users/michel/Dev/OpenSource/GIS/Test/tilemaker/src/osm_store.cpp:72:20: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]
                mls.emplace_back(move(ls));
                                 ^
                                 std::
2 warnings generated.
[31/35] Building CXX object CMakeFiles/tilemaker.dir/src/tile_worker.cpp.o
/Users/michel/Dev/OpenSource/GIS/Test/tilemaker/src/tile_worker.cpp:374:9: warning: unqualified call to 'std::move' [-Wunqualified-std-cast-call]
                                g = move(reordered);
                                    ^
                                    std::
1 warning generated.
[35/35] Linking CXX executable tilemaker
cmake --build . -j 32  85.33s user 3.64s system 734% cpu 12.120 total

[FunMiles]

@cldellow My use of godbolt is monkey-see/monkey-do 😄 Do you have a link to your attempt? I can have a look.

[cldellow]

Great, thanks for your help!

The godbolt links are these:

I tried dumping them into godbolt -- with restrict, and without restrict.

I didn't copy the values of the lookup tables, just declared them. I'm also passing uninitialized pointers into the function. I imagine either of these might invalidate the analysis, but don't know for certain.

[cldellow]

Ha, and in fact, looking at those again now with fresher eyes, I see that my excerpt was returning the input (bytes), not the output (ints). If I change that, the assembly looks to be the same.

[FunMiles]

@cldellow glad you found that out. Also, one thing to be wary of with godbolt is that one tries to do a minimal example. In such case, the compiler can often see through the whole process and replace by something that is perfect for the usage that is in main() but is not indicative of what would happen in more general cases. It's a similar problem to the caveats of micro-benchmarks. Sometimes not writing a main() but having a function that receives pointers for which the compiler does not see the data can give more meaningful observations.
Finally, as you noted, your data table is not realistic and in this case, the compiler has seen that it is all zeroes. Notice how the length table is not even used. if you replace static with external and remove the initialization, then you get added lines that do load the length table:

        adrp    x4, lengthTable+1