Investigate HIP performance difference with spatial indices

[bmhan12]

PR #1278 records an unexpected performance difference between HIP and CUDA (~4x slower) for the BVH and Implicit Grid spatial indices.

Some potential sources of improvement:

• Verify Axom is using HIP intrinsics:

  axom/src/axom/core/utilities/BitUtilities.hpp

  Lines 21 to 36 in 0a47cb2

  	// CUDA intrinsics: https://docs.nvidia.com/cuda/cuda-math-api/group__CUDA__MATH__INTRINSIC__INT.html
  	// TODO: Support HIP intrinsics (https://rocm.docs.amd.com/projects/HIP/en/latest/reference/kernel_language.html)
  	// Check for and setup defines for platform-specific intrinsics
  	// Note: `__GNUC__` is defined for the gnu, clang and intel compilers
  	#if defined(AXOM_USE_CUDA)
  	// Intrinsics included implicitly
  	#elif defined(_WIN64) && (_MSC_VER >= 1600)
  	#define _AXOM_CORE_USE_INTRINSICS_MSVC
  	#include <intrin.h>
  	#elif defined(__x86_64__) && defined(__GNUC__)
  	#define _AXOM_CORE_USE_INTRINSICS_GCC
  	#include <x86intrin.h>
  	#elif defined(__powerpc64__) && (defined(__GNUC__) || defined(__ibmxl__))
  	#define _AXOM_CORE_USE_INTRINSICS_PPC
  	#endif

• Try newer rocm versions (>= 5.7.0)
• Consider using RAJA::unsafeAtomicAdd (link and link to suggestions)

[kennyweiss]

Presumably, given the updated query results in #1419, this is still an issue for our hip-based vs. cuda-based ImplicitGrid