From nobody Mon Nov 08 18:40:02 2021 X-Original-To: dev-commits-ports-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 3CB3B1846526; Mon, 8 Nov 2021 18:40:03 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Hp0KH1H81z3h8q; Mon, 8 Nov 2021 18:40:03 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 081F21687A; Mon, 8 Nov 2021 18:40:03 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 1A8Ie20G024578; Mon, 8 Nov 2021 18:40:02 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 1A8Ie2lI024575; Mon, 8 Nov 2021 18:40:02 GMT (envelope-from git) Date: Mon, 8 Nov 2021 18:40:02 GMT Message-Id: <202111081840.1A8Ie2lI024575@gitrepo.freebsd.org> To: ports-committers@FreeBSD.org, dev-commits-ports-all@FreeBSD.org, dev-commits-ports-main@FreeBSD.org From: Mikael Urankar Subject: git: 752601257a53 - main - databases/clickhouse: Port to aarch64. List-Id: Commits to the main branch of the FreeBSD ports repository List-Archive: https://lists.freebsd.org/archives/dev-commits-ports-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-ports-main@freebsd.org X-BeenThere: dev-commits-ports-main@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: mikael X-Git-Repository: ports X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 752601257a53022940feb142be2943d839d016d9 Auto-Submitted: auto-generated X-ThisMailContainsUnwantedMimeParts: N The branch main has been updated by mikael: URL: https://cgit.FreeBSD.org/ports/commit/?id=752601257a53022940feb142be2943d839d016d9 commit 752601257a53022940feb142be2943d839d016d9 Author: Mikael Urankar AuthorDate: 2021-11-08 18:38:59 +0000 Commit: Mikael Urankar CommitDate: 2021-11-08 18:38:59 +0000 databases/clickhouse: Port to aarch64. PR: 259047 Approved by: Oleg Ginzburg (maintainer) --- databases/clickhouse/Makefile | 5 +- .../clickhouse/files/patch-cmake_find_ldap.cmake | 10 +++ ...ch-contrib_boringssl_crypto_cpu-aarch64-linux.c | 75 ++++++++++++++++++++++ ...jemalloc_internal_jemalloc__internal__defs.h.in | 11 ++++ .../patch-contrib_rocksdb-cmake_CMakeLists.txt | 14 ++++ .../files/patch-contrib_rocksdb_CMakeLists.txt | 13 ++++ .../files/patch-contrib_rocksdb_util_crc32c.cc | 38 +++++++++++ .../patch-contrib_rocksdb_util_crc32c__arm64.cc | 45 +++++++++++++ .../files/patch-src_Common_StackTrace.cpp | 11 ++++ 9 files changed, 220 insertions(+), 2 deletions(-) diff --git a/databases/clickhouse/Makefile b/databases/clickhouse/Makefile index e443c0544ed0..24a07784d476 100644 --- a/databases/clickhouse/Makefile +++ b/databases/clickhouse/Makefile @@ -11,8 +11,8 @@ COMMENT= Fast open-source OLAP database management system LICENSE= APACHE20 -ONLY_FOR_ARCHS= amd64 -ONLY_FOR_ARCHS_REASON= only supported on amd64 +ONLY_FOR_ARCHS= aarch64 amd64 +ONLY_FOR_ARCHS_REASON= only supported on aarch64 and amd64 BUILD_DEPENDS= bash:shells/bash RUN_DEPENDS= bash:shells/bash @@ -215,6 +215,7 @@ SOVERSION= ${PORTVERSION} post-patch: @${REINPLACE_CMD} -e 's|/var/lib/clickhouse|/var/db/clickhouse|;s|/var/log/clickhouse-server|/var/log/clickhouse|;s|/etc/clickhouse-server|${PREFIX}/etc/clickhouse-server|' ${WRKSRC}/programs/server/config.xml + cd ${WRKSRC}/contrib/openldap-cmake && ${LN} -s freebsd_x86_64 freebsd_aarch64 post-install: @${RM} ${STAGEDIR}${PREFIX}/bin/config-processor diff --git a/databases/clickhouse/files/patch-cmake_find_ldap.cmake b/databases/clickhouse/files/patch-cmake_find_ldap.cmake new file mode 100644 index 000000000000..4ccad4821cf7 --- /dev/null +++ b/databases/clickhouse/files/patch-cmake_find_ldap.cmake @@ -0,0 +1,10 @@ +--- cmake/find/ldap.cmake.orig 2021-09-13 15:27:47.655692000 +0200 ++++ cmake/find/ldap.cmake 2021-09-13 15:28:02.970539000 +0200 +@@ -64,6 +64,7 @@ if (NOT OPENLDAP_FOUND AND NOT MISSING_INTERNAL_LDAP_L + ( "${_system_name}" STREQUAL "linux" AND "${_system_processor}" STREQUAL "aarch64" ) OR + ( "${_system_name}" STREQUAL "linux" AND "${_system_processor}" STREQUAL "ppc64le" ) OR + ( "${_system_name}" STREQUAL "freebsd" AND "${_system_processor}" STREQUAL "x86_64" ) OR ++ ( "${_system_name}" STREQUAL "freebsd" AND "${_system_processor}" STREQUAL "aarch64" ) OR + ( "${_system_name}" STREQUAL "darwin" AND "${_system_processor}" STREQUAL "x86_64" ) OR + ( "${_system_name}" STREQUAL "darwin" AND "${_system_processor}" STREQUAL "aarch64" ) + ) diff --git a/databases/clickhouse/files/patch-contrib_boringssl_crypto_cpu-aarch64-linux.c b/databases/clickhouse/files/patch-contrib_boringssl_crypto_cpu-aarch64-linux.c new file mode 100644 index 000000000000..50e88deb39e0 --- /dev/null +++ b/databases/clickhouse/files/patch-contrib_boringssl_crypto_cpu-aarch64-linux.c @@ -0,0 +1,75 @@ +--- contrib/boringssl/crypto/cpu-aarch64-linux.c.orig 2019-10-21 10:14:54 UTC ++++ contrib/boringssl/crypto/cpu-aarch64-linux.c +@@ -14,49 +14,47 @@ + + #include + +-#if defined(OPENSSL_AARCH64) && defined(OPENSSL_LINUX) && \ +- !defined(OPENSSL_STATIC_ARMCAP) ++#if defined(OPENSSL_AARCH64) + +-#include +- + #include + + #include "internal.h" + +- + extern uint32_t OPENSSL_armcap_P; + +-void OPENSSL_cpuid_setup(void) { +- unsigned long hwcap = getauxval(AT_HWCAP); ++#include + +- // See /usr/include/asm/hwcap.h on an aarch64 installation for the source of +- // these values. +- static const unsigned long kNEON = 1 << 1; +- static const unsigned long kAES = 1 << 3; +- static const unsigned long kPMULL = 1 << 4; +- static const unsigned long kSHA1 = 1 << 5; +- static const unsigned long kSHA256 = 1 << 6; ++#ifndef ID_AA64ISAR0_AES_VAL ++#define ID_AA64ISAR0_AES_VAL ID_AA64ISAR0_AES ++#endif ++#ifndef ID_AA64ISAR0_AES_VAL ++#define ID_AA64ISAR0_AES_VAL ID_AA64ISAR0_AES ++#endif ++#ifndef ID_AA64ISAR0_SHA1_VAL ++#define ID_AA64ISAR0_SHA1_VAL ID_AA64ISAR0_SHA1 ++#endif ++#ifndef ID_AA64ISAR0_SHA2_VAL ++#define ID_AA64ISAR0_SHA2_VAL ID_AA64ISAR0_SHA2 ++#endif + +- if ((hwcap & kNEON) == 0) { +- // Matching OpenSSL, if NEON is missing, don't report other features +- // either. +- return; +- } ++void OPENSSL_cpuid_setup(void) { ++ uint64_t id_aa64isar0; + ++ id_aa64isar0 = READ_SPECIALREG(id_aa64isar0_el1); ++ + OPENSSL_armcap_P |= ARMV7_NEON; + +- if (hwcap & kAES) { ++ if (ID_AA64ISAR0_AES_VAL(id_aa64isar0) >= ID_AA64ISAR0_AES_BASE) { + OPENSSL_armcap_P |= ARMV8_AES; + } +- if (hwcap & kPMULL) { ++ if (ID_AA64ISAR0_AES_VAL(id_aa64isar0) == ID_AA64ISAR0_AES_PMULL) { + OPENSSL_armcap_P |= ARMV8_PMULL; + } +- if (hwcap & kSHA1) { ++ if (ID_AA64ISAR0_SHA1_VAL(id_aa64isar0) == ID_AA64ISAR0_SHA1_BASE) { + OPENSSL_armcap_P |= ARMV8_SHA1; + } +- if (hwcap & kSHA256) { ++ if(ID_AA64ISAR0_SHA2_VAL(id_aa64isar0) >= ID_AA64ISAR0_SHA2_BASE) { + OPENSSL_armcap_P |= ARMV8_SHA256; + } + } +- +-#endif // OPENSSL_AARCH64 && !OPENSSL_STATIC_ARMCAP ++#endif // OPENSSL_AARCH64 diff --git a/databases/clickhouse/files/patch-contrib_jemalloc-cmake_include__freebsd__aarch64_jemalloc_internal_jemalloc__internal__defs.h.in b/databases/clickhouse/files/patch-contrib_jemalloc-cmake_include__freebsd__aarch64_jemalloc_internal_jemalloc__internal__defs.h.in new file mode 100644 index 000000000000..388d9bdec34a --- /dev/null +++ b/databases/clickhouse/files/patch-contrib_jemalloc-cmake_include__freebsd__aarch64_jemalloc_internal_jemalloc__internal__defs.h.in @@ -0,0 +1,11 @@ +--- contrib/jemalloc-cmake/include_freebsd_aarch64/jemalloc/internal/jemalloc_internal_defs.h.in.orig 2021-07-22 13:50:06.073516000 +0200 ++++ contrib/jemalloc-cmake/include_freebsd_aarch64/jemalloc/internal/jemalloc_internal_defs.h.in 2021-07-22 13:50:28.909237000 +0200 +@@ -161,7 +161,7 @@ + * JEMALLOC_DSS enables use of sbrk(2) to allocate extents from the data storage + * segment (DSS). + */ +-#define JEMALLOC_DSS ++/* #undef JEMALLOC_DSS */ + + /* Support memory filling (junk/zero). */ + #define JEMALLOC_FILL diff --git a/databases/clickhouse/files/patch-contrib_rocksdb-cmake_CMakeLists.txt b/databases/clickhouse/files/patch-contrib_rocksdb-cmake_CMakeLists.txt new file mode 100644 index 000000000000..6db0e41ea70d --- /dev/null +++ b/databases/clickhouse/files/patch-contrib_rocksdb-cmake_CMakeLists.txt @@ -0,0 +1,14 @@ +--- contrib/rocksdb-cmake/CMakeLists.txt.orig 2021-09-13 16:50:13.827707000 +0200 ++++ contrib/rocksdb-cmake/CMakeLists.txt 2021-09-13 16:51:37.118209000 +0200 +@@ -233,6 +233,11 @@ if(HAVE_AUXV_GETAUXVAL) + add_definitions(-DROCKSDB_AUXV_GETAUXVAL_PRESENT) + endif() + ++check_cxx_symbol_exists(elf_aux_info sys/auxv.h HAVE_ELF_AUX_INFO) ++if(HAVE_ELF_AUX_INFO) ++ add_definitions(-DROCKSDB_AUXV_GETAUXVAL_PRESENT) ++endif() ++ + include_directories(${ROCKSDB_SOURCE_DIR}) + include_directories("${ROCKSDB_SOURCE_DIR}/include") + if(WITH_FOLLY_DISTRIBUTED_MUTEX) diff --git a/databases/clickhouse/files/patch-contrib_rocksdb_CMakeLists.txt b/databases/clickhouse/files/patch-contrib_rocksdb_CMakeLists.txt new file mode 100644 index 000000000000..67a092844f6f --- /dev/null +++ b/databases/clickhouse/files/patch-contrib_rocksdb_CMakeLists.txt @@ -0,0 +1,13 @@ +--- contrib/rocksdb/CMakeLists.txt.orig 2021-09-13 16:40:55.686967000 +0200 ++++ contrib/rocksdb/CMakeLists.txt 2021-09-13 16:44:42.046383000 +0200 +@@ -553,6 +553,10 @@ check_cxx_symbol_exists(getauxval auvx.h HAVE_AUXV_GET + if(HAVE_AUXV_GETAUXVAL) + add_definitions(-DROCKSDB_AUXV_GETAUXVAL_PRESENT) + endif() ++check_cxx_symbol_exists(elf_aux_info sys/auxv.h HAVE_AUXV_ELF_AUX_INFO) ++if(HAVE_AUXV_ELF_AUX_INFO) ++ add_definitions(-DROCKSDB_AUXV_GETAUXVAL_PRESENT) ++endif() + + include_directories(${PROJECT_SOURCE_DIR}) + include_directories(${PROJECT_SOURCE_DIR}/include) diff --git a/databases/clickhouse/files/patch-contrib_rocksdb_util_crc32c.cc b/databases/clickhouse/files/patch-contrib_rocksdb_util_crc32c.cc new file mode 100644 index 000000000000..1c4a8699b3d2 --- /dev/null +++ b/databases/clickhouse/files/patch-contrib_rocksdb_util_crc32c.cc @@ -0,0 +1,38 @@ +--- contrib/rocksdb/util/crc32c.cc.orig 2021-07-21 16:35:20.404459000 +0200 ++++ contrib/rocksdb/util/crc32c.cc 2021-07-22 09:48:10.980923000 +0200 +@@ -41,7 +41,7 @@ + + #endif + +-#if defined(__linux__) && defined(HAVE_ARM64_CRC) ++#if (defined(__FreeBSD__) || defined(__linux__)) && defined(HAVE_ARM64_CRC) + bool pmull_runtime_flag = false; + #endif + +@@ -474,7 +474,7 @@ static bool isAltiVec() { + } + #endif + +-#if defined(__linux__) && defined(HAVE_ARM64_CRC) ++#if (defined(__FreeBSD__) || defined(__linux__)) && defined(HAVE_ARM64_CRC) + uint32_t ExtendARMImpl(uint32_t crc, const char *buf, size_t size) { + return crc32c_arm64(crc, (const unsigned char *)buf, size); + } +@@ -494,7 +494,7 @@ std::string IsFastCrc32Supported() { + has_fast_crc = false; + arch = "PPC"; + #endif +-#elif defined(__linux__) && defined(HAVE_ARM64_CRC) ++#elif (defined(__FreeBSD__) || defined(__linux__)) && defined(HAVE_ARM64_CRC) + if (crc32c_runtime_check()) { + has_fast_crc = true; + arch = "Arm64"; +@@ -1227,7 +1227,7 @@ uint32_t crc32c_3way(uint32_t crc, const char* buf, si + static inline Function Choose_Extend() { + #ifdef HAVE_POWER8 + return isAltiVec() ? ExtendPPCImpl : ExtendImpl; +-#elif defined(__linux__) && defined(HAVE_ARM64_CRC) ++#elif (defined(__FreeBSD__) || defined(__linux__)) && defined(HAVE_ARM64_CRC) + if(crc32c_runtime_check()) { + pmull_runtime_flag = crc32c_pmull_runtime_check(); + return ExtendARMImpl; diff --git a/databases/clickhouse/files/patch-contrib_rocksdb_util_crc32c__arm64.cc b/databases/clickhouse/files/patch-contrib_rocksdb_util_crc32c__arm64.cc new file mode 100644 index 000000000000..1b37e82527ef --- /dev/null +++ b/databases/clickhouse/files/patch-contrib_rocksdb_util_crc32c__arm64.cc @@ -0,0 +1,45 @@ +--- contrib/rocksdb/util/crc32c_arm64.cc.orig 2021-05-13 00:21:57.000000000 +0200 ++++ contrib/rocksdb/util/crc32c_arm64.cc 2000-01-01 02:32:56.500264000 +0100 +@@ -5,9 +5,11 @@ + + #include "util/crc32c_arm64.h" + +-#if defined(__linux__) && defined(HAVE_ARM64_CRC) ++#if (defined(__FreeBSD__) || defined(__linux__)) && defined(HAVE_ARM64_CRC) + ++#if defined(__linux__) + #include ++#endif // linux + #ifdef ROCKSDB_AUXV_GETAUXVAL_PRESENT + #include + #endif +@@ -42,7 +44,14 @@ extern bool pmull_runtime_flag; + + uint32_t crc32c_runtime_check(void) { + #ifdef ROCKSDB_AUXV_GETAUXVAL_PRESENT ++#if defined(__FreeBSD__) ++ uint64_t auxv; ++ int result = elf_aux_info(AT_HWCAP, &auxv, sizeof auxv); ++ if (result != 0) ++ return 0; ++#else + uint64_t auxv = getauxval(AT_HWCAP); ++#endif + return (auxv & HWCAP_CRC32) != 0; + #else + return 0; +@@ -51,7 +60,14 @@ uint32_t crc32c_runtime_check(void) { + + bool crc32c_pmull_runtime_check(void) { + #ifdef ROCKSDB_AUXV_GETAUXVAL_PRESENT ++#if defined(__FreeBSD__) ++ uint64_t auxv; ++ int result = elf_aux_info(AT_HWCAP, &auxv, sizeof auxv); ++ if (result != 0) ++ return 0; ++#else + uint64_t auxv = getauxval(AT_HWCAP); ++#endif + return (auxv & HWCAP_PMULL) != 0; + #else + return false; diff --git a/databases/clickhouse/files/patch-src_Common_StackTrace.cpp b/databases/clickhouse/files/patch-src_Common_StackTrace.cpp new file mode 100644 index 000000000000..bac5c5e25a8d --- /dev/null +++ b/databases/clickhouse/files/patch-src_Common_StackTrace.cpp @@ -0,0 +1,11 @@ +--- src/Common/StackTrace.cpp.orig 2021-09-13 15:30:01.608345000 +0200 ++++ src/Common/StackTrace.cpp 2021-09-13 15:30:10.009331000 +0200 +@@ -186,6 +186,8 @@ static void * getCallerAddress(const ucontext_t & context) + #elif defined(__APPLE__) && defined(__aarch64__) + return reinterpret_cast(context.uc_mcontext->__ss.__pc); + ++#elif defined(__FreeBSD__) && defined(__aarch64__) ++ return reinterpret_cast(context.uc_mcontext.mc_gpregs.gp_elr); + #elif defined(__aarch64__) + return reinterpret_cast(context.uc_mcontext.pc); + #elif defined(__powerpc64__)