From nobody Tue Sep 13 20:32:40 2022 X-Original-To: dev-commits-ports-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4MRwBc2MM5z4cNCQ; Tue, 13 Sep 2022 20:32:40 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4MRwBc1ZtRz3bwR; Tue, 13 Sep 2022 20:32:40 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1663101160; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=43UT8KBU4LaJxQpaTXvZ5pgh29s15mhUrfjxTcuQNJY=; b=BdsErb1Ec8Le7dbv6mcpQYzVBYfiAq5hJ9TWI99UQftbH8E7edmwPzHMwU/3trX53aQI6p lsQEEsTdtkzdm723TOA9+1dvGBP49f+pqU8VGe6huIwiERCUdnfTNBK1kWoGhlVNa5gYbG zZ44HqEy839XnWUFDTJlwyvb50lWdi2bfs3old5gE51paayc3yxmv7AFUJAFGT6x0Xmw9T VdbuMftopeHjzeMZsvCWCOyHeBeMRaYEFh/XGKLq1TYp79wz0q5eSwavIN8rv2UCGlOgLV 7Fos6PKKzOXhO4e1MkVS2S2u7BkhCPkxVMo+uFs4GKPKJk3vLK9XwHg8zjWrVg== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4MRwBc0bxqzRxw; Tue, 13 Sep 2022 20:32:40 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 28DKWePu072551; Tue, 13 Sep 2022 20:32:40 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 28DKWenv072550; Tue, 13 Sep 2022 20:32:40 GMT (envelope-from git) Date: Tue, 13 Sep 2022 20:32:40 GMT Message-Id: <202209132032.28DKWenv072550@gitrepo.freebsd.org> To: ports-committers@FreeBSD.org, dev-commits-ports-all@FreeBSD.org, dev-commits-ports-main@FreeBSD.org From: Yuri Victorovich Subject: git: 2dd8869d4b07 - main - math/py-keras: Update 2.4.3 -> 2.9.0 List-Id: Commits to the main branch of the FreeBSD ports repository List-Archive: https://lists.freebsd.org/archives/dev-commits-ports-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-ports-main@freebsd.org X-BeenThere: dev-commits-ports-main@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: yuri X-Git-Repository: ports X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 2dd8869d4b07e101d86d00bb01f83c9787ed9007 Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1663101160; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=43UT8KBU4LaJxQpaTXvZ5pgh29s15mhUrfjxTcuQNJY=; b=L02pWrARW3p6Ar6WbaLV5bgRc7VADGkgk1I/tHTw8H145J1gcp6h4sq9p8gT76EjMm69wZ ivMj6d0T4WLUOe357VSnkJKUzz7npejIIB5c6mbf3NCHiUcbMgzj2tBnrtAURUo9Y20w3Y z7F5Eh6wbaMKezKep+fJtQm/X569J9eSR3u1qrumujUFeVyfHxICoeCM/YVmvoDGB2rkjU HGWM2YvwSMogScsy7Is6hoekin5OjSG5j3kriZSYNfSEHrWEagbyxyBjG62UlWfD2Y7nSK b2DSIbbud6VOeexClmw4t8YySsbTyShkUg13kuv1BQrE/tLx7Cov7APY9Bj+OA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1663101160; a=rsa-sha256; cv=none; b=vCEW46u6Vyo1qcSCFQH5qxvCL1v8BgeYDOgWkp5D4WxkA/IfIx0KAdMdZbNnT3C3suVV5Z tR2OjjKZa8yNUGXWcmxxWmFQGGngK/eCf0NiNh5gqAVsxgz3YiT5SvKZPUMlTK4P0FI3AR E5omV/toVYt/SEg3GMyLzCgnOC6Av/Xo6CYsyIu22+U/f2CMMHAswCIOiBX4XlzoAca3P5 LRFG6sUaQAciSmBWGQsk84gM+j3CdpdGYg4ja/eCPfg0SEei9c3RZcOlIwTAfdFmnbVqdE EijBKpgeM8uH1ioFC5bBPthhiUwR1+PR2LwPSSBKAu5eIW7EVUejNXZlVeK4ZA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch main has been updated by yuri: URL: https://cgit.FreeBSD.org/ports/commit/?id=2dd8869d4b07e101d86d00bb01f83c9787ed9007 commit 2dd8869d4b07e101d86d00bb01f83c9787ed9007 Author: Anthony Donnelly AuthorDate: 2022-09-13 20:30:31 +0000 Commit: Yuri Victorovich CommitDate: 2022-09-13 20:32:36 +0000 math/py-keras: Update 2.4.3 -> 2.9.0 PR: 266311 --- math/py-keras/Makefile | 100 +++- math/py-keras/Makefile.MASTER_SITES | 5 + math/py-keras/distinfo | 10 +- math/py-keras/files/bazelrc | 18 + math/py-keras/files/freebsd/BUILD | 88 ++++ .../py-keras/files/freebsd/cc_toolchain_config.bzl | 287 +++++++++++ math/py-keras/files/patch-.bazelrc | 13 + math/py-keras/files/patch-WORKSPACE | 57 ++ math/py-keras/files/patch-keras_protobuf_BUILD | 11 + .../files/patch-keras_protobuf_protobuf.bzl | 220 ++++++++ ...keras_tools_pip__package_build__pip__package.sh | 20 + math/py-keras/files/patch-third__party_BUILD | 19 + math/py-keras/files/pkg-message.in | 9 - math/py-keras/pkg-descr | 2 + math/py-keras/pkg-plist | 572 +++++++++++++++++++++ 15 files changed, 1407 insertions(+), 24 deletions(-) diff --git a/math/py-keras/Makefile b/math/py-keras/Makefile index d1b5992aac08..4cccb73aadbf 100644 --- a/math/py-keras/Makefile +++ b/math/py-keras/Makefile @@ -1,30 +1,106 @@ PORTNAME= keras -DISTVERSION= 2.4.3 +DISTVERSION= 2.9.0 CATEGORIES= math # machine-learning -MASTER_SITES= CHEESESHOP PKGNAMEPREFIX= ${PYTHON_PKGNAMEPREFIX} -DISTNAME= Keras-${PORTVERSION} MAINTAINER= yuri@FreeBSD.org COMMENT= Deep learning library for Python -WWW= https://keras.io/ + +USE_GITHUB= yes +GH_ACCOUNT= keras-team +GH_PROJECT= ${PORTNAME} +GH_TAGNAME= r2.9 LICENSE= MIT LICENSE_FILE= ${WRKSRC}/LICENSE -RUN_DEPENDS= ${PYNUMPY} \ +RUN_DEPENDS= ${PYNUMPY} \ + ${PYTHON_PKGNAMEPREFIX}absl-py>=1.0.0:devel/py-absl-py \ ${PYTHON_PKGNAMEPREFIX}h5py>=0:science/py-h5py@${PY_FLAVOR} \ - ${PYTHON_PKGNAMEPREFIX}keras-applications>=1.0.6:math/py-keras-applications@${PY_FLAVOR} \ - ${PYTHON_PKGNAMEPREFIX}keras-preprocessing>=1.0.5:math/py-keras-preprocessing@${PY_FLAVOR} \ + ${PYTHON_PKGNAMEPREFIX}pandas>=1.3.5:math/py-pandas@${PY_FLAVOR} \ + ${PYTHON_PKGNAMEPREFIX}pillow>=9.2.0:graphics/py-pillow@${PY_FLAVOR} \ + ${PYTHON_PKGNAMEPREFIX}keras-preprocessing>=1.1.0:math/py-keras-preprocessing@${PY_FLAVOR} \ ${PYTHON_PKGNAMEPREFIX}scipy>=0.14:science/py-scipy@${PY_FLAVOR} \ ${PYTHON_PKGNAMEPREFIX}six>=1.9.0:devel/py-six@${PY_FLAVOR} \ - ${PYTHON_PKGNAMEPREFIX}theano>=0.8.0:math/py-theano@${PY_FLAVOR} \ - ${PYTHON_PKGNAMEPREFIX}yaml>0:devel/py-yaml@${PY_FLAVOR} + ${PYTHON_PKGNAMEPREFIX}protobuf>=3.13.0:devel/py-protobuf@${PY_FLAVOR} \ + ${PYTHON_PKGNAMEPREFIX}keras-applications>=1.0.8:math/py-keras-applications@${PY_FLAVOR} \ + ${PYTHON_PKGNAMEPREFIX}keras-preprocessing>=1.1.1:math/py-keras-preprocessing@${PY_FLAVOR} + +BUILD_DEPENDS= ${RUN_DEPENDS} \ + ${PYTHON_PKGNAMEPREFIX}tensorflow>=2.9.0:science/py-tensorflow@${PY_FLAVOR} \ + bazel:devel/bazel \ + bash:shells/bash USES= python:3.7+ -USE_PYTHON= autoplist distutils -NO_ARCH= yes -SUB_FILES= pkg-message +.include "Makefile.MASTER_SITES" + +post-patch: + # don't cache stuff to $HOME/.cache/ + @${MKDIR} ${WRKDIR}/bazel-cache + @${MKDIR} ${WRKDIR}/bazel-dist + +.for file in ${DISTFILES:C/\:(.*)//} + @${ECHO} "Moving ${file} to ${WRKDIR}/bazel-dist" + @${CP} ${DISTDIR}/${DIST_SUBDIR}/${file} ${WRKDIR}/bazel-dist +.endfor + + # place the generate wheel file here + @${MKDIR} ${WRKDIR}/whl + + # These files are from my tensorflow port + # if both make it into the port they can be shared + + # copy the toolchain over + @${CP} -R ${PATCHDIR}/freebsd \ + ${WRKSRC}/ + + @${CP} ${PATCHDIR}/bazelrc \ + ${WRKSRC}/ + + #setup localbase + @${REINPLACE_CMD} "s#%%LOCALBASE%%#${LOCALBASE}#" \ + ${WRKSRC}/freebsd/cc_toolchain_config.bzl \ + ${WRKSRC}/.bazelrc \ + ${WRKSRC}/bazelrc + + # setup our bazelrc + @${REINPLACE_CMD} "s#%%BAZEL_DIR%%#${WRKDIR}#" ${WRKSRC}/bazelrc + @${REINPLACE_CMD} "s#%%BAZEL_DIST%%#${WRKDIR}#" ${WRKSRC}/bazelrc + @${REINPLACE_CMD} "s#%%PATH%%#${PATH}#" ${WRKSRC}/bazelrc + @${REINPLACE_CMD} "s#%%PYTHON%%#${PYTHON_CMD}#" ${WRKSRC}/.bazelrc + + # bazel is horrible, it will won't propagated enviroment variable throughout the build + # and will ignore variables set by us. This causing a lot of issues, especially with python + # auto generated files will sometimes ignore our py_runtime toolchain and our PYTHON_BIN_PATH + # causing it to still set the shebang line to bin/env python3. Even on the github page the solution + # has been to use a symlink to work around so do that here as a fix + @${MKDIR} ${WRKDIR}/.bin + ${LN} -s ${PYTHON_CMD} ${WRKDIR}/.bin/python3 + + @${REINPLACE_CMD} "s#%%PYTHON%%#${PYTHON_CMD}#" \ + ${WRKSRC}/keras/tools/pip_package/build_pip_package.sh + +do-build: + @cd ${WRKSRC} && ${SETENV} \ + PYTHON_BIN_PATH=${PYTHON_CMD} \ + PYTHON_LIB_PATH="${PYTHON_SITELIBDIR}" \ + PYTHON_BINARY=${PYTHON_CMD} \ + bazel --bazelrc="${WRKSRC}/bazelrc" \ + build --config=freebsd \ + //keras/tools/pip_package:build_pip_package + +# @${REINPLACE_CMD} "s#python#${PYTHON_CMD}#" \ +# ${WRKSRC}/bazel-bin/keras/tools/pip_package/build_pip_package + + @cd ${WRKSRC} && \ + bazel-bin/keras/tools/pip_package/build_pip_package \ + ${WRKDIR}/whl + + +do-install: + @${MKDIR} ${STAGEDIR}/${PYTHON_SITELIBDIR} + + ${UNZIP_NATIVE_CMD} -d ${STAGEDIR}/${PYTHON_SITELIBDIR} ${WRKDIR}/whl/${PORTNAME}-${PORTVERSION}-*.whl .include diff --git a/math/py-keras/Makefile.MASTER_SITES b/math/py-keras/Makefile.MASTER_SITES new file mode 100644 index 000000000000..cd5c112cf8ea --- /dev/null +++ b/math/py-keras/Makefile.MASTER_SITES @@ -0,0 +1,5 @@ +DISTFILES+= b1c40e1de81913a3c40e5948f78719c28152486d.zip:rulescc \ + 7cf3cefd652008d0a64a419c34c13bdca6c8f178.zip:javarules + +MASTER_SITES+= https://mirror.bazel.build/github.com/bazelbuild/rules_cc/archive/:rulescc \ + https://github.com/bazelbuild/rules_java/archive/:javarules diff --git a/math/py-keras/distinfo b/math/py-keras/distinfo index e5dff4bae627..865f68978d88 100644 --- a/math/py-keras/distinfo +++ b/math/py-keras/distinfo @@ -1,3 +1,7 @@ -TIMESTAMP = 1605075225 -SHA256 (Keras-2.4.3.tar.gz) = fedd729b52572fb108a98e3d97e1bac10a81d3917d2103cc20ab2a5f03beb973 -SIZE (Keras-2.4.3.tar.gz) = 291047 +TIMESTAMP = 1662706867 +SHA256 (b1c40e1de81913a3c40e5948f78719c28152486d.zip) = d0c573b94a6ef20ef6ff20154a23d0efcb409fb0e1ff0979cec318dfe42f0cdd +SIZE (b1c40e1de81913a3c40e5948f78719c28152486d.zip) = 201360 +SHA256 (7cf3cefd652008d0a64a419c34c13bdca6c8f178.zip) = bc81f1ba47ef5cc68ad32225c3d0e70b8c6f6077663835438da8d5733f917598 +SIZE (7cf3cefd652008d0a64a419c34c13bdca6c8f178.zip) = 9422 +SHA256 (keras-team-keras-2.9.0-r2.9_GH0.tar.gz) = 90bded417306a1fad2d7088009d87972afbac8d6eee29d3f24711df6994ac51a +SIZE (keras-team-keras-2.9.0-r2.9_GH0.tar.gz) = 2142794 diff --git a/math/py-keras/files/bazelrc b/math/py-keras/files/bazelrc new file mode 100644 index 000000000000..f1efb9602f3e --- /dev/null +++ b/math/py-keras/files/bazelrc @@ -0,0 +1,18 @@ +startup --output_user_root="%%BAZEL_DIR%%/.cache/" + +# make bazel only fetch distfiles from the cache +fetch --repository_cache="%%BAZEL_DIR%%/bazel-cache/" --distdir="%%BAZEL_DIST%%/bazel-dist/" +build --repository_cache="%%BAZEL_DIR%%/bazel-cache/" --distdir="%%BAZEL_DIST%%/bazel-dist/" + +build --define=PREFIX=%%LOCALBASE%% +build --define=LIBDIR=%%LOCALBASE%%/lib +build --define=INCLUDEDIR=%%LOCALBASE%%/include +build --define=PROTOBUF_INCLUDE_PATH=%%LOCALBASE%%/include + +build --extra_toolchains=//freebsd:cc-toolchain-freebsd + +# for python +build --action_env=PATH=%%PATH%% +build --host_action_env=PATH=%%PATH%% + +build --config=freebsd diff --git a/math/py-keras/files/freebsd/BUILD b/math/py-keras/files/freebsd/BUILD new file mode 100644 index 000000000000..4096b1c74d42 --- /dev/null +++ b/math/py-keras/files/freebsd/BUILD @@ -0,0 +1,88 @@ +# Copyright 2018 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# This becomes the BUILD file for @local_config_cc// under FreeBSD and OpenBSD. +load(":cc_toolchain_config.bzl", "cc_toolchain_config") +load("@rules_cc//cc:defs.bzl", "cc_library", "cc_toolchain", "cc_toolchain_suite") +load("@bazel_tools//tools/python:toolchain.bzl", "py_runtime_pair") + +package(default_visibility = ["//visibility:public"]) +cc_library( + name = "malloc", +) +filegroup( + name = "empty", + srcs = [], +) + +# Hardcoded toolchain, legacy behaviour. +cc_toolchain_suite( + name = "freebsd_clang", + toolchains = { + "k8": ":cc-compiler-freebsd", + "freebsd": ":cc-compiler-freebsd", + }, +) + +cc_toolchain( + name = "cc-compiler-freebsd", + all_files = ":empty", + ar_files = ":empty", + as_files = ":empty", + compiler_files = ":empty", + dwp_files = ":empty", + linker_files = ":empty", + objcopy_files = ":empty", + strip_files = ":empty", + supports_param_files = 0, + toolchain_config = ":local_freebsd", + toolchain_identifier = "local_freebsd", +) +cc_toolchain_config( + name = "local_freebsd", + cpu = "k8", +) + +toolchain( + name = "cc-toolchain-freebsd", + exec_compatible_with = [ + "@platforms//cpu:x86_64", + "@platforms//os:freebsd", + ], + target_compatible_with = [ + "@platforms//cpu:x86_64", + "@platforms//os:freebsd", + ], + toolchain = ":cc-compiler-freebsd", + toolchain_type = "@bazel_tools//tools/cpp:toolchain_type", +) + +py_runtime( + name = "freebsd_python", + interpreter_path = "/usr/local/bin/python3.9", + python_version = "PY3", + visibility = ["//visibility:public"], +) + +py_runtime_pair( + name = "bsd_py_runtime_pair", + py2_runtime = None, + py3_runtime = ":freebsd_python", +) + +toolchain( + name = "freebsd_py_toolchain", + toolchain = ":bsd_py_runtime_pair", + toolchain_type = "@bazel_tools//tools/python:toolchain_type", +) + diff --git a/math/py-keras/files/freebsd/cc_toolchain_config.bzl b/math/py-keras/files/freebsd/cc_toolchain_config.bzl new file mode 100644 index 000000000000..51f2393e7bd2 --- /dev/null +++ b/math/py-keras/files/freebsd/cc_toolchain_config.bzl @@ -0,0 +1,287 @@ +# Copyright 2019 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +"""A Starlark cc_toolchain configuration rule for FreeBSD and OpenBSD.""" +load( + "@bazel_tools//tools/cpp:cc_toolchain_config_lib.bzl", + "action_config", + "feature", + "flag_group", + "flag_set", + "tool", + "tool_path", + "with_feature_set", +) + +load("@bazel_tools//tools/build_defs/cc:action_names.bzl", "ACTION_NAMES") +all_compile_actions = [ + ACTION_NAMES.c_compile, + ACTION_NAMES.cpp_compile, + ACTION_NAMES.linkstamp_compile, + ACTION_NAMES.assemble, + ACTION_NAMES.preprocess_assemble, + ACTION_NAMES.cpp_header_parsing, + ACTION_NAMES.cpp_module_compile, + ACTION_NAMES.cpp_module_codegen, + ACTION_NAMES.clif_match, + ACTION_NAMES.lto_backend, +] +all_cpp_compile_actions = [ + ACTION_NAMES.cpp_compile, + ACTION_NAMES.linkstamp_compile, + ACTION_NAMES.cpp_header_parsing, + ACTION_NAMES.cpp_module_compile, + ACTION_NAMES.cpp_module_codegen, + ACTION_NAMES.clif_match, +] +all_link_actions = [ + ACTION_NAMES.cpp_link_executable, + ACTION_NAMES.cpp_link_dynamic_library, + ACTION_NAMES.cpp_link_nodeps_dynamic_library, +] +def _impl(ctx): + cpu = ctx.attr.cpu + is_bsd = cpu == "k8" or cpu == "freebsd" or cpu == "openbsd" + compiler = "compiler" + toolchain_identifier = "local_freebsd" + host_system_name = "local" if is_bsd else "armeabi-v7a" + target_system_name = "local" if is_bsd else "armeabi-v7a" + target_libc = "local" if is_bsd else "armeabi-v7a" + abi_version = "local" if is_bsd else "armeabi-v7a" + abi_libc_version = "local" if is_bsd else "armeabi-v7a" + objcopy_embed_data_action = action_config( + action_name = "objcopy_embed_data", + enabled = True, + tools = [tool(path = "/usr/bin/objcopy")], + ) + action_configs = [objcopy_embed_data_action] if is_bsd else [] + default_link_flags_feature = feature( + name = "default_link_flags", + enabled = True, + flag_sets = [ + flag_set( + actions = all_link_actions, + flag_groups = [ + flag_group( + flags = [ + "-Wl,-undefined,dynamic_lookup", + "-lc++", + "-Wl,-z,relro,-z,now", + "-no-canonical-prefixes", + "-L%%LOCALBASE%%/lib", + ], + ), + ], + ), + flag_set( + actions = all_link_actions, + flag_groups = [flag_group(flags = ["-Wl,--gc-sections"])], + with_features = [with_feature_set(features = ["opt"])], + ), + ], + ) + unfiltered_compile_flags_feature = feature( + name = "unfiltered_compile_flags", + enabled = True, + flag_sets = [ + flag_set( + actions = all_compile_actions, + flag_groups = [ + flag_group( + flags = [ + "-no-canonical-prefixes", + "-Wno-builtin-macro-redefined", + "-D__DATE__=\"redacted\"", + "-D__TIMESTAMP__=\"redacted\"", + "-D__TIME__=\"redacted\"", + ], + ), + ], + ), + ], + ) + supports_pic_feature = feature(name = "supports_pic", enabled = True) + default_compile_flags_feature = feature( + name = "default_compile_flags", + enabled = True, + flag_sets = [ + flag_set( + actions = all_compile_actions, + flag_groups = [ + flag_group( + flags = [ + "-U_FORTIFY_SOURCE", + "-D_FORTIFY_SOURCE=1", + "-fstack-protector", + "-Wall", + "-fno-omit-frame-pointer", + ], + ), + ], + ), + flag_set( + actions = all_compile_actions, + flag_groups = [flag_group(flags = ["-g"])], + with_features = [with_feature_set(features = ["dbg"])], + ), + flag_set( + actions = all_compile_actions, + flag_groups = [ + flag_group( + flags = [ + "-g0", + "-O2", + "-DNDEBUG", + "-ffunction-sections", + "-fdata-sections", + ], + ), + ], + with_features = [with_feature_set(features = ["opt"])], + ), + flag_set( + actions = all_cpp_compile_actions + [ACTION_NAMES.lto_backend], + flag_groups = [flag_group(flags = ["-std=c++17"])], + ), + ], + ) + opt_feature = feature(name = "opt") + supports_dynamic_linker_feature = feature(name = "supports_dynamic_linker", enabled = True) + objcopy_embed_flags_feature = feature( + name = "objcopy_embed_flags", + enabled = True, + flag_sets = [ + flag_set( + actions = ["objcopy_embed_data"], + flag_groups = [flag_group(flags = ["-I", "binary"])], + ), + ], + ) + dbg_feature = feature(name = "dbg") + user_compile_flags_feature = feature( + name = "user_compile_flags", + enabled = True, + flag_sets = [ + flag_set( + actions = all_compile_actions, + flag_groups = [ + flag_group( + flags = ["%{user_compile_flags}"], + iterate_over = "user_compile_flags", + expand_if_available = "user_compile_flags", + ), + ], + ), + ], + ) + sysroot_feature = feature( + name = "sysroot", + enabled = True, + flag_sets = [ + flag_set( + actions = [ + ACTION_NAMES.c_compile, + ACTION_NAMES.cpp_compile, + ACTION_NAMES.linkstamp_compile, + ACTION_NAMES.preprocess_assemble, + ACTION_NAMES.cpp_header_parsing, + ACTION_NAMES.cpp_module_compile, + ACTION_NAMES.cpp_module_codegen, + ACTION_NAMES.clif_match, + ACTION_NAMES.lto_backend, + ] + all_link_actions, + flag_groups = [ + flag_group( + flags = ["--sysroot=%{sysroot}"], + expand_if_available = "sysroot", + ), + ], + ), + ], + ) + if is_bsd: + features = [ + default_compile_flags_feature, + default_link_flags_feature, + supports_dynamic_linker_feature, + supports_pic_feature, + objcopy_embed_flags_feature, + opt_feature, + dbg_feature, + user_compile_flags_feature, + sysroot_feature, + unfiltered_compile_flags_feature, + ] + else: + features = [supports_dynamic_linker_feature, supports_pic_feature] + if (is_bsd): + cxx_builtin_include_directories = ["/usr/lib/clang/13.0.0/include", "/usr/include/c++/v1", "/usr/local/include", "/usr/include"] + else: + cxx_builtin_include_directories = [] + if is_bsd: + tool_paths = [ + tool_path(name = "ar", path = "/usr/bin/ar"), + tool_path(name = "cpp", path = "/usr/bin/clang-cpp"), + tool_path(name = "dwp", path = "/usr/bin/dwp"), + tool_path(name = "gcc", path = "/usr/bin/clang"), + tool_path(name = "gcov", path = "/usr/bin/gcov"), + tool_path(name = "ld", path = "/usr/bin/ld"), + tool_path(name = "nm", path = "/usr/bin/nm"), + tool_path(name = "objcopy", path = "/usr/bin/objcopy"), + tool_path(name = "objdump", path = "/usr/bin/objdump"), + tool_path(name = "strip", path = "/usr/bin/strip"), + ] + else: + tool_paths = [ + tool_path(name = "ar", path = "/bin/false"), + tool_path(name = "cpp", path = "/bin/false"), + tool_path(name = "dwp", path = "/bin/false"), + tool_path(name = "gcc", path = "/bin/false"), + tool_path(name = "gcov", path = "/bin/false"), + tool_path(name = "ld", path = "/bin/false"), + tool_path(name = "nm", path = "/bin/false"), + tool_path(name = "objcopy", path = "/bin/false"), + tool_path(name = "objdump", path = "/bin/false"), + tool_path(name = "strip", path = "/bin/false"), + ] + out = ctx.actions.declare_file(ctx.label.name) + ctx.actions.write(out, "Fake executable") + return [ + cc_common.create_cc_toolchain_config_info( + ctx = ctx, + features = features, + action_configs = action_configs, + cxx_builtin_include_directories = cxx_builtin_include_directories, + toolchain_identifier = toolchain_identifier, + host_system_name = host_system_name, + target_system_name = target_system_name, + target_cpu = cpu, + target_libc = target_libc, + compiler = compiler, + abi_version = abi_version, + abi_libc_version = abi_libc_version, + tool_paths = tool_paths, + ), + DefaultInfo( + executable = out, + ), + ] + +cc_toolchain_config = rule( + implementation = _impl, + attrs = { + "cpu": attr.string(mandatory = True), + }, + provides = [CcToolchainConfigInfo], + executable = True, +) diff --git a/math/py-keras/files/patch-.bazelrc b/math/py-keras/files/patch-.bazelrc new file mode 100644 index 000000000000..31d69a3184cb --- /dev/null +++ b/math/py-keras/files/patch-.bazelrc @@ -0,0 +1,13 @@ +--- .bazelrc.orig 2022-04-22 17:34:55 UTC ++++ .bazelrc +@@ -106,7 +106,9 @@ build:macos --define=PREFIX=/usr + build:macos --define=LIBDIR=$(PREFIX)/lib + build:macos --define=INCLUDEDIR=$(PREFIX)/include + build:macos --define=PROTOBUF_INCLUDE_PATH=$(PREFIX)/include +-# TF_SYSTEM_LIBS do not work on windows. ++ ++build:freebsd --define force_libcpp=enabled ++build:freebsd --action_env=CXXFLAGS=-stdlib=libc++ + + # On windows, we still link everything into a single DLL. + build:windows --config=monolithic diff --git a/math/py-keras/files/patch-WORKSPACE b/math/py-keras/files/patch-WORKSPACE new file mode 100644 index 000000000000..c4ce74c61bd8 --- /dev/null +++ b/math/py-keras/files/patch-WORKSPACE @@ -0,0 +1,57 @@ +--- WORKSPACE.orig 2022-09-08 09:13:13 UTC ++++ WORKSPACE +@@ -1,54 +1 @@ + workspace(name = "org_keras") +- +-load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive") +- +-# Needed by protobuf +-load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive") +-http_archive( +- name = "bazel_skylib", +- url = "https://github.com/bazelbuild/bazel-skylib/releases/download/1.0.1/bazel-skylib-1.0.1.tar.gz", +- sha256 = "f1c8360c01fcf276778d3519394805dc2a71a64274a3a0908bc9edff7b5aebc8", +-) +-load("@bazel_skylib//:workspace.bzl", "bazel_skylib_workspace") +-bazel_skylib_workspace() +- +-# Needed by protobuf +-http_archive( +- name = "six_archive", +- build_file = "//third_party:six.BUILD", +- sha256 = "d16a0141ec1a18405cd4ce8b4613101da75da0e9a7aec5bdd4fa804d0e0eba73", +- strip_prefix = "six-1.12.0", +- urls = [ +- "http://mirror.bazel.build/pypi.python.org/packages/source/s/six/six-1.12.0.tar.gz", +- "https://pypi.python.org/packages/source/s/six/six-1.12.0.tar.gz", # 2018-12-10 +- ], +-) +- +-bind( +- name = "six", +- actual = "@six_archive//:six", +-) +- +-http_archive( +- name = "com_google_protobuf", +- sha256 = "1fbf1c2962af287607232b2eddeaec9b4f4a7a6f5934e1a9276e9af76952f7e0", +- strip_prefix = "protobuf-3.9.2", +- urls = ["https://github.com/protocolbuffers/protobuf/archive/v3.9.2.tar.gz"], +-) +- +-# ZLIB +-# Need by com_google_protobuf. Note that the original URL from zlib side is not +-# available for now. We need to use bazel mirror as a backup. +-http_archive( +- name = "zlib", +- build_file = "@com_google_protobuf//:third_party/zlib.BUILD", +- sha256 = "c3e5e9fdd5004dcb542feda5ee4f0ff0744628baf8ed2dd5d66f8ca1197cb1a1", +- strip_prefix = "zlib-1.2.11", +- urls = [ +- "https://mirror.bazel.build/zlib.net/zlib-1.2.11.tar.gz", +- "https://zlib.net/zlib-1.2.11.tar.gz", +- ], +-) +- +-load("@com_google_protobuf//:protobuf_deps.bzl", "protobuf_deps") +-protobuf_deps() diff --git a/math/py-keras/files/patch-keras_protobuf_BUILD b/math/py-keras/files/patch-keras_protobuf_BUILD new file mode 100644 index 000000000000..e8ba4c72a808 --- /dev/null +++ b/math/py-keras/files/patch-keras_protobuf_BUILD @@ -0,0 +1,11 @@ +--- keras/protobuf/BUILD.orig 2022-09-08 09:14:15 UTC ++++ keras/protobuf/BUILD +@@ -1,7 +1,7 @@ + # Description: + # Contains Keras protobufs + +-load("@com_google_protobuf//:protobuf.bzl", "py_proto_library") ++load("//third_party:protobuf.bzl", "py_proto_library") + + package( + default_visibility = [ diff --git a/math/py-keras/files/patch-keras_protobuf_protobuf.bzl b/math/py-keras/files/patch-keras_protobuf_protobuf.bzl new file mode 100644 index 000000000000..0ffcff335127 --- /dev/null +++ b/math/py-keras/files/patch-keras_protobuf_protobuf.bzl @@ -0,0 +1,220 @@ +--- /dev/null 2022-09-08 10:18:38.049755000 +0100 ++++ third_party/protobuf.bzl 2022-09-08 10:17:38.840977000 +0100 +@@ -0,0 +1,217 @@ ++def _GetPath(ctx, path): ++ if ctx.label.workspace_root: ++ return ctx.label.workspace_root + "/" + path ++ else: ++ return path ++ ++def _IsNewExternal(ctx): ++ # Bazel 0.4.4 and older have genfiles paths that look like: ++ # bazel-out/local-fastbuild/genfiles/external/repo/foo ++ # After the exec root rearrangement, they look like: ++ # ../repo/bazel-out/local-fastbuild/genfiles/foo ++ return ctx.label.workspace_root.startswith("../") ++ ++def _GenDir(ctx): ++ if _IsNewExternal(ctx): ++ # We are using the fact that Bazel 0.4.4+ provides repository-relative paths ++ # for ctx.genfiles_dir. ++ return ctx.genfiles_dir.path + ( ++ "/" + ctx.attr.includes[0] if ctx.attr.includes and ctx.attr.includes[0] else "" ++ ) ++ ++ # This means that we're either in the old version OR the new version in the local repo. ++ # Either way, appending the source path to the genfiles dir works. ++ return ctx.var["GENDIR"] + "/" + _SourceDir(ctx) ++ ++def _SourceDir(ctx): ++ if not ctx.attr.includes: ++ return ctx.label.workspace_root ++ if not ctx.attr.includes[0]: ++ return _GetPath(ctx, ctx.label.package) ++ if not ctx.label.package: ++ return _GetPath(ctx, ctx.attr.includes[0]) ++ return _GetPath(ctx, ctx.label.package + "/" + ctx.attr.includes[0]) ++ ++def _PyOuts(srcs, use_grpc_plugin = False): ++ ret = [s[:-len(".proto")] + "_pb2.py" for s in srcs] ++ if use_grpc_plugin: ++ ret += [s[:-len(".proto")] + "_pb2_grpc.py" for s in srcs] ++ return ret ++ ++def _proto_gen_impl(ctx): ++ """General implementation for generating protos""" ++ srcs = ctx.files.srcs ++ deps = [] ++ deps += ctx.files.srcs ++ source_dir = _SourceDir(ctx) ++ gen_dir = _GenDir(ctx) ++ if source_dir: ++ import_flags = ["-I" + source_dir, "-I" + gen_dir] ++ else: ++ import_flags = ["-I."] ++ ++ for dep in ctx.attr.deps: ++ import_flags += dep.proto.import_flags ++ deps += dep.proto.deps ++ import_flags = depset(import_flags).to_list() ++ deps = depset(deps).to_list() ++ ++ args = [] ++ if ctx.attr.gen_cc: ++ args += ["--cpp_out=" + gen_dir] ++ if ctx.attr.gen_py: ++ args += ["--python_out=" + gen_dir] ++ ++ inputs = srcs + deps ++ tools = [ctx.executable.protoc] ++ if ctx.executable.plugin: ++ plugin = ctx.executable.plugin ++ lang = ctx.attr.plugin_language ++ if not lang and plugin.basename.startswith("protoc-gen-"): ++ lang = plugin.basename[len("protoc-gen-"):] ++ if not lang: ++ fail("cannot infer the target language of plugin", "plugin_language") ++ ++ outdir = gen_dir ++ if ctx.attr.plugin_options: ++ outdir = ",".join(ctx.attr.plugin_options) + ":" + outdir ++ args += ["--plugin=protoc-gen-%s=%s" % (lang, plugin.path)] ++ args += ["--%s_out=%s" % (lang, outdir)] ++ tools.append(plugin) ++ ++ if args: ++ ctx.actions.run( ++ inputs = inputs, ++ outputs = ctx.outputs.outs, ++ arguments = args + import_flags + [s.path for s in srcs], ++ executable = ctx.executable.protoc, ++ mnemonic = "ProtoCompile", ++ tools = tools, ++ use_default_shell_env = True, ++ ) ++ ++ return struct( ++ proto = struct( ++ srcs = srcs, ++ import_flags = import_flags, ++ deps = deps, ++ ), ++ ) ++ ++proto_gen = rule( ++ attrs = { ++ "srcs": attr.label_list(allow_files = True), ++ "deps": attr.label_list(providers = ["proto"]), ++ "includes": attr.string_list(), ++ "protoc": attr.label( ++ cfg = "host", ++ executable = True, ++ allow_single_file = True, ++ mandatory = True, ++ ), ++ "plugin": attr.label( ++ cfg = "host", ++ allow_files = True, ++ executable = True, ++ ), ++ "plugin_language": attr.string(), ++ "plugin_options": attr.string_list(), ++ "gen_cc": attr.bool(), ++ "gen_py": attr.bool(), ++ "outs": attr.output_list(), ++ }, ++ output_to_genfiles = True, ++ implementation = _proto_gen_impl, ++) ++"""Generates codes from Protocol Buffers definitions. ++ ++This rule helps you to implement Skylark macros specific to the target ++language. You should prefer more specific `cc_proto_library `, ++`py_proto_library` and others unless you are adding such wrapper macros. ++ ++Args: ++ srcs: Protocol Buffers definition files (.proto) to run the protocol compiler ++ against. ++ deps: a list of dependency labels; must be other proto libraries. ++ includes: a list of include paths to .proto files. ++ protoc: the label of the protocol compiler to generate the sources. ++ plugin: the label of the protocol compiler plugin to be passed to the protocol ++ compiler. ++ plugin_language: the language of the generated sources ++ plugin_options: a list of options to be passed to the plugin ++ gen_cc: generates C++ sources in addition to the ones from the plugin. ++ gen_py: generates Python sources in addition to the ones from the plugin. ++ outs: a list of labels of the expected outputs from the protocol compiler. ++""" ++ ++def py_proto_library( ++ name, ++ srcs = [], ++ deps = [], ++ py_libs = [], ++ py_extra_srcs = [], ++ include = None, ++ default_runtime = None, ++ protoc = "//third_party:protoc", ++ use_grpc_plugin = False, ++ **kwargs): ++ """Bazel rule to create a Python protobuf library from proto source files ++ ++ NOTE: the rule is only an internal workaround to generate protos. The ++ interface may change and the rule may be removed when bazel has introduced ++ the native rule. ++ ++ Args: ++ name: the name of the py_proto_library. ++ srcs: the .proto files of the py_proto_library. ++ deps: a list of dependency labels; must be py_proto_library. ++ py_libs: a list of other py_library targets depended by the generated ++ py_library. ++ py_extra_srcs: extra source files that will be added to the output ++ py_library. This attribute is used for internal bootstrapping. ++ include: a string indicating the include path of the .proto files. ++ default_runtime: the implicitly default runtime which will be depended on by ++ the generated py_library target. ++ protoc: the label of the protocol compiler to generate the sources. ++ use_grpc_plugin: a flag to indicate whether to call the Python C++ plugin ++ when processing the proto files. ++ **kwargs: other keyword arguments that are passed to py_library. ++ ++ """ ++ outs = _PyOuts(srcs, use_grpc_plugin) ++ ++ includes = [] ++ if include != None: ++ includes = [include] ++ ++ grpc_python_plugin = None ++ if use_grpc_plugin: ++ grpc_python_plugin = "//external:grpc_python_plugin" ++ # Note: Generated grpc code depends on Python grpc module. This dependency ++ # is not explicitly listed in py_libs. Instead, host system is assumed to ++ # have grpc installed. ++ ++ proto_gen( ++ name = name + "_genproto", ++ srcs = srcs, ++ deps = [s + "_genproto" for s in deps], ++ includes = includes, ++ protoc = protoc, ++ gen_py = 1, ++ outs = outs, ++ visibility = ["//visibility:public"], ++ plugin = grpc_python_plugin, ++ plugin_language = "grpc", ++ ) ++ ++ if default_runtime and not default_runtime in py_libs + deps: ++ py_libs = py_libs + [default_runtime] ++ ++ native.py_library( ++ name = name, ++ srcs = outs + py_extra_srcs, ++ deps = py_libs + deps, ++ imports = includes, ++ **kwargs ++ ) ++ diff --git a/math/py-keras/files/patch-keras_tools_pip__package_build__pip__package.sh b/math/py-keras/files/patch-keras_tools_pip__package_build__pip__package.sh new file mode 100644 index 000000000000..cdce5d8b7fce --- /dev/null +++ b/math/py-keras/files/patch-keras_tools_pip__package_build__pip__package.sh @@ -0,0 +1,20 @@ +--- keras/tools/pip_package/build_pip_package.sh.orig 2022-04-22 17:34:55 UTC ++++ keras/tools/pip_package/build_pip_package.sh +@@ -39,7 +39,7 @@ function prepare_src() { + + # Verifies all expected files are in pip. + # Creates init files in all directory in pip. +- python keras/tools/pip_package/create_pip_helper.py --pip-root "${TMPDIR}/keras/" --bazel-root "./keras" ++ %%PYTHON%% keras/tools/pip_package/create_pip_helper.py --pip-root "${TMPDIR}/keras/" --bazel-root "./keras" + } + + function build_wheel() { +@@ -54,7 +54,7 @@ function build_wheel() { + + pushd ${TMPDIR} > /dev/null + echo $(date) : "=== Building wheel" +- "${PYTHON_BIN_PATH:-python}" setup.py bdist_wheel --universal --project_name $PROJECT_NAME ++ %%PYTHON%% setup.py bdist_wheel --universal --project_name $PROJECT_NAME + mkdir -p ${DEST} + cp dist/* ${DEST} + popd > /dev/null diff --git a/math/py-keras/files/patch-third__party_BUILD b/math/py-keras/files/patch-third__party_BUILD new file mode 100644 index 000000000000..0a12377854e7 --- /dev/null +++ b/math/py-keras/files/patch-third__party_BUILD @@ -0,0 +1,19 @@ +--- third_party/BUILD.orig 2022-09-08 09:14:55 UTC ++++ third_party/BUILD +@@ -1,4 +1,5 @@ + package( ++ default_visibility = ["//visibility:public"], + licenses = ["notice"], # Apache 2.0 + ) + +@@ -7,4 +8,9 @@ exports_files([ + "six.BUILD", + ]) + +-package(default_visibility = ["//visibility:public"]) ++genrule( ++ name = "protoc", ++ outs = ["protoc.bin"], ++ cmd = "ln -s $$(which protoc) $@", ++ executable = 1, ++) diff --git a/math/py-keras/files/pkg-message.in b/math/py-keras/files/pkg-message.in deleted file mode 100644 index cd438590cdea..000000000000 --- a/math/py-keras/files/pkg-message.in +++ /dev/null @@ -1,9 +0,0 @@ -[ -{ type: install - message: <