You are now in the directory for the port "textproc/py-tokenizers".
The package name of this port is "py312-tokenizers-0.23.1_2".
The license for this port is "%%LICENSE%%".
This is the one-line description for this port:
Fast state-of-the-art tokenizers optimized for research and production
Please read the "description file" for a longer description.
If needed, you may contact the maintainer of this port or the port mailing-list.
This port requires package(s) "brotli-1.2.0,1 curl-8.23.0 gettext-runtime-0.23.1 indexinfo-0.3.1_1 libffi-3.7.1 libiconv-1.18 libidn2-2.3.8_1 liblz4-1.10.0_1,1 libnghttp2-1.68.0 libpsl-0.21.5_1 libunistring-1.4.2 mpdecimal-4.0.0 ncurses-6.5 oniguruma-6.9.8 p5-Locale-libintl-1.37 p5-Text-Unidecode-1.30 p5-Unicode-EastAsianWidth-12.0 perl5-5.40.5 pkgconf-2.4.3,1 py311-build-1.4.0_1 py312-pyproject-hooks-1.2.0_1 py313-installer-1.0.0_2 py313-packaging-26.2 py313t-maturin-1.13.3_1 python311-3.11.14_2 python312-3.12.12_1 python313-3.13.11_1 python313t-3.13.11_1 readline-8.2.10 rust-1.95.0 texinfo-7.1_2,1 wolfssl-5.9.1 zstd-1.5.7" to build.
This port requires package(s) "gettext-runtime-0.23.1 indexinfo-0.3.1_1 libffi-3.7.1 libiconv-1.18 liblz4-1.10.0_1,1 libyaml-0.2.5 mpdecimal-4.0.0 ncurses-6.5 oniguruma-6.9.8 py310-h11-0.16.0_1 py310-pyyaml-6.0.3_2 py310-truststore-0.10.4_1 py311-httpcore-1.0.9_1 py312-filelock-3.13.3_1 py312-fsspec-2024.10.0_1 py312-hf-xet-1.3.0_1 py312-huggingface-hub-1.4.1_1 py312-idna-3.19 py312-shellingham-1.5.4_2 py312-socksio-1.0.0_2 py312-tqdm-4.66.2_1 py312-typer-0.21.1_1 py313-certifi-2026.4.22 py313-click-8.3.3_1 py313-httpx-0.28.1_3 py313-markdown-it-py-4.2.0 py313-packaging-26.2 py313-pygments-2.20.0_1 py313t-anyio-4.13.0_1 py313t-h2-4.1.0_1 py313t-hpack-4.0.0_1 py313t-hyperframe-6.0.0_1 py313t-mdurl-0.1.2_1 py313t-typing-extensions-4.15.0_1 py314-rich-15.0.0_1 python310-3.10.19_1 python311-3.11.14_2 python312-3.12.12_1 python313-3.13.11_1 python313t-3.13.11_1 python314-3.14.7_1 readline-8.2.10 zstd-1.5.7" to run.
Go to the top of the ports tree for a summary on how to use the ports collection.