pytorch-tokenizers
Source code: https://github.com/meta-pytorch/tokenizers
pip install pytorch-tokenizers --index-url https://pypi.riseproject.dev/simple/
License: BSD-3-Clause
Download files: pytorch-tokenizers-v1.5.0-20260922100339
pip install pytorch-tokenizers==1.4.1 --index-url https://pypi.riseproject.dev/simple/
License: BSD-3-Clause
Download files: pytorch-tokenizers-v1.4.1-20260921134719