Overview
Hugging Face encoder inference on Apple Silicon using float16 weights, with state_dict caching, streaming, and a CLI. Halves memory on every model. Speed depends on model size: about 2.3x faster on distilbert-base-uncased, roughly a third slower on models small enough to be dominated by dispatch overhead. All measurements are CPU; MPS runs but is unmeasured.
Installation
git clone https://github.com/coccinella-labs/bitinfer