Petals – Decentralized platform for working 100B+ language fashions

Run 100B+ language fashions at house, BitTorrent‑fashion

• Run giant language fashions like BLOOM-176B
  collaboratively — you load a small a part of the mannequin, then workforce up with folks serving the opposite elements
  to run inference or fine-tuning.
• Single-batch inference runs at ≈ 1 sec per step (token) —
  up to 10x faster than offloading, sufficient for
  chatbots and different interactive apps.
  Parallel inference reaches tons of of tokens/sec.
• Past basic language mannequin APIs —
  you may make use of any fine-tuning and sampling strategies, execute customized paths by the mannequin, or see its hidden states.
  You get the comforts of an API with the pliability of PyTorch.

Featured on:

See Also

Folklore.org: Scrooge McDuck

This venture is part of the BigScience analysis workshop.