Now Reading
Petals – Decentralized platform for working 100B+ language fashions

Petals – Decentralized platform for working 100B+ language fashions

2023-03-20 17:50:03

Run 100B+ language fashions at house, BitTorrent‑fashion

  • Run giant language fashions like BLOOM-176B
    collaboratively — you load a small a part of the mannequin, then workforce up with folks serving the opposite elements
    to run inference or fine-tuning.
  • Single-batch inference runs at ≈ 1 sec per step (token) —
    up to 10x faster than offloading, sufficient for
    chatbots and different interactive apps.
    Parallel inference reaches tons of of tokens/sec.
  • Past basic language mannequin APIs —
    you may make use of any fine-tuning and sampling strategies, execute customized paths by the mannequin, or see its hidden states.
    You get the comforts of an API with the pliability of PyTorch.

Be a part of our Discord
or subscribe through e mail
to observe Petals growth:


We despatched you an e mail to verify your tackle. Click on it and also you’re in!

Featured on:

See Also

This venture is part of the BigScience analysis workshop.

Source Link

What's Your Reaction?
In Love
Not Sure
View Comments (0)

Leave a Reply

Your email address will not be published.

2022 Blinking Robots.
WordPress by Doejo

Scroll To Top