Replit’s new AI Mannequin now out there on Hugging Face
At Replit, our mission is to empower the subsequent billion software program creators. Yesterday, we strengthened our dedication by asserting that Replit AI is now free for all customers. Over the previous yr, we’ve witnessed the transformative energy of constructing software program collaboratively with the ability of AI. We consider AI shall be a part of each software program developer’s toolkit and we’re excited to offer Replit AI without spending a dime to our 25+ million developer group.
To accompany AI for all, we’re releasing our new code era language mannequin Replit Code V1.5 3B on Hugging Face. We consider in open supply language fashions – anybody can use it as a foundational mannequin for application-specific fine-tuning with out strict limitations on business use.
Key Options
- Intensive Permissively Licensed Coaching Knowledge: Educated on 1 trillion tokens of code from permissively licensed code from the Stack dataset and publicly out there dev-oriented content material from StackExchange.
- State of the Artwork Outcomes: Main HumanEval and Multi-PLe analysis scores for a 3B code completion mannequin.
- Broad Multi-Language Help: Encompasses Replit’s high 30 programming languages with a customized skilled 32K vocabulary for top efficiency and protection.
- Newest Strategies: Constructed with all newest strategies akin to Grouped Question Consideration with Flash Consideration Triton Kernels, ALiBi positional embeddings, and extra, for low latency and excessive era high quality. Educated with the most recent strategies like LionW optimizer, studying fee cooling down, QKV clipping and extra.
- Excessive High quality Curated Coaching Knowledge: Integrated specialised code high quality filtering heuristics, parsability checking, poisonous and profane content material removing that result in larger high quality generations.
This is how the mannequin carried out towards main benchmarks:
When fine-tuned on public Replit person code, the mannequin outperforms fashions of a lot bigger measurement akin to CodeLlama7B:
The mannequin is meant for use by anybody as a basis for application-specific fine-tuning and is skilled particularly for code completion duties.
Methods to use Replit V1.5 3B
The mannequin is skilled particularly for code completion duties. You’ll be able to generate code utilizing the transformers library as follows:
from transformers import AutoModelForCausalLM, AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained('replit/replit-code-v1_5-3b', trust_remote_code=True)
mannequin = AutoModelForCausalLM.from_pretrained('replit/replit-code-v1_5-3b', trust_remote_code=True)
x = tokenizer.encode('def fibonacci(n): ', return_tensors='pt')
y = mannequin.generate(x, max_length=100, do_sample=True, top_p=0.95, top_k=4, temperature=0.2, num_return_sequences=1, eos_token_id=tokenizer.eos_token_id)
generated_code = tokenizer.decode(y[0], skip_special_tokens=True, clean_up_tokenization_spaces=False)
print(generated_code)
Take a look at our Hugging Face README for extra particulars on the way to use GPUs with optimized the Triton kernel that our mannequin helps.
Experiment with totally different decoding strategies and parameters to get the very best outcomes to your use case.
For a deeper dive, take a look at the guide on training your own LLMs that we printed alongside the primary iteration of this mannequin. Keep tuned for a technical deepdive from a member of our AI workforce the place we’ll focus on our effective tuning course of and challenges alongside the way in which.