Ask HN: What do you utilize for ML Internet hosting
2023-05-02 13:58:43
 |
|
Try JetML.com (I am the founder). Completely happy to assist get you began with a demo if you wish to attain out nick@jetml.com.
|
|
|
For serverless: examine the checklist I posted right here https://news.ycombinator.com/item?id=34742087 (I ended up utilizing Banana, it was tremendous)
For non-serverless, some to take a look at are these (although seemingly all overkill for those who simply want a single GPU) huge.ai Lambda labs |
|
|
Listed here are some candidates:
– HuggingFace Inference Endpoints: https://huggingface.co/inference-endpoints – Amazon SageMaker: https://aws.amazon.com/sagemaker/ – Replicate: https://replicate.com/ The primary two are extra customizable than the final. SageMaker is the most cost effective. |
|
|
Vultr GPU: https://www.vultr.com/products/cloud-gpu/
|
|
|
Huge.ai
No person has higher costs. |
|
|
I am at the moment working a Discord bot with a 7B mannequin off a free Oracle Ampere occasion with their Pytorch Accelerated[0] picture. It is not terribly quick, however completely usable for group chats that wish to interrogate an AI. When you’re performing some form of offline processing or non-time-imperative operation, one thing like this could be price wanting into.
[0] https://cloudmarketplace.oracle.com/marketplace/en_US/adf.ta…
|
|
|
Does that use all 4 OCPUs / 24GB reminiscence?
|
|
|
It could actually! I am utilizing 2 cores per request although, and I’ve acquired reminiscence to spare.
|
|
|
what discord bot? 🙂
|
What's Your Reaction?
Excited
0
Happy
0
In Love
0
Not Sure
0
Silly
0