Now Reading
Ask HN: What do you utilize for ML Internet hosting

Ask HN: What do you utilize for ML Internet hosting

2023-05-02 13:58:43

Try (I am the founder). Completely happy to assist get you began with a demo if you wish to attain out

For serverless: examine the checklist I posted right here (I ended up utilizing Banana, it was tremendous)

For non-serverless, some to take a look at are these (although seemingly all overkill for those who simply want a single GPU)

Lambda labs

Listed here are some candidates:
– HuggingFace Inference Endpoints:
– Amazon SageMaker:
– Replicate:

The primary two are extra customizable than the final. SageMaker is the most cost effective.
No person has higher costs.

I am at the moment working a Discord bot with a 7B mannequin off a free Oracle Ampere occasion with their Pytorch Accelerated[0] picture. It is not terribly quick, however completely usable for group chats that wish to interrogate an AI. When you’re performing some form of offline processing or non-time-imperative operation, one thing like this could be price wanting into.


See Also

Does that use all 4 OCPUs / 24GB reminiscence?

It could actually! I am utilizing 2 cores per request although, and I’ve acquired reminiscence to spare.

what discord bot? 🙂

Source Link

What's Your Reaction?
In Love
Not Sure
View Comments (0)

Leave a Reply

Your email address will not be published.

2022 Blinking Robots.
WordPress by Doejo

Scroll To Top