Now Reading
Expo Day | ML Contests

Expo Day | ML Contests

2023-12-10 21:07:21

Sunday was Expo day — that includes a combination of talks, workshops, and demonstrations.

Small Giant Language Fashions

One recurring theme all through the Expo was effectivity — (how) can we do extra with much less?

Much less energy, fewer mannequin parameters, smaller chips, …

The preferred session based mostly on attendee numbers within the convention app
(and judging by individuals being unable to get into the room, inflicting the discuss to be repeated later in a bigger room)
was a chat by Amazon titled “Smaller fashions can pack a punch within the period of huge language fashions”.
The discuss explored how
comparatively small 7-billion-parameter fashions like Mistral 7B are in a position to outperform 13B fashions,
and be aggressive in some areas in opposition to different fashions with as many as 34B parameters.

Different classes round effectivity in generative functions included Qualcomm’s demonstration of operating Steady Diffusion
on their Snapdragon cell processor; Positron AI introducing their new inference chips which purpose to function at considerably
improved efficiency/watt and efficiency/$ in comparison with the present state-of-the-art, and AMD’s discuss on enhancing power
effectivity by means of quantization.

Positron AI’s inference chip
Positron AI’s inference chip

This may stay a theme all through the remainder of the week — for instance, within the Efficient Learning
Oral Session (Tuesday 3:40-4:40pm, together with QLoRA at 3:55pm) and the LLM Efficiency Challenge (Friday 1.30pm-4.30pm),
with the latter exploring what groups can obtain by fine-tuning on a single GPU for twenty-four hours.

Different talks

Some transient highlights from different classes…

Amazon’s AutoML discuss did an important job of introducing model 1.0 of their open-source AutoGluon library.
Helpfully, the workshop has an accompanying website with numerous useful assets
together with cheatsheets and a few Colab notebooks to get began.

AutoGluon
Supply: AWS Presentation

Most of the talks had been very well-attended, and never simply those associated to generative fashions. Different notably fashionable talks
had been MathWorks’ Reinforcement Studying discuss, Google’s Graph Studying discuss, and Microsoft’s Autonomous Brokers discuss.

See Also

AstraZeneca’s talk on AI & ML Across the Entire Drug Development Pipeline
AstraZeneca’s discuss on AI & ML Throughout the Total Drug Improvement Pipeline

Tomorrow is the beginning of the principle convention, beginning with the Affinity Workshops,
adopted by the Tutorials,
the primary of the Invited Talks at 5:25pm,
a welcome reception, and the primary of the Creative AI Performances at 6:30pm.

Regulate our NeurIPS 2023 web page for extra day by day blogs because the week goes on.

Source Link

What's Your Reaction?
Excited
0
Happy
0
In Love
0
Not Sure
0
Silly
0
View Comments (0)

Leave a Reply

Your email address will not be published.

2022 Blinking Robots.
WordPress by Doejo

Scroll To Top