the 100 most cited AI papers in 2022
Who Is publishing essentially the most Impactful AI analysis proper now? With the breakneck tempo of innovation in AI, it’s essential to choose up some sign as quickly as potential. Nobody has the time to learn all the things, however these 100 papers are certain to bend the highway as to the place our AI know-how goes. The true take a look at of influence of R&D groups is after all how the know-how seems in merchandise, and OpenAI shook the world by releasing ChatGPT on the finish of November 2022, following quick on their March 2022 paper “Coaching language fashions to comply with directions with human suggestions”. Such quick product adoption is uncommon, so to see a bit additional, we have a look at a traditional educational metric: the variety of citations. An in depth evaluation of the 100 most cited papers per 12 months, for 2022, 2021, and 2020 permits us to attract some early conclusions. America and Google nonetheless dominate, and DeepMind has had a stellar 12 months of success, however given its quantity of output, OpenAI is basically in a league of its personal each in product influence, and in analysis that turns into rapidly and broadly cited. The complete top-100 checklist for 2022 is included under on this publish.
Determine 1. Supply: Zeta Alpha
Utilizing information from the Zeta Alpha platform mixed with cautious human curation (extra about methodology under), we have gathered the highest cited papers in AI from 2022, 2021, and 2020, and analyzed authors’ affiliations, and nation. This permits us to rank these by R&D influence reasonably than pure publication quantity.
What are a few of these prime papers we’re speaking about?
However earlier than we dive into the numbers, let’s get a way of what papers we’re speaking about: the blockbusters from these previous 3 years. You may most likely acknowledge just a few of them!
2022
1️⃣ AlphaFold Protein Structure Database: massively expanding the structural coverage of protein-sequence space with high-accuracy models -> (From DeepMind, 1372 citations) Utilizing AlphaFold to reinforce protein construction database protection.
2️⃣ ColabFold: making protein folding accessible to all -> (From a number of establishments, 1162 citations) An open-source and environment friendly protein folding mannequin.
3️⃣ Hierarchical Text-Conditional Image Generation with CLIP Latents -> (From OpenAI, 718 citations) DALL·E 2, complicated prompted picture technology that left most in awe.
4️⃣ A ConvNet for the 2020s -> (From Meta and UC Berkeley, 690 citations) A profitable modernization of CNNs at a time of increase for Transformers in Pc Imaginative and prescient.
5️⃣ PaLM: Scaling Language Modeling with Pathways -> (From Google, 452 citations) Google’s mammoth 540B Massive Language Mannequin, a brand new MLOps infrastructure, and the way it performs.
2021
1️⃣ Highly accurate protein structure prediction with AlphaFold -> (From DeepMind, 8965) AlphaFold, a breakthrough in protein construction prediction utilizing Deep Studying.
2️⃣ Swin Transformer: Hierarchical Vision Transformer using Shifted Windows -> (From Microsoft, 4810 citations) A sturdy variant of Transformers for Imaginative and prescient.
3️⃣ Learning Transferable Visual Models From Natural Language Supervision -> (From OpenAI, 3204 citations) CLIP, image-text pairs at scale to be taught joint image-text representations in a self supervised trend
4️⃣ On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? -> (From U. Washington, Black in AI, The Aether, 1266 citations) Well-known place paper very essential of the development of ever-growing language fashions, highlighting their limitations and risks.
5️⃣ Emerging Properties in Self-Supervised Vision Transformers -> (From Meta, 1219 citations) DINO, exhibiting how self-supervision on pictures led to the emergence of some type of proto-object segmentation in Transformers.
2020
1️⃣ An Image is Worth 16×16 Words: Transformers for Image Recognition at Scale -> (From Google, 11914 citations) The primary work exhibiting how a plain Transformer may do nice in Pc Imaginative and prescient.
2️⃣ Language Models are Few-Shot Learners -> (From OpenAI, 8070 citations) GPT-3, This paper doesn’t want additional clarification at this stage.
3️⃣ YOLOv4: Optimal Speed and Accuracy of Object Detection -> (From Academia Sinica, Taiwan, 8014 citations) Sturdy and quick object detection sells like hotcakes.
4️⃣ Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer -> (From Google, 5906 citations) A rigorous examine of switch studying with Transformers, ensuing within the well-known T5.
5️⃣ Bootstrap your own latent: A new approach to self-supervised Learning -> (From DeepMind and Imperial Faculty, 2873 citations) Displaying that negatives usually are not even vital for illustration studying.
Learn on under to see the total checklist of 100 papers for 2022, however let’s first dive into the analyses for international locations and establishments.
Essentially the most cited papers from the previous 3 years
After we have a look at the place these top-cited papers come from (Determine 1), we see that the US continues to dominate and the distinction among the many main powers varies solely barely per 12 months. Earlier reports that China may have overtaken the US in AI R&D appear to be extremely exaggerated if we have a look at it from the attitude of citations. We additionally see an influence considerably above expectation from Singapore and Australia.
To correctly assess the US dominance, let’s look past paper depend numbers. If we take into account the collected citations by nation as an alternative, the distinction seems even stronger. We’ve normalized by the full variety of citations in a 12 months, so as to have the ability to evaluate meaningfully throughout years.
Determine 2. Supply: Zeta Alpha
The UK is clearly the strongest participant exterior of the US and China. Nevertheless, the contribution of the UK is much more strongly dominated by DeepMind in 2022 (69% of the UK whole), than within the earlier years (60%). DeepMind has really had a really productive 2022.
Now let us take a look at how the main organizations evaluate by variety of papers within the prime 100.
Determine 3. Supply: Zeta Alpha
Google is persistently the strongest participant adopted by Meta, Microsoft, UC Berkeley, DeepMind and Stanford. Whereas trade calls the photographs in AI analysis lately, and single educational establishments do not produce as a lot influence, the tail for these establishments is for much longer, in order that after we combination by group sort, it evens out.
Determine 4. Supply: Zeta Alpha
If we glance into whole analysis output, what number of papers have organizations printed in these previous 3 years?
Determine 5. Supply: Zeta Alpha
In whole publication quantity, Google remains to be within the lead, however variations are a lot much less drastic in comparison with the quotation prime 100. You will not see OpenAI or DeepMind among the many prime 20 within the quantity of publications. These establishments publish much less however with increased influence. The next chart reveals the speed at which organizations handle to transform their publications into top-100 papers.
Determine 6. Supply: Zeta Alpha
Now we see that OpenAI is just in a league of its personal in relation to turning publications into absolute blockbusters. Whereas definitely their advertising magic helps quite a bit to propel their recognition, it is plain that a few of their current analysis is of excellent high quality.
The highest 100 most cited papers for 2022
And at last, right here is our top-100 checklist itself, with titles, quotation counts, and affiliations.
We’ve additionally added twitter mentions, that are generally seen as an early influence indicator, nevertheless the correlation to this point appears to be weak. Additional work is required.
Methodology
To create the evaluation above, we now have first collected essentially the most cited papers per 12 months within the Zeta Alpha platform, after which manually checked the primary publication date (often an arXiv pre-print), in order that we place papers in the precise 12 months. We supplemented this checklist by mining for extremely cited AI papers on Semantic Scholar with its broader protection and skill to kind by quotation depend. This primarily turns up further papers from extremely impactful closed supply publishers (e.g. Nature, Elsevier, Springer and different journals). We then take for every paper the variety of citations on Google Scholar because the consultant metric, and type the papers by this quantity to yield the top-100 for a 12 months. For these papers we used GPT-3 to extract the authors, their affiliations and their nation and manually checked these outcomes (if the nation was not clearly seen from the publication, we take the nation of the group’s headquarters). A paper with authors from a number of affiliations counts as soon as for every of the affiliations.
This concludes our evaluation; what shocked you essentially the most about these numbers? Comply with us on Twitter @zetavector and tell us when you’ve got any suggestions or want to obtain a extra detailed evaluation on your area or group.