default search action
Google
Google Scholar
Semantic Scholar
Internet Archive Scholar
CiteSeerX
ORCIDMostofa Patwary
Person information
Refine list
refinements active!
zoomed in on ?? of ?? records
view refined list in
showing all ?? records
2020 – today
- 2026
[j1]Tao Feng, Haozhen Zhang, Zijie Lei, Pengrui Han, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro, Jiaxuan You:
FusionFactory: Fusing LLM Capabilities with Multi-LLM Log Data. Trans. Mach. Learn. Res. 2026 (2026)- [c22]Syeda Nahida Akter, Shrimai Prabhumoye, Matvei Novikov, Seungju Han, Ying Lin, Evelina Bakhturina, Eric Nyberg, Yejin Choi, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro:
Nemotron-CrossThink: Scaling Self-Learning beyond Math Reasoning. EACL (Volume 1: Long Papers) 2026: 984-1002 - [i36]Venmugil Elango, Nidhi Bhatia, Roger Waleffe, Rasoul Shafipour, Tomer Asida, Abhinav Khattar, Nave Assaf, Maximilian Golub, Joey Guman, Tiyasa Mitra, Ritchie Zhao, Ritika Borkar, Ran Zilberstein, Mostofa Patwary, Mohammad Shoeybi, Bita Rouhani:
LatentMoE: Toward Optimal Accuracy per FLOP and Parameter in Mixture of Experts. CoRR abs/2601.18089 (2026) - [i35]Fitsum A. Reda, John Kamalu, Roger Waleffe, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro:
Nemotron-Labs-TwoTower: Diffusion Language Modeling with Pretrained Autoregressive Context. CoRR abs/2606.26493 (2026) - 2025
- [c21]Dan Su, Kezhi Kong, Ying Lin, Joseph Jennings, Brandon Norick, Markus Kliegl, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro:
Nemotron-CC: Transforming Common Crawl into a Refined Long-Horizon Pretraining Dataset. ACL (1) 2025: 2459-2475 - [c20]Syeda Nahida Akter, Shrimai Prabhumoye, John Kamalu, Sanjeev Satheesh, Eric Nyberg, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro:
MIND: Math Informed syNthetic Dialogues for Pretraining LLMs. ICLR 2025 - [c19]Jaehun Jung, Seungju Han, Ximing Lu, Skyler Hallinan, David Acuna, Shrimai Prabhumoye, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro, Yejin Choi:
Prismatic Synthesis: Gradient-based Data Diversification Boosts Generalization in LLM Reasoning. NeurIPS 2025 - [c18]Ali Taghibakhshi, Sharath Turuvekere Sreenivas, Saurav Muralidharan, Marcin Chochowski, Yashaswi Karnati, Raviraj Joshi, Ameya Mahabaleshwarkar, Zijia Chen, Yoshi Suhara, Oluwatobi Olabiyi, Daniel Korzekwa, Mostofa Patwary, Mohammad Shoeybi, Jan Kautz, Bryan Catanzaro, Ashwath Aithal, Nima Tajbakhsh, Pavlo Molchanov:
Efficient Hybrid Language Model Compression through Group-Aware SSM Pruning. NeurIPS 2025 - [i34]Ximing Lu, Seungju Han, David Acuna, Hyunwoo Kim, Jaehun Jung, Shrimai Prabhumoye, Niklas Muennighoff, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro, Yejin Choi:
Retro-Search: Exploring Untaken Paths for Deeper and Efficient Reasoning. CoRR abs/2504.04383 (2025) - [i33]Ali Taghibakhshi, Sharath Turuvekere Sreenivas, Saurav Muralidharan, Marcin Chochowski, Yashaswi Karnati, Raviraj Joshi, Ameya Sunil Mahabaleshwarkar, Zijia Chen, Yoshi Suhara, Oluwatobi Olabiyi, Daniel Korzekwa, Mostofa Patwary, Mohammad Shoeybi, Jan Kautz, Bryan Catanzaro, Ashwath Aithal, Nima Tajbakhsh, Pavlo Molchanov:
Efficient Hybrid Language Model Compression through Group-Aware SSM Pruning. CoRR abs/2504.11409 (2025) - [i32]Shizhe Diao, Yu Yang, Yonggan Fu, Xin Dong, Dan Su, Markus Kliegl, Zijia Chen, Peter Belcak, Yoshi Suhara, Hongxu Yin, Mostofa Patwary, Yingyan Lin, Jan Kautz, Pavlo Molchanov:
CLIMB: CLustering-based Iterative Data Mixture Bootstrapping for Language Model Pre-training. CoRR abs/2504.13161 (2025) - [i31]Syeda Nahida Akter, Shrimai Prabhumoye, Matvei Novikov, Seungju Han, Ying Lin, Evelina Bakhturina, Eric Nyberg, Yejin Choi, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro:
Nemotron-CrossThink: Scaling Self-Learning beyond Math Reasoning. CoRR abs/2504.13941 (2025) - [i30]Tao Feng, Haozhen Zhang, Zijie Lei, Pengrui Han, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro, Jiaxuan You:
Fusing LLM Capabilities with Routing Data. CoRR abs/2507.10540 (2025) - [i29]Rabeeh Karimi Mahabadi, Sanjeev Satheesh, Shrimai Prabhumoye, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro:
Nemotron-CC-Math: A 133 Billion-Token-Scale High Quality Math Pretraining Dataset. CoRR abs/2508.15096 (2025) - [i28]Ali Hatamizadeh, Syeda Nahida Akter, Shrimai Prabhumoye, Jan Kautz, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro, Yejin Choi:
RLP: Reinforcement as a Pretraining Objective. CoRR abs/2510.01265 (2025) - [i27]Syeda Nahida Akter, Shrimai Prabhumoye, Eric Nyberg, Mostofa Patwary, Mohammad Shoeybi, Yejin Choi, Bryan Catanzaro:
Front-Loading Reasoning: The Synergy between Pretraining and Post-Training Data. CoRR abs/2510.03264 (2025) - [i26]Yixing Chen, Yiding Wang, Siqi Zhu
, Haofei Yu, Tao Feng, Muhan Zhang, Mostofa Patwary, Jiaxuan You:
Multi-Agent Evolve: LLM Self-Improve through Co-evolution. CoRR abs/2510.23595 (2025) - [i25]Ali Taghibakhshi, Sharath Turuvekere Sreenivas, Saurav Muralidharan, Ruisi Cai, Marcin Chochowski, Ameya Sunil Mahabaleshwarkar, Yoshi Suhara, Oluwatobi Olabiyi, Daniel Korzekwa, Mostofa Patwary, Mohammad Shoeybi, Jan Kautz, Bryan Catanzaro, Ashwath Aithal, Nima Tajbakhsh, Pavlo Molchanov:
Nemotron Elastic: Towards Efficient Many-in-One Reasoning LLMs. CoRR abs/2511.16664 (2025) - 2024
- [c17]Jupinder Parmar, Shrimai Prabhumoye, Joseph Jennings, Bo Liu, Aastha Jhunjhunwala, Zhilin Wang, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro:
Data, Data Everywhere: A Guide for Pretraining Dataset Construction. EMNLP 2024: 10671-10695 - [c16]Jiaxuan You, Mingjie Liu, Shrimai Prabhumoye, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro:
LLM-Evolve: Evaluation for LLM's Evolving Capability on Benchmarks. EMNLP 2024: 16937-16942 - [c15]Saurav Muralidharan, Sharath Turuvekere Sreenivas, Raviraj Joshi, Marcin Chochowski, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro, Jan Kautz, Pavlo Molchanov:
Compact Language Models via Pruning and Knowledge Distillation. NeurIPS 2024 - [i24]Jupinder Parmar, Shrimai Prabhumoye, Joseph Jennings, Mostofa Patwary, Sandeep Subramanian, Dan Su, Chen Zhu, Deepak Narayanan, Aastha Jhunjhunwala, Ayush Dattagupta, Vibhu Jawa, Jiwei Liu, Ameya Mahabaleshwarkar, Osvald Nitski, Annika Brundyn, James Maki, Miguel Martinez, Jiaxuan You, John Kamalu, Patrick LeGresley, Denys Fridman, Jared Casper, Ashwath Aithal, Oleksii Kuchaiev, Mohammad Shoeybi, Jonathan M. Cohen, Bryan Catanzaro:
Nemotron-4 15B Technical Report. CoRR abs/2402.16819 (2024) - [i23]Jupinder Parmar, Shrimai Prabhumoye, Joseph Jennings, Bo Li, Aastha Jhunjhunwala, Zhilin Wang, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro:
Data, Data Everywhere: A Guide for Pretraining Dataset Construction. CoRR abs/2407.06380 (2024) - [i22]Jupinder Parmar, Sanjeev Satheesh, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro:
Reuse, Don't Retrain: A Recipe for Continued Pretraining of Language Models. CoRR abs/2407.07263 (2024) - [i21]Saurav Muralidharan, Sharath Turuvekere Sreenivas, Raviraj Joshi, Marcin Chochowski, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro, Jan Kautz, Pavlo Molchanov:
Compact Language Models via Pruning and Knowledge Distillation. CoRR abs/2407.14679 (2024) - [i20]Sharath Turuvekere Sreenivas, Saurav Muralidharan, Raviraj Joshi, Marcin Chochowski, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro, Jan Kautz, Pavlo Molchanov:
LLM Pruning and Distillation in Practice: The Minitron Approach. CoRR abs/2408.11796 (2024) - [i19]Syeda Nahida Akter, Shrimai Prabhumoye, John Kamalu, Sanjeev Satheesh, Eric Nyberg, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro:
MIND: Math Informed syNthetic Dialogues for Pretraining LLMs. CoRR abs/2410.12881 (2024) - [i18]Dan Su, Kezhi Kong, Ying Lin, Joseph Jennings, Brandon Norick, Markus Kliegl, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro:
Nemotron-CC: Transforming Common Crawl into a Refined Long-Horizon Pretraining Dataset. CoRR abs/2412.02595 (2024) - [i17]Steven Feng, Shrimai Prabhumoye, Kezhi Kong, Dan Su, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro:
Maximize Your Data's Potential: Enhancing LLM Accuracy with Two-Phase Pretraining. CoRR abs/2412.15285 (2024) - 2023
- [c14]Dan Su, Mostofa Patwary, Shrimai Prabhumoye, Peng Xu, Ryan Prenger, Mohammad Shoeybi, Pascale Fung, Anima Anandkumar, Bryan Catanzaro:
Context Generation Improves Open Domain Question Answering. EACL (Findings) 2023: 781-796 - [c13]Shrimai Prabhumoye, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro:
Adding Instructions during Pretraining: Effective way of Controlling Toxicity in Language Models. EACL 2023: 2628-2643 - [i16]Shrimai Prabhumoye, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro:
Adding Instructions during Pretraining: Effective Way of Controlling Toxicity in Language Models. CoRR abs/2302.07388 (2023) - 2022
- [c12]Zihan Liu, Mostofa Patwary, Ryan Prenger, Shrimai Prabhumoye, Wei Ping, Mohammad Shoeybi, Bryan Catanzaro:
Multi-Stage Prompting for Knowledgeable Dialogue Generation. ACL (Findings) 2022: 1317-1337 - [c11]Peng Xu, Mostofa Patwary, Shrimai Prabhumoye, Virginia Adams, Ryan Prenger, Wei Ping, Nayeon Lee, Mohammad Shoeybi, Bryan Catanzaro:
Evaluating Parameter Efficient Learning for Generation. EMNLP 2022: 4824-4833 - [c10]Mostofa Patwary:
Keynote Talk 2 Training Large Language Models: Challenges and Opportunities. IPDPS Workshops 2022: 1245 - [c9]Nayeon Lee, Wei Ping, Peng Xu, Mostofa Patwary, Pascale Fung, Mohammad Shoeybi, Bryan Catanzaro:
Factuality Enhanced Language Models for Open-Ended Text Generation. NeurIPS 2022 - [c8]Boxin Wang, Wei Ping, Chaowei Xiao, Peng Xu, Mostofa Patwary, Mohammad Shoeybi, Bo Li, Anima Anandkumar, Bryan Catanzaro:
Exploring the Limits of Domain-Adaptive Training for Detoxifying Large-Scale Language Models. NeurIPS 2022 - [i15]Shaden Smith, Mostofa Patwary, Brandon Norick, Patrick LeGresley, Samyam Rajbhandari, Jared Casper, Zhun Liu, Shrimai Prabhumoye, George Zerveas, Vijay Korthikanti, Elton Zheng, Rewon Child, Reza Yazdani Aminabadi, Julie Bernauer, Xia Song, Mohammad Shoeybi, Yuxiong He, Michael Houston, Saurabh Tiwary, Bryan Catanzaro:
Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model. CoRR abs/2201.11990 (2022) - [i14]Boxin Wang, Wei Ping, Chaowei Xiao, Peng Xu, Mostofa Patwary, Mohammad Shoeybi, Bo Li, Anima Anandkumar, Bryan Catanzaro:
Exploring the Limits of Domain-Adaptive Training for Detoxifying Large-Scale Language Models. CoRR abs/2202.04173 (2022) - [i13]Zihan Liu, Mostofa Patwary, Ryan Prenger, Shrimai Prabhumoye, Wei Ping, Mohammad Shoeybi, Bryan Catanzaro:
Multi-Stage Prompting for Knowledgeable Dialogue Generation. CoRR abs/2203.08745 (2022) - [i12]Nayeon Lee, Wei Ping, Peng Xu, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro:
Factuality Enhanced Language Models for Open-Ended Text Generation. CoRR abs/2206.04624 (2022) - [i11]Dan Su, Mostofa Patwary, Shrimai Prabhumoye, Peng Xu, Ryan Prenger, Mohammad Shoeybi, Pascale Fung, Anima Anandkumar, Bryan Catanzaro:
Context Generation Improves Open Domain Question Answering. CoRR abs/2210.06349 (2022) - [i10]Peng Xu, Mostofa Patwary, Shrimai Prabhumoye, Virginia Adams, Ryan J. Prenger, Wei Ping, Nayeon Lee, Mohammad Shoeybi, Bryan Catanzaro:
Evaluating Parameter Efficient Learning for Generation. CoRR abs/2210.13673 (2022) - 2021
- [c7]Devendra Singh Sachan, Mostofa Patwary, Mohammad Shoeybi, Neel Kant, Wei Ping, William L. Hamilton, Bryan Catanzaro:
End-to-End Training of Neural Retrievers for Open-Domain Question Answering. ACL/IJCNLP (1) 2021: 6648-6662 - [c6]Deepak Narayanan, Mohammad Shoeybi, Jared Casper, Patrick LeGresley, Mostofa Patwary, Vijay Korthikanti, Dmitri Vainbrand, Prethvi Kashinkunti, Julie Bernauer, Bryan Catanzaro, Amar Phanishayee, Matei Zaharia:
Efficient large-scale language model training on GPU clusters using megatron-LM. SC 2021: 58 - [i9]Devendra Singh Sachan, Mostofa Patwary, Mohammad Shoeybi, Neel Kant, Wei Ping, William L. Hamilton, Bryan Catanzaro:
End-to-End Training of Neural Retrievers for Open-Domain Question Answering. CoRR abs/2101.00408 (2021) - [i8]Deepak Narayanan, Mohammad Shoeybi, Jared Casper, Patrick LeGresley, Mostofa Patwary, Vijay Korthikanti, Dmitri Vainbrand, Prethvi Kashinkunti, Julie Bernauer, Bryan Catanzaro, Amar Phanishayee, Matei Zaharia:
Efficient Large-Scale Language Model Training on GPU Clusters. CoRR abs/2104.04473 (2021) - 2020
- [c5]Alex Boyd, Raul Puri, Mohammad Shoeybi, Mostofa Patwary, Bryan Catanzaro:
Large Scale Multi-Actor Generative Dialog Modeling. ACL 2020: 66-84 - [c4]Peng Xu, Mostofa Patwary, Mohammad Shoeybi, Raul Puri, Pascale Fung, Anima Anandkumar, Bryan Catanzaro:
MEGATRON-CNTRL: Controllable Story Generation with External Knowledge Using Large-Scale Language Models. EMNLP (1) 2020: 2831-2845 - [c3]Hoo-Chang Shin, Yang Zhang, Evelina Bakhturina, Raul Puri, Mostofa Patwary, Mohammad Shoeybi, Raghav Mani:
BioMegatron: Larger Biomedical Domain Language Model. EMNLP (1) 2020: 4700-4706 - [c2]Raul Puri, Ryan Spring, Mohammad Shoeybi, Mostofa Patwary, Bryan Catanzaro:
Training Question Answering Models From Synthetic Data. EMNLP (1) 2020: 5811-5826 - [i7]Raul Puri, Ryan Spring, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro:
Training Question Answering Models From Synthetic Data. CoRR abs/2002.09599 (2020) - [i6]Alex Boyd, Raul Puri, Mohammad Shoeybi, Mostofa Patwary, Bryan Catanzaro:
Large Scale Multi-Actor Generative Dialog Modeling. CoRR abs/2005.06114 (2020) - [i5]Peng Xu, Mostofa Patwary, Mohammad Shoeybi, Raul Puri, Pascale Fung, Anima Anandkumar, Bryan Catanzaro:
MEGATRON-CNTRL: Controllable Story Generation with External Knowledge Using Large-Scale Language Models. CoRR abs/2010.00840 (2020) - [i4]Hoo-Chang Shin, Yang Zhang, Evelina Bakhturina, Raul Puri, Mostofa Patwary, Mohammad Shoeybi, Raghav Mani:
BioMegatron: Larger Biomedical Domain Language Model. CoRR abs/2010.06060 (2020) - [i3]Sashank Santhanam, Wei Ping, Raul Puri, Mohammad Shoeybi, Mostofa Patwary, Bryan Catanzaro:
Local Knowledge Powered Conversational Agents. CoRR abs/2010.10150 (2020)
2010 – 2019
- 2019
- [c1]Adam Rupe, Prabhat, James P. Crutchfield, Nalini Kumar, Vladislav Epifanov, Karthik Kashinath, Oleksandr Pavlyk, Frank Schlimbach, Mostofa Patwary, Sergey Maidanov, Victor W. Lee:
DisCo: Physics-Based Unsupervised Discovery of Coherent Structures in Spatiotemporal Systems. MLHPC@SC 2019: 75-87 - [i2]Mohammad Shoeybi, Mostofa Patwary, Raul Puri, Patrick LeGresley, Jared Casper, Bryan Catanzaro:
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism. CoRR abs/1909.08053 (2019) - [i1]Adam Rupe, Nalini Kumar, Vladislav Epifanov, Karthik Kashinath, Oleksandr Pavlyk, Frank Schlimbach, Mostofa Patwary, Sergey Maidanov, Victor W. Lee, Prabhat, James P. Crutchfield:
DisCo: Physics-Based Unsupervised Discovery of Coherent Structures in Spatiotemporal Systems. CoRR abs/1909.11822 (2019)
Coauthor Index
[j1] [c22] [i36] [i35] [c21] [c20] [c19] [c18] [i34] [i33] [i31] [i30] [i29] [i28] [i27] [i25] [c17] [c16] [c15] [i24] [i23] [i22] [i21] [i20] [i19] [i18] [i17] [c14] [c13] [i16] [c12] [c11] [c9] [c8] [i15] [i14] [i13] [i12] [i11] [i10] [c7] [c6] [i9] [i8] [c5] [c4] [c3] [c2] [i7] [i6] [i5] [i4] [i3] [i2]
manage site settings
To protect your privacy, all features that rely on external API calls from your browser are turned off by default. You need to opt-in for them to become active. All settings here will be stored as cookies with your web browser. For more information see our F.A.Q.
Unpaywalled article links
Add open access links from 
to the list of external document links (if available).
Privacy notice: By enabling the option above, your browser will contact the API of unpaywall.org to load hyperlinks to open access articles. Although we do not have any reason to believe that your call will be tracked, we do not have any control over how the remote server uses your data. So please proceed with care and consider checking the Unpaywall privacy policy.
Archived links via Wayback Machine
For web page which are no longer available, try to retrieve content from the 
of the Internet Archive (if available).
Privacy notice: By enabling the option above, your browser will contact the API of archive.org to check for archived content of web pages that are no longer available. Although we do not have any reason to believe that your call will be tracked, we do not have any control over how the remote server uses your data. So please proceed with care and consider checking the Internet Archive privacy policy.
Reference lists
Add a list of references from 
, 
, and 
to record detail pages.
load references from crossref.org and opencitations.net
Privacy notice: By enabling the option above, your browser will contact the APIs of crossref.org, opencitations.net, and semanticscholar.org to load article reference information. Although we do not have any reason to believe that your call will be tracked, we do not have any control over how the remote server uses your data. So please proceed with care and consider checking the Crossref privacy policy and the OpenCitations privacy policy, as well as the AI2 Privacy Policy covering Semantic Scholar.
Citation data
Add a list of citing articles from and to record detail pages.
load citations from opencitations.net
Privacy notice: By enabling the option above, your browser will contact the API of opencitations.net and semanticscholar.org to load citation information. Although we do not have any reason to believe that your call will be tracked, we do not have any control over how the remote server uses your data. So please proceed with care and consider checking the OpenCitations privacy policy as well as the AI2 Privacy Policy covering Semantic Scholar.
OpenAlex data
Load additional information about publications from 
.
Privacy notice: By enabling the option above, your browser will contact the API of openalex.org to load additional information. Although we do not have any reason to believe that your call will be tracked, we do not have any control over how the remote server uses your data. So please proceed with care and consider checking the information given by OpenAlex.
last updated on 2026-07-15 01:05 CEST by the dblp team
all metadata released as open data under CC0 1.0 license
see also: Terms of Use | Privacy Policy | Imprint
dblp was originally created in 1993 at:
since 2018, dblp has been operated, funded, and maintained by:
Schloss Dagstuhl - Leibniz Center for Informatics is a member of the Leibniz Association and is funded by the federal government of Germany and the states of Germany, especially Saarland and Rhineland‑Palatinate.
