PhD Position (100%, E13, m/f/d) | Automating Large Language Models
Universität Leipzig
Universität Leipzig, 04105 Leipzig
Stellenangebot, Vollzeit
01.09.2024
About research group
A recurring criticism of machine learning (ML) is that, in practice, training a model is usually
more of an art than a science. It often relies on a process of trial and error that consumes
substantial amounts of human and computational time. Automated machine learning
(AutoML) is a young branch of machine learning that aims to streamline the ML workflow by
progressively automating parts that typically rely on human expert intuition. AutoML has
been quite successful; for example, it has outperformed human experts in Kaggle
competitions and configured the Monte-Carlo Tree Search of AlphaGo.
The Automated Machine Learning group at ScaDS.AI explores approaches for machine
learning systems that can configure themselves automatically by learning from past data.
We focus on hyperparameter optimization to automatically configure machine learning
models and neural architecture search to design better and more efficient neural network
architectures.
About the project
Large language models mark the beginning of a new era in artificial intelligence. These
models have achieved several breakthroughs in text understanding, code generation, and
machine translation. Conceptually quite simple, most of their success can be attributed to
the continuous scaling of compute, datasets, and model size. Unfortunately, the
computational demands of LLMs make it challenging to deploy and train them in practice.
The goal of this project is to develop new AutoML approaches that enhance the efficiency
and effectiveness of LLMs to democratize and increase their widespread adoption. More
specifically, the project aims to address the following research questions:
- How can we compress LLMs for faster inference?
- How can we accelerate the training process for faster convergence?
- How can we overcome current shortcomings of the transformer architecture?
- Can we exploit scaling laws for LLMs to enable large scale hyperparameter and neural
architecture search?
About ScaDS.AI
ScaDS.AI (Center for Scalable Data Analytics and Artificial Intelligence) Dresden/Leipzig is
one of the permanently established national center for artificial intelligence at the University
of Leipzig and the TU Dresden, which is financed by the federal government and the Free
State of Saxony. The Leipzig sub-center will be established as a central facility of Leipzig
University and in the medium term will bring together more than 200 staff members at the
Leipzig site alone. In ScaDS.AI, various research topics will be worked on within the
framework of a graduate school on the fundamentals and applications of data science and
artificial intelligence. In addition, service-oriented solutions are developed and there is close
cooperation with a large number of partner organizations from science and industry. The
center offers an excellent working environment with access to state-of-the-art technologies
and an outstanding high-performance computing infrastructure.
Requirements
- An excellent MSc degree (or about to finish) in Artificial Intelligence, Machine
Learning, Deep Learning, Computer Science or a related discipline
- Solid knowledge of (and experience with) machine learning and deep learning
methods
- Python knowledge with good working knowledge in applying/evaluating machine
learning & deep learning methods
- Knowledge in one or more of the following is beneficial: Hyperparameter Optimization
| Neural Architecture Search | Large Language Models |
To apply, please upload the following documents (as one PDF) to the plattform
- Preferred starting date (including earliest and latest possible date)
- CV & Transcript of records
- Research Statement (max one page) including why you want to start a PhD and why you want to do research in AutoML
Deadline for applications: none
For further questions, please also do reach out to
Dr. Aaron Klein
Dr. Eric Peukert
hr-scads@uni-leipzig.de.
Kontakt
Universität Leipzig
HR Team
hr-scads@uni-leipzig.de