master thesis „benchmarking large-language models (llms) lms for robotic mission planning”

About

As Austria's largest research and technology organisation for applied research, we are dedicated to make substantial contributions to solving the major challenges of our time, climate change and digitisation. To achieve our goals, we rely on our specific research, development and technology competencies, which are the basis of our commitment to excellence in all areas. With our open culture of innovation and our motivated, international teams, we are working to position AIT as Austria's leading research institution at the highest international level and to make a positive contribution to the economy and society.

Our Center for Vision, Automation & Control is looking for new Ingenious Partner for our location in Vienna. The Center for Vision, Automation & Control exploits the opportunities provided by automation and digitisation to initiate and advance innovation for industry, primarily in Austria and Europe. Our main goal is to take away the monotonous, heavy, difficult and dangerous aspects of people’s work through innovation. We help our partners by making technical systems more robust, reliable, flexible and easy to use. At the same time we increase the resource efficiency of industrial processes by reducing waste, product failures and emissions.

Our Competence Unit Assistive & Autonomous Systems focuses on the development and deployment of assistive and autonomous systems, particularly unmanned aircrafts and land-based mobile machines, in cooperation with national and international research organisations and companies.

Your tasks

As part of this master’s thesis, you will deal with the topic of automated robotic systems, which perceive the environment and can classify, locate, and track objects for various applications in domain-specific areas.
Under the guidance of our Assistive & Autonomous Systems teams you will support us with the planning of new tasks, that require lengthy adaptations of our existing systems. Large-Language Models (LLMs), for e.g. ChatGPT, promise generalisation for many tasks and the ability to handle complex special cases automatically - However, the general nature of LLMs means they can often struggle with very specific requirements.
You will participate in the development of a benchmarking system to compare LLMs.
You will setup and run lm-evaluation-harness from EleutherAI (Open Source) to compare pre-trained LLMs and identify existing tasks in the benchmark that are relevant to robotic path planning.
You will create new tasks for the benchmark focusing on robotic mission planning in general.
You will extend the generic tasks to specific mission plans of existing robotic tasks (e.g. AIT’s autonomous forklift used in our Large-Scale Robotics Lab)
You contribute to a rapidly growing field in Machine Learning and Robotics.
You have the opportunity to investigate the state-of-the-art and apply it in real-world scenarios and – as a final step – design a proof-of-concept demonstration.
You will evaluate and document concepts and test-runs in predefined scenarios,.
You may publish your results at a conference or in a journal.

Your profile

Your qualifications as an Ingenious Partner:
Ongoing master’s studies in Machine Learning, Informatics, Robotics, Mechatronics or a comparable technical field.  
Experience with Machine Learning.
Programming experience in Python / C++.
Knowledge of Open Source LLMs is advantageous.
Knowledge of Reinforcement Learning is highly advantageous.
Good knowledge of English and German in word and writing

What to expect:

Duration of the master’s thesis project: 6 months
EUR 890 gross per month for 20 hours/week based on the collective agreement. There will be additional company benefits. You will be part of our international Young AIT network. As a research institution, we are familiar with the supervision and execution of master’s theses, and we are looking forward to supporting you accordingly.

Your application

At AIT diversity and inclusion are of great importance. This is why we strive to inspire women to join our teams in the field of technology. We welcome applications from women, who will be given preference in case of equal qualifications after taking into account all relevant facts and circumstances of all applications.

Please submit your application documents including your CV, cover letter and certificates (transcript of records) online.