Model distillation for large language models (LLMs) presents a key challenge in AI research: how to compress massive, computationally expensive models into smaller, more efficient versions while preserving their performance. Large models, such as GPT-4, require vast amounts of memory and processing power, making them impractical for real-time applications on edge devices or personal computers. … full description “Distilling large language models (available)”
Project List
Exploring optimization algorithms for deep sequence models (available)
Deep sequence models such as recurrent neural networks (RNNs) are a key type of architecture in modern deep learning, particularly for processing sequential data such as language text, speech, video, and time series data. RNNs have loops that allow information to persist and be passed from one step to the next, enabling them to effectively … full description “Exploring optimization algorithms for deep sequence models (available)”
Implementation of biologically inspired efficient deep learning models (available)
As deep learning models continue to grow in size and complexity to tackle increasingly difficult tasks, the need for efficient and scalable models becomes ever more important. Extremely large language models like GPT-4 require massive computational resources and expensive hardware to train and run. This makes them impractical to deploy at scale in many real-world … full description “Implementation of biologically inspired efficient deep learning models (available)”
AI-powered COVID-19 detection via cough sounds (available)
Respiratory diseases such as COVID-19 are known to physically damage our airway and lungs, which in turn alters the produced respiratory sounds (e.g., cough, breadth). During the pandemic, cough classification has emerged as an accessible, low-cost, and environmentally friendly COVID-19 screening alternative, needing only a smartphone to collect and process cough samples. However, audio processing … full description “AI-powered COVID-19 detection via cough sounds (available)”
Cough recognition system (ongoing)
The project requires the automated sorting of files and Raspberry Pi implementation of a Matlab trained CNN model for cough recognition and storage. This will involve acquiring data from a low-cost IoT based processing system (Raspberry Pi) connected to a network of CO2, temperature, humidity, PM 2.5 and sound sensors array. This project would suit … full description “Cough recognition system (ongoing)”
Engineering ROTOR: a Refactoring Tool for OCaml (ongoing)
OCaml [1] is a mature functional programming language with an expressive type system. Recently, we have developed a prototype tool, called ROTOR, for automatically refactoring OCaml codebases [2]. Currently, ROTOR handles renaming of functions. This is surprisingly hard due to OCaml’s powerful module system: renaming a function in one module may actually require renaming functions … full description “Engineering ROTOR: a Refactoring Tool for OCaml (ongoing)”
Natural Language Understanding: Measuring the Semantic Similarity between Sentences (ongoing)
Overview To implement and design various deep neural networks for measuring the semantic similarities between sentence pairs. Background Natural language understanding (NLU) is widely viewed as a grand challenge in Artificial Intelligence (AI). An important sub-task in NLU is to measure the semantic similarity between sentence pairs, also known as the Semantic Textual Similarity (STS) task. A … full description “Natural Language Understanding: Measuring the Semantic Similarity between Sentences (ongoing)”
Teaching Small LLMs to Reason (ongoing)
Large Language Models (LLMs) such as GPT4, are a game-changer for AI. Equipped with hundreds of billions of parameters, and trained on vast amounts of textual data totalling hundreds of terabytes, these models have revolutionised operations across numerous domains. But despite their considerable capabilities, their sheer size often means that they require substantial computational resources … full description “Teaching Small LLMs to Reason (ongoing)”