Domain Adaptation and Few-Shot Learning

Knowledge distillation (KD) transfers knowledge from large language models (LLMs) to smaller or similarly sized models in order to obtain efficient yet capable systems. However, performing distillation over all tokens is computationally expensive and may weaken the transfer signal. To address this limitation, Knowledg…

No abstract yet.

Existing hyperspectral image (HSI) classification (HSIC) methods often fall short in effectively modeling long-range spatial-spectral dependencies and preserving fine-grained relationships across high-dimensional data. To address these limitations, this work introduces DiMAMamba, an innovative architecture that combin…

Abstract Continual Learning (CL) addresses the challenge of enabling models to adapt to evolving data and tasks while retaining previously acquired knowledge. The main challenge in this paradigm is catastrophic forgetting, where models lose prior knowledge upon learning new tasks. While much of the CL literature has f…

The continual learning literature has rapidly shifted from traditional class incremental learning (CIL) techniques to foundation model (FM)-based CIL methods without a clear understanding of how these newer approaches compare to strong, lightweight convolutional baselines. This abrupt transition has created a substant…

Continual learning requires models to integrate new classes or domains over time while preserving previously acquired knowledge. Within this paradigm, foundation models often achieve strong performance, but they still remain subject to the stability–plasticity tradeoff, where excessive plasticity leads to forgetting o…

Abstract In this work, we present Learning to Learn with Optimal Transport for Unsupervised Scenarios (LOTUS), a simple yet effective method to perform model selection for multiple unsupervised machine learning (ML) tasks such as outlier detection and clustering. Our intuition behind this work is that a machine learni…

Mixture of Experts (MoE) architectures enable efficient scaling of neural networks but suffer from expert collapse, where routing converges to a few dominant experts. This reduces model capacity and causes catastrophic interference during adaptation. We propose the Spectrally-Regularized Mixture of Experts (SR-MoE), w…

Mixture of Experts (MoE) architectures enable efficient scaling of neural networks but suffer from expert collapse, where routing converges to a few dominant experts. This reduces model capacity and causes catastrophic interference during adaptation. We propose the Spectrally-Regularized Mixture of Experts (SR-MoE), w…

Given the multifaceted nature of reality, phenomena can be interpreted not only through singular perspectives but also by bringing together various dimensions. Meaning often emerges from the convergence of diverse perspectives, contexts, and forms of representation. The construction of systems capable of analyzing thi…

Dynamic Rank Reinforcement Learning (DR-RL) approximations rely on static rank assumptions, limiting their flexibility across diverse linguistic contexts. Our method dynamically modulates ranks based on real-time sequence dynamics, layer-specific sensitivities, and hardware constraints. The core innovation is a deep r…

No abstract yet.