Search Results for author:
Found 285 papers, 93 papers with code
Dual Expert Distillation Network for Generalized Zero-Shot Learning
Zero-shot learning has consistently yielded remarkable progress via modeling nuanced one-to-one visual-attribute correlation.
SkinGEN: an Explainable Dermatology Diagnosis-to-Generation Framework with Interactive Vision-Language Models
With the continuous advancement of vision language models (VLMs) technology, remarkable research achievements have emerged in the dermatology field, the fourth most prevalent human disease category.
ID-Animator: Zero-Shot Identity-Preserving Human Video Generation
Based on this pipeline, a random face reference training method is further devised to precisely capture the ID-relevant embeddings from reference images, thus improving the fidelity and generalization capacity of our model for ID-specific video generation.
High Noise Scheduling is a Must
The results show that the polynomial noise distribution outperforms the model trained with log-normal noise distribution, yielding a 33. 54 FID score after 100, 000 training steps with constant discretization steps.
Decision Transformer for Wireless Communications: A New Paradigm of Resource Management
By leveraging the power of DT models learned over extensive datasets, the proposed architecture is expected to achieve rapid convergence with many fewer training epochs and higher performance in a new context, e. g., similar tasks with different state and action spaces, compared with DRL.
Jailbreaking Prompt Attack: A Controllable Adversarial Attack against Diffusion Models
The fast advance of the image generation community has attracted attention worldwide.
How memories are stored in the brain: the declarative memory model
The ability to form memories is a basic feature of learning and accumulating knowledge.
RU22Fact: Optimizing Evidence for Multilingual Explainable Fact-Checking on Russia-Ukraine Conflict
Furthermore, we construct RU22Fact, a novel multilingual explainable fact-checking dataset on the Russia-Ukraine conflict in 2022 of 16K samples, each containing real-world claims, optimized evidence, and referenced explanation.
Re2LLM: Reflective Reinforcement Large Language Model for Session-based Recommendation
However, the former methods struggle with optimal prompts to elicit the correct reasoning of LLMs due to the lack of task-specific feedback, leading to unsatisfactory recommendations.
DiPrompT: Disentangled Prompt Tuning for Multiple Latent Domain Generalization in Federated Learning
Federated learning (FL) has emerged as a powerful paradigm for learning from decentralized data, and federated domain generalization further considers the test dataset (target domain) is absent from the decentralized training data (source domains).
DiffClass: Diffusion-Based Class Incremental Learning
On top of that, Exemplar-free Class Incremental Learning is even more challenging due to forbidden access to previous task data.
Towards Tracing Trustworthiness Dynamics: Revisiting Pre-training Period of Large Language Models
This research provides an initial exploration of trustworthiness modeling during LLM pre-training, seeking to unveil new insights and spur further developments in the field.
RSAM-Seg: A SAM-based Approach with Prior Knowledge Integration for Remote Sensing Image Semantic Segmentation
While the direct application of SAM to remote sensing image segmentation tasks does not yield satisfactory results, we propose RSAM-Seg, which stands for Remote Sensing SAM with Semantic Segmentation, as a tailored modification of SAM for the remote sensing field and eliminates the need for manual intervention to provide prompts.
Model X-ray:Detect Backdoored Models via Decision Boundary
To address it, in this paper, we begin by presenting an intriguing observation: the decision boundary of the backdoored model exhibits a greater degree of closeness than that of the clean model.
FinGPT-HPC: Efficient Pretraining and Finetuning Large Language Models for Financial Applications with High-Performance Computing
However, the resulting model still consumes a large amount of GPU memory.
Pan-Mamba: Effective pan-sharpening with State Space Model
To the best of our knowledge, this work is the first attempt in exploring the potential of the Mamba model and establishes a new frontier in the pan-sharpening techniques.
Towards Cross-Domain Continual Learning
In this work, we introduce a novel approach called Cross-Domain Continual Learning (CDCL) that addresses the limitations of being limited to single supervised domains.
SIBO: A Simple Booster for Parameter-Efficient Fine-Tuning
Fine-tuning all parameters of large language models (LLMs) necessitates substantial computational power and extended time.
Large Language Model with Graph Convolution for Recommendation
To adapt text-based LLMs with structured graphs, We use the LLM as an aggregator in graph processing, allowing it to understand graph-based information step by step.
Symbol: Generating Flexible Black-Box Optimizers through Symbolic Equation Learning
Recent Meta-learning for Black-Box Optimization (MetaBBO) methods harness neural networks to meta-learn configurations of traditional black-box optimizers.
PRIME: Protect Your Videos From Malicious Editing
With the development of generative models, the quality of generated content keeps increasing.
Multi-granularity Correspondence Learning from Long-term Noisy Videos
Existing video-language studies mainly focus on learning short video clips, leaving long-term temporal dependencies rarely explored due to over-high computational cost of modeling long videos.
Ranked #1 on
Zero-Shot Video Retrieval
on YouCook2
Action Segmentation
Long Video Retrieval (Background Removed)
+2
From GPT-4 to Gemini and Beyond: Assessing the Landscape of MLLMs on Generalizability, Trustworthiness and Causality through Four Modalities
Multi-modal Large Language Models (MLLMs) have shown impressive abilities in generating reasonable responses with respect to multi-modal contents.
Adversarial speech for voice privacy protection from Personalized Speech generation
For validation, we employ the open-source pre-trained YourTTS model for speech generation and protect the target speaker's speech in the white-box scenario.
CBVS: A Large-Scale Chinese Image-Text Benchmark for Real-World Short Video Search Scenarios
Differently, video covers in short video search scenarios are presented as user-originated contents that provide important visual summaries of videos.
Ranked #1 on
Image Retrieval
on CBVS
Generalized Face Liveness Detection via De-spoofing Face Generator
In this paper, we conduct an Anomalous cue Guided FAS (AG-FAS) method, which leverages real faces for improving model generalization via a De-spoofing Face Generator (DFG).
Collaboratively Self-supervised Video Representation Learning for Action Recognition
Considering the close connection between action recognition and human pose estimation, we design a Collaboratively Self-supervised Video Representation (CSVR) learning framework specific to action recognition by jointly considering generative pose prediction and discriminative context matching as pretext tasks.
Pre-trained Model Guided Fine-Tuning for Zero-Shot Adversarial Robustness
Specifically, PMG-AFT minimizes the distance between the features of adversarial examples in the target model and those in the pre-trained model, aiming to preserve the generalization features already captured by the pre-trained model.
TeleChat Technical Report
Subsequently, the model undergoes fine-tuning to align with human preferences, following a detailed methodology that we describe.
Multichannel AV-wav2vec2: A Framework for Learning Multichannel Multi-Modal Speech Representation
Considering that visual information helps to improve speech recognition performance in noisy scenes, in this work we propose a multichannel multi-modal speech self-supervised learning framework AV-wav2vec2, which utilizes video and multichannel audio data as inputs.
Audio-Visual Speech Recognition
Automatic Speech Recognition
+7
Frequency-Adaptive Pan-Sharpening with Mixture of Experts
Pan-sharpening involves reconstructing missing high-frequency information in multi-spectral images with low spatial resolution, using a higher-resolution panchromatic image as guidance.
Enhancing RAW-to-sRGB with Decoupled Style Structure in Fourier Domain
However, current methods often ignore the difference between cell phone RAW images and DSLR camera RGB images, a difference that goes beyond the color matrix and extends to spatial structure due to resolution variations.
Towards Robust Semantic Segmentation against Patch-based Attack via Attention Refinement
The attention mechanism has been proven effective on various visual tasks in recent years.
FullLoRA-AT: Efficiently Boosting the Robustness of Pretrained Vision Transformers
In recent years, the Vision Transformer (ViT) model has gradually become mainstream in various computer vision tasks, and the robustness of the model has received increasing attention.
SAME: Sample Reconstruction against Model Extraction Attacks
While deep learning models have shown significant performance across various domains, their deployment needs extensive resources and advanced computing infrastructure.
ParsNets: A Parsimonious Orthogonal and Low-Rank Linear Networks for Zero-Shot Learning
Then, to facilitate the generalization of local linearities, we construct a maximal margin geometry on the learned features by enforcing low-rank constraints on intra-class samples and high-rank constraints on inter-class samples, resulting in orthogonal subspaces for different classes and each subspace lies on a compact manifold.
iComMa: Inverting 3D Gaussian Splatting for Camera Pose Estimation via Comparing and Matching
We present a method named iComMa to address the 6D camera pose estimation problem in computer vision.
MaTe3D: Mask-guided Text-based 3D-aware Portrait Editing
To address this limitation, we propose \textbf{MaTe3D}: mask-guided text-based 3D-aware portrait editing.
Calibration-free quantitative phase imaging in multi-core fiber endoscopes using end-to-end deep learning
Quantitative phase imaging (QPI) through multi-core fibers (MCFs) has been an emerging in vivo label-free endoscopic imaging modality with minimal invasiveness.
Control Risk for Potential Misuse of Artificial Intelligence in Science
In this study, we aim to raise awareness of the dangers of AI misuse in science, and call for responsible AI development and use in this domain.
Data-Free Hard-Label Robustness Stealing Attack
In response to these identified gaps, we introduce a novel Data-Free Hard-Label Robustness Stealing (DFHL-RS) attack in this paper, which enables the stealing of both model accuracy and robustness by simply querying hard labels of the target model without the help of any natural data.
Singular Regularization with Information Bottleneck Improves Model's Adversarial Robustness
However, these works lack analysis of adversarial information or perturbation, which cannot reveal the mystery of adversarial examples and lose proper interpretation.
FreePIH: Training-Free Painterly Image Harmonization with Diffusion Model
Specifically, we find that the very last few steps of the denoising (i. e., generation) process strongly correspond to the stylistic information of images, and based on this, we propose to augment the latent features of both the foreground and background images with Gaussians for a direct denoising-based harmonization.
Attribute-Aware Representation Rectification for Generalized Zero-Shot Learning
In this paper, we propose a simple yet effective Attribute-Aware Representation Rectification framework for GZSL, dubbed $\mathbf{(AR)^{2}}$, to adaptively rectify the feature extractor to learn novel features while keeping original valuable features.
Sparsity-Driven EEG Channel Selection for Brain-Assisted Speech Enhancement
In this work, we therefore propose a novel end-to-end brain-assisted speech enhancement network (BASEN), which incorporates the listeners' EEG signals and adopts a temporal convolutional network together with a convolutional multi-layer cross attention module to fuse EEG-audio features.
Improving Adversarial Transferability by Stable Diffusion
Various techniques have emerged to enhance the transferability of adversarial attacks for the black-box scenario.
Segue: Side-information Guided Generative Unlearnable Examples for Facial Privacy Protection in Real World
To address these concerns, researchers are actively exploring the concept of ``unlearnable examples", by adding imperceptible perturbation to data in the model training stage, which aims to prevent the model from learning discriminate features of the target face.
Pix2HDR -- A pixel-wise acquisition and deep learning-based synthesis approach for high-speed HDR videos
By combining the versatility of pixel-wise sampling patterns with the strength of deep neural networks at decoding complex scenes, our method greatly enhances the vision system's adaptability and performance in dynamic conditions.
IntentDial: An Intent Graph based Multi-Turn Dialogue System with Reasoning Path Visualization
Intent detection and identification from multi-turn dialogue has become a widely explored technique in conversational agents, for example, voice assistants and intelligent customer services.
Real-Fake: Effective Training Data Synthesis Through Distribution Matching
Synthetic training data has gained prominence in numerous learning tasks and scenarios, offering advantages such as dataset augmentation, generalization evaluation, and privacy preservation.
Multiview Transformer: Rethinking Spatial Information in Hyperspectral Image Classification
To aggregate the multiview information, a fully-convolutional SED with a U-shape in spectral dimension is introduced to extract a multiview feature map.
Latent Diffusion Model for Medical Image Standardization and Enhancement
Finally, the decoder uses the transformed latent representation to generate a standardized CT image, providing a more consistent basis for downstream analysis.
PGDQN: Preference-Guided Deep Q-Network
Stochastic exploration is the key to the success of the Deep Q-network (DQN) algorithm.
Warfare:Breaking the Watermark Protection of AI-Generated Content
We propose Warfare, a unified methodology to achieve both attacks in a holistic way.
Cluster-based Method for Eavesdropping Identification and Localization in Optical Links
We propose a cluster-based method to detect and locate eavesdropping events in optical line systems characterized by small power losses.
Hierarchical Audio-Visual Information Fusion with Multi-label Joint Decoding for MER 2023
Three different structures based on attention-guided feature gathering (AFG) are designed for deep feature fusion.
Understanding Data Augmentation from a Robustness Perspective
In the realm of visual recognition, data augmentation stands out as a pivotal technique to amplify model robustness.
Bias Testing and Mitigation in LLM-based Code Generation
To mitigate bias for code generation models, we evaluate five bias mitigation prompt strategies, i. e., utilizing bias testing results to refine the code (zero-shot), one-, few-shot, and two Chain-of-Thought (CoT) prompts.
Rep2wav: Noise Robust text-to-speech Using self-supervised representations
Noise-robust TTS models are often trained using the enhanced speech, which thus suffer from speech distortion and background noise that affect the quality of the synthesized speech.
Patch Is Not All You Need
Vision Transformers have achieved great success in computer visions, delivering exceptional performance across various tasks.
Backdooring Textual Inversion for Concept Censorship
Users can easily download the word embedding from public websites like Civitai and add it to their own stable diffusion model without fine-tuning for personalization.
Semantic-Human: Neural Rendering of Humans from Monocular Video with Human Parsing
In this paper, we present Semantic-Human, a novel method that achieves both photorealistic details and viewpoint-consistent human parsing for the neural rendering of humans.
Overlap Bias Matching is Necessary for Point Cloud Registration
Specifically, we propose a plug-and-play Overlap Bias Matching Module (OBMM) comprising two integral components, overlap sampling module and bias prediction module.
Mobile Supply: The Last Piece of Jigsaw of Recommender System
In order to address the problem of pagination trigger mechanism, we propose a completely new module in the pipeline of recommender system named Mobile Supply.
Sampling to Distill: Knowledge Transfer from Open-World Data
Then, we introduce a low-noise representation to alleviate the domain shifts and build a structured relationship of multiple data examples to exploit data knowledge.
The RoboDepth Challenge: Methods and Advancements Towards Robust Depth Estimation
In this paper, we summarize the winning solutions from the RoboDepth Challenge -- an academic competition designed to facilitate and advance robust OoD depth estimation.
Rethinking Data Distillation: Do Not Overlook Calibration
Neural networks trained on distilled data often produce over-confident output and require correction by calibration methods.
Enhancing Job Recommendation through LLM-based Generative Adversarial Networks
However, directly leveraging LLMs to enhance recommendation results is not a one-size-fits-all solution, as LLMs may suffer from fabricated generation and few-shot problems, which degrade the quality of resume completion.
Our Model Achieves Excellent Performance on MovieLens: What Does it Mean?
First, there are significant differences in user interactions at the different stages when a user interacts with the MovieLens platform.
ESMC: Entire Space Multi-Task Model for Post-Click Conversion Rate via Parameter Constraint
We propose a novel Entire Space Multi-Task Model for Post-Click Conversion Rate via Parameter Constraint (ESMC) and two alternatives: Entire Space Multi-Task Model with Siamese Network (ESMS) and Entire Space Multi-Task Model in Global Domain (ESMG) to address the PSC issue.
Learning Subjective Time-Series Data via Utopia Label Distribution Approximation
In this work, we propose Utopia Label Distribution Approximation (ULDA) for time-series data, which makes the training label distribution closer to real-world but unknown (utopia) label distribution.
Combating Data Imbalances in Federated Semi-supervised Learning with Dual Regulators
Theoretically, we show the convergence guarantee of the dual regulators.
Using electrical impedance spectroscopy to identify equivalent circuit models of lubricated contacts with complex geometry: in-situ application to mini traction machine
Overall, the proposed method is applicable to general lubricated interfaces for the identification of equivalent circuit models, which in turn facilitates in-situ tribo-contacts with electric impedance measurement of oil film thickness.
That's BAD: Blind Anomaly Detection by Implicit Local Feature Clustering
They consider an unsupervised setting, specifically the one-class setting, in which we assume the availability of a set of normal (\textit{i. e.}, anomaly-free) images for training.
Contextual Affinity Distillation for Image Anomaly Detection
The local student, which is used in previous studies mainly focuses on structural anomaly detection while the global student pays attention to logical anomalies.
Ranked #13 on
Anomaly Detection
on MVTec LOCO AD
Federated Generative Learning with Foundation Models
Existing federated learning solutions focus on transmitting features, parameters or gadients between clients and server, which suffer from serious low-efficiency and privacy-leakage problems.
Phonon dynamic behaviors induced by amorphous interlayer at heterointerfaces
However, due to mode conversion and inelastic scattering, we found a portion of high-frequency TA phonons, which are higher than the cut-off frequency and cannot transmit across the ideal sharp interface, can partially transmit across the amorphous interlayer, which introduces additional thermal transport channels through the interface and has positive effect on interfacial thermal conductance.
On Knowledge Editing in Federated Learning: Perspectives, Challenges, and Future Directions
As Federated Learning (FL) has gained increasing attention, it has become widely acknowledged that straightforwardly applying stochastic gradient descent (SGD) on the overall framework when learning over a sequence of tasks results in the phenomenon known as ``catastrophic forgetting''.
Dissecting Arbitrary-scale Super-resolution Capability from Pre-trained Diffusion Generative Models
Diffusion-based Generative Models (DGMs) have achieved unparalleled performance in synthesizing high-quality visual content, opening up the opportunity to improve image super-resolution (SR) tasks.
Towards Omni-generalizable Neural Methods for Vehicle Routing Problems
Learning heuristics for vehicle routing problems (VRPs) has gained much attention due to the less reliance on hand-crafted rules.
Towards Efficient Deep Hashing Retrieval: Condensing Your Data via Feature-Embedding Matching
The expenses involved in training state-of-the-art deep hashing retrieval models have witnessed an increase due to the adoption of more sophisticated models and large-scale datasets.
Toward stochastic neural computing
To account for the propagation of correlated neural variability, we derive from first principles a moment embedding for spiking neural network (SNN).
Layer-adaptive Structured Pruning Guided by Latency
Structured pruning can simplify network architecture and improve inference speed.
Eeg2vec: Self-Supervised Electroencephalographic Representation Learning
Recently, many efforts have been made to explore how the brain processes speech using electroencephalographic (EEG) signals, where deep learning-based approaches were shown to be applicable in this field.
Semantic VAD: Low-Latency Voice Activity Detection for Speech Interaction
For speech interaction, voice activity detection (VAD) is often used as a front-end.
CASA-ASR: Context-Aware Speaker-Attributed ASR
In addition, a two-pass decoding strategy is further proposed to fully leverage the contextual modeling ability resulting in a better recognition performance.
Personalization as a Shortcut for Few-Shot Backdoor Attack against Text-to-Image Diffusion Models
Although recent personalization methods have democratized high-resolution image synthesis by enabling swift concept acquisition with minimal examples and lightweight computation, they also present an exploitable avenue for high accessible backdoor attacks.
BASEN: Time-Domain Brain-Assisted Speech Enhancement Network with Convolutional Cross Attention in Multi-talker Conditions
In this paper, we thus propose a novel time-domain brain-assisted SE network (BASEN) incorporating electroencephalography (EEG) signals recorded from the listener for extracting the target speaker from monaural speech mixtures.
Watermarking Text Generated by Black-Box Language Models
To allow third-parties to autonomously inject watermarks into generated text, we develop a watermarking framework for black-box language model usage scenarios.
A Black-Box Attack on Code Models via Representation Nearest Neighbor Search
Furthermore, the perturbation of adversarial examples introduced by RNNS is smaller compared to the baselines in terms of the number of replaced variables and the change in variable length.
Vision Language Pre-training by Contrastive Learning with Cross-Modal Similarity Regulation
Cross-modal contrastive learning in vision language pretraining (VLP) faces the challenge of (partial) false negatives.
Retraining A Graph-based Recommender with Interests Disentanglement
Then we blend the historical and new preferences in the form of node embeddings in the new graph, through a Disentanglement Module.
Towards Unbiased Training in Federated Open-world Semi-supervised Learning
Federated Semi-supervised Learning (FedSSL) has emerged as a new paradigm for allowing distributed clients to collaboratively train a machine learning model over scarce labeled data and abundant unlabeled data.
Inhomogeneous graph trend filtering via a l2,0 cardinality penalty
We study estimation of piecewise smooth signals over a graph.
Multi-Sample Consensus Driven Unsupervised Normal Estimation for 3D Point Clouds
Comprehensive experiments demonstrate that the two proposed unsupervised methods are noticeably superior to some supervised deep normal estimators on the most common synthetic dataset.
CCLAP: Controllable Chinese Landscape Painting Generation via Latent Diffusion Model
While the style aggregator module is to generate paintings of a style corresponding to a reference image.
Real Face Foundation Representation Learning for Generalized Deepfake Detection
In this study, we propose Real Face Foundation Representation Learning (RFFR), which aims to learn a general representation from large-scale real face datasets and detect potential artifacts outside the distribution of RFFR.
TARGET: Federated Class-Continual Learning via Exemplar-Free Distillation
This paper focuses on an under-explored yet important problem: Federated Class-Continual Learning (FCCL), where new classes are dynamically added in federated learning.
Neural Airport Ground Handling
Airport ground handling (AGH) offers necessary operations to flights during their turnarounds and is of great importance to the efficiency of airport management and the economics of aviation.
Learning Large Neighborhood Search for Vehicle Routing in Airport Ground Handling
Dispatching vehicle fleets to serve flights is a key task in airport ground handling (AGH).
Pseudo Label-Guided Model Inversion Attack via Conditional Generative Adversarial Network
At first, a top-n selection strategy is proposed to provide pseudo-labels for public data, and use pseudo-labels to guide the training of the cGAN.
Delving into the Adversarial Robustness of Federated Learning
In this work, we propose a novel algorithm called Decision Boundary based Federated Adversarial Training (DBFAT), which consists of two components (local re-weighting and global regularization) to improve both accuracy and robustness of FL systems.
Speech Enhancement with Multi-granularity Vector Quantization
With advances in deep learning, neural network based speech enhancement (SE) has developed rapidly in the last decade.
Towards Fairer and More Efficient Federated Learning via Multidimensional Personalized Edge Models
Federated learning (FL) is an emerging technique that trains massive and geographically distributed edge data while maintaining privacy.
TAP: Accelerating Large-Scale DNN Training Through Tensor Automatic Parallelisation
However, finding a suitable model parallel schedule for an arbitrary neural network is a non-trivial task due to the exploding search space.
DiffusionCT: Latent Diffusion Model for CT Image Standardization
This work addresses the issue of CT image harmonization using a new diffusion-based model, named DiffusionCT, to standardize CT images acquired from different vendors and protocols.
Pyramid Dual Domain Injection Network for Pan-sharpening
To this end, we first revisit the degradation process of pan-sharpening in Fourier space, and then devise a Pyramid Dual Domain Injection Pan-sharpening Network upon the above observation by fully exploring and exploiting the distinguished information in both the spatial and frequency domains.
Quantum-Inspired Spectral-Spatial Pyramid Network for Hyperspectral Image Classification
Using QSSN as the building block, we propose an end-to-end quantum-inspired spectral-spatial pyramid network (QSSPN) for HSI feature extraction and classification.
Unveiling Code Pre-Trained Models: Investigating Syntax and Semantics Capacities
These structures are fundamental to understanding code.
Exploring Optimal Substructure for Out-of-distribution Generalization via Feature-targeted Model Pruning
Recent studies show that even highly biased dense networks contain an unbiased substructure that can achieve better out-of-distribution (OOD) generalization than the original model.
Accelerating Dataset Distillation via Model Augmentation
Dataset Distillation (DD), a newly emerging field, aims at generating much smaller but efficient synthetic training datasets from large ones.
Efficient Stein Variational Inference for Reliable Distribution-lossless Network Pruning
Extensive experiments based on small Cifar-10 and large-scaled ImageNet demonstrate that our method can obtain sparser networks with great generalization performance while providing quantified reliability for the pruned model.
Ada3Diff: Defending against 3D Adversarial Point Clouds via Adaptive Diffusion
Deep 3D point cloud models are sensitive to adversarial attacks, which poses threats to safety-critical applications such as autonomous driving.
Boosting Personalised Musculoskeletal Modelling with Physics-informed Knowledge Transfer
The salient advantages of the proposed framework are twofold: 1) For the generic model, physics-based domain knowledge is embedded into the loss function of the data-driven model as soft constraints to penalise/regularise the data-driven model.
Decision-making with Speculative Opponent Models
To address this issue, we introduce Distributional Opponent-aided Multi-agent Actor-Critic (DOMAC), the first speculative opponent modelling algorithm that relies solely on local information (i. e., the controlled agent's observations, actions, and rewards).
Exploiting Personalized Invariance for Better Out-of-distribution Generalization in Federated Learning
Recently, data heterogeneity among the training datasets on the local clients (a. k. a., Non-IID data) has attracted intense interest in Federated Learning (FL), and many personalized federated learning methods have been proposed to handle it.
Out-of-Distribution Generalization
Personalized Federated Learning
VATLM: Visual-Audio-Text Pre-Training with Unified Masked Prediction for Speech Representation Learning
Although speech is a simple and effective way for humans to communicate with the outside world, a more realistic speech interaction contains multimodal information, e. g., vision, text.
Deep Reinforcement Learning Guided Improvement Heuristic for Job Shop Scheduling
Recent studies in using deep reinforcement learning (DRL) to solve Job-shop scheduling problems (JSSP) focus on construction heuristics.
FedTune: A Deep Dive into Efficient Federated Fine-Tuning with Pre-trained Transformers
Federated Learning (FL) is an emerging paradigm that enables distributed users to collaboratively and iteratively train machine learning models without sharing their private data.
Feature Correlation-guided Knowledge Transfer for Federated Self-supervised Learning
We prove that FedFoA is a model-agnostic training framework and can be easily compatible with state-of-the-art unsupervised FL methods.
Reinforcement Learning Enhanced Weighted Sampling for Accurate Subgraph Counting on Fully Dynamic Graph Streams
Specifically, we propose a weighted sampling algorithm called WSD for estimating the subgraph count in a fully dynamic graph stream, which samples the edges based on their weights that indicate their importance and reflect their properties.
Demystify Self-Attention in Vision Transformers from a Semantic Perspective: Analysis and Application
In contrast, visual data exhibits a fundamentally different structure: Its basic unit (pixel) is a natural low-level representation with significant redundancies in the neighbourhood, which poses obvious challenges to the interpretability of MSA mechanism in ViT.
A Comparative Study on Multichannel Speaker-Attributed Automatic Speech Recognition in Multi-party Meetings
Speaker-attributed automatic speech recognition (SA-ASR) in multi-party meeting scenarios is one of the most valuable and challenging ASR task.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Robust Data2vec: Noise-robust Speech Representation Learning for ASR by Combining Regression and Improved Contrastive Learning
Self-supervised pre-training methods based on contrastive learning or regression tasks can utilize more unlabeled data to improve the performance of automatic speech recognition (ASR).
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
From Intelligent Agents to Trustworthy Human-Centred Multiagent Systems
The Agents, Interaction and Complexity research group at the University of Southampton has a long track record of research in multiagent systems (MAS).
Speech Enhancement Using Self-Supervised Pre-Trained Model and Vector Quantization
Specifically, the encoder and bottleneck layer of the DEMUCS model are initialized using the self-supervised pretrained WavLM model, the convolution in the encoder is replaced by causal convolution, and the transformer encoder in the bottleneck layer is based on causal attention mask.
Learning to Solve Multiple-TSP with Time Window and Rejections via Deep Reinforcement Learning
We propose a manager-worker framework based on deep reinforcement learning to tackle a hard yet nontrivial variant of Travelling Salesman Problem (TSP), \ie~multiple-vehicle TSP with time window and rejections (mTSPTWR), where customers who cannot be served before the deadline are subject to rejections.
Class-Incremental Learning via Knowledge Amalgamation
Catastrophic forgetting has been a significant problem hindering the deployment of deep learning algorithms in the continual learning setting.
A Multi-Channel Next POI Recommendation Framework with Multi-Granularity Check-in Signals
Being equipped with three modules (i. e., global user behavior encoder, local multi-channel encoder, and region-aware weighting strategy), MCMG is capable of capturing both fine- and coarse-grained sequential regularities as well as exploring the dynamic impact of multi-channel by differentiating the region check-in patterns.
Federated Learning with Label Distribution Skew via Logits Calibration
Extensive experiments on federated datasets and real-world datasets demonstrate that FedLC leads to a more accurate global model and much improved performance.
Fed-FSNet: Mitigating Non-I.I.D. Federated Learning via Fuzzy Synthesizing Network
Concretely, we maintain an edge-agnostic hidden model in the cloud server to estimate a less-accurate while direction-aware inversion of the global model.
Hierarchical Compositional Representations for Few-shot Action Recognition
Recently action recognition has received more and more attention for its comprehensive and practical applications in intelligent surveillance and human-computer interaction.
Multi-modal Transformer Path Prediction for Autonomous Vehicle
To better utilize the lane information, the lanes which are in opposite direction to target agent are not likely to be taken by the target agent and are consequently filtered out.
Microwave QR Code: An IRS-Based Solution
This letter proposes to employ intelligent reflecting surface (IRS) as an information media to display a microwave quick response (QR) code for Internet-of-Things applications.
Learning Pseudo Labels for Semi-and-Weakly Supervised Semantic Segmentation
Firstly, we introduce a class-aware cross entropy (CCE) loss for network training.
An IRS Backscatter Enabled Integrated Sensing, Communication and Computation System
This paper proposes to leverage intelligent reflecting surface (IRS) backscatter to realize radio-frequency-chain-free uplink-transmissions (RFCF-UT).
Physics-informed Deep Learning for Musculoskeletal Modelling: Predicting Muscle Forces and Joint Kinematics from Surface EMG
At the same time, the physics law between muscle forces and joint kinematics is used the soft constraint.
On the Performance of Data Compression in Clustered Fog Radio Access Networks
The fog-radio-access-network (F-RAN) has been proposed to address the strict latency requirements, which offloads computation tasks generated in user equipments (UEs) to the edge to reduce the processing latency.
DaisyRec 2.0: Benchmarking Recommendation for Rigorous Evaluation
Recently, one critical issue looms large in the field of recommender systems -- there are no effective benchmarks for rigorous evaluation -- which consequently leads to unreproducible evaluation and unfair comparison.
Sampling Efficient Deep Reinforcement Learning through Preference-Guided Stochastic Exploration
We theoretically prove that the policy improvement theorem holds for the preference-guided $\epsilon$-greedy policy and experimentally show that the inferred action preference distribution aligns with the landscape of corresponding Q-values.
Joint Training of Speech Enhancement and Self-supervised Model for Noise-robust ASR
Speech enhancement (SE) is usually required as a front end to improve the speech quality in noisy environments, while the enhanced speech might not be optimal for automatic speech recognition (ASR) systems due to speech distortion.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
IDEAL: Query-Efficient Data-Free Learning from Black-box Models
Knowledge Distillation (KD) is a typical method for training a lightweight student model with the help of a well-trained teacher model.
Snake net and balloon force with a neural network for detecting multiple phases
In order to obtain a physical phase diagram, the snake model with an artificial neural network is applied in an unsupervised learning way by the authors of [Phys. Rev. Lett.
Boosting Pruned Networks with Linear Over-parameterization
To address this challenge, we propose a novel method that first linearly over-parameterizes the compact layers in pruned networks to enlarge the number of fine-tuning parameters and then re-parameterizes them to the original layers after fine-tuning.
Sign Bit is Enough: A Learning Synchronization Framework for Multi-hop All-reduce with Ultimate Compression
Traditional one-bit compressed stochastic gradient descent can not be directly employed in multi-hop all-reduce, a widely adopted distributed training paradigm in network-intensive high-performance computing systems such as public clouds.
On Scheduling Mechanisms Beyond the Worst Case
That is, mechanism K is pointwise better than mechanism P. Next, for each task $j$, when machines' execution costs $t_i^j$ are independent and identically drawn from a task-specific distribution $F^j(t)$, we show that the average-case approximation ratio of mechanism K converges to a constant.
Adaptive Modulation for Wobbling UAV Air-to-Ground Links in Millimeter-wave Bands
Based on the designed detectors, we propose an adaptive modulation scheme to maximize the average transmission rate under imperfect CSI by optimizing the data transmission time subject to the maximum tolerable BEP.
Do Loyal Users Enjoy Better Recommendations? Understanding Recommender Accuracy from a Time Perspective
Our study offers a different perspective to understand recommender accuracy, and our findings could trigger a revisit of recommender model design.
PICASSO: Unleashing the Potential of GPU-centric Training for Wide-and-deep Recommender Systems
However, we observe that GPU devices in training recommender systems are underutilized, and they cannot attain an expected throughput improvement as what it has achieved in CV and NLP areas.
A Complementary Joint Training Approach Using Unpaired Speech and Text for Low-Resource Automatic Speech Recognition
Unpaired data has shown to be beneficial for low-resource automatic speech recognition~(ASR), which can be involved in the design of hybrid models with multi-task training or language model dependent pre-training.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Collaborative Intelligent Reflecting Surface Networks with Multi-Agent Reinforcement Learning
Intelligent reflecting surface (IRS) is envisioned to be widely applied in future wireless networks.
Multi-agent Reinforcement Learning
reinforcement-learning
+1
Spectral Graph Clustering for Intentional Islanding Operations in Resilient Hybrid Energy Systems
Establishing cleaner energy generation therefore improving the sustainability of the power system is a crucial task in this century, and one of the key strategies being pursued is to shift the dependence on fossil fuel to renewable technologies such as wind, solar, and nuclear.
Learning Contextually Fused Audio-visual Representations for Audio-visual Speech Recognition
The proposed approach explores both the complementarity of audio-visual modalities and long-term context dependency using a transformer-based fusion module and a flexible masking strategy.
Learning to Solve Routing Problems via Distributionally Robust Optimization
Recent deep models for solving routing problems always assume a single distribution of nodes for training, which severely impairs their cross-distribution generalization ability.
A Coalition Formation Game Approach for Personalized Federated Learning
Facing the challenge of statistical diversity in client local data distribution, personalized federated learning (PFL) has become a growing research hotspot.
Adversarial Examples for Good: Adversarial Examples Guided Imbalanced Learning
Adversarial examples are inputs for machine learning models that have been designed by attackers to cause the model to make mistakes.
A Noise-Robust Self-supervised Pre-training Model Based Speech Representation Learning for Automatic Speech Recognition
In this work, we therefore first analyze the noise robustness of wav2vec2. 0 via experiments.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Supervised and Self-supervised Pretraining Based COVID-19 Detection Using Acoustic Breathing/Cough/Speech Signals
By using the acoustic signals to train the network, respectively, we can build individual models for three tasks, whose parameters are averaged to obtain an average model, which is then used as the initialization for the BiLSTM model training of each task.
FedDTG:Federated Data-Free Knowledge Distillation via Three-Player Generative Adversarial Networks
This technique mitigates the user heterogeneity problem and better protects user privacy.
Enhancing Face Recognition With Self-Supervised 3D Reconstruction
In this paper, we propose to enhance face recognition with a bypass of self-supervised 3D reconstruction, which enforces the neural backbone to focus on the identity-related depth and albedo information while neglects the identity-irrelevant pose and illumination information.
Towards Efficient Data Free Black-Box Adversarial Attack
The proposed method can efficiently imitate the target model through a small number of queries and achieve high attack success rate.
DENSE: Data-Free One-Shot Federated Learning
One-shot Federated Learning (FL) has recently emerged as a promising approach, which allows the central server to learn a model in a single communication round.
Initiative Defense against Facial Manipulation
To this end, we first imitate the target manipulation model with a surrogate model, and then devise a poison perturbation generator to obtain the desired venom.
From Deterioration to Acceleration: A Calibration Approach to Rehabilitating Step Asynchronism in Federated Optimization
In the setting of federated optimization, where a global model is aggregated periodically, step asynchronism occurs when participants conduct model training by efficiently utilizing their computational resources.
GIMIRec: Global Interaction Information Aware Multi-Interest Framework for Sequential Recommendation
Sequential recommendation based on multi-interest framework models the user's recent interaction sequence into multiple different interest vectors, since a single low-dimensional vector cannot fully represent the diversity of user interests.
Tracing Text Provenance via Context-Aware Lexical Substitution
Tracing text provenance can help claim the ownership of text content or identify the malicious users who distribute misleading content like machine-generated fake news.
Deep Auto-encoder with Neural Response
In this study, we propose an integrated framework called Deep Autoencoder with Neural Response (DAE-NR), which incorporates information from ANN and the visual cortex to achieve better image reconstruction performance and higher neural representation similarity between biological and artificial neurons.
Adaptive Perturbation for Adversarial Attack
Instead of using the sign function, we propose to directly utilize the exact gradient direction with a scaling factor for generating adversarial perturbations, which improves the attack success rates of adversarial examples even with fewer perturbations.
Adaptive Image Transformations for Transfer-based Adversarial Attack
Adversarial attacks provide a good way to study the robustness of deep learning models.
3D Lip Event Detection via Interframe Motion Divergence at Multiple Temporal Resolutions
We define a motion divergence measure using 3D lip landmarks to quantify the interframe dynamics of a 3D speaking lip.
Learning an Adaptive Meta Model-Generator for Incrementally Updating Recommender Systems
Recommender Systems (RSs) in real-world applications often deal with billions of user interactions daily.
Parameterized Knowledge Transfer for Personalized Federated Learning
To deal with such model constraints, we exploit the potentials of heterogeneous model settings and propose a novel training framework to employ personalized models for different clients.
Learning Large Neighborhood Search Policy for Integer Programming
We then design a neural network to learn policies for each variable in parallel, trained by a customized actor-critic algorithm.
ICDM 2020 Knowledge Graph Contest: Consumer Event-Cause Extraction
To this end, we introduce a fresh perspective to revisit the relational event-cause extraction task and propose a novel sequence tagging framework, instead of extracting event types and events-causes separately.
SenseMag: Enabling Low-Cost Traffic Monitoring using Non-invasive Magnetic Sensing
Extensive experiment based on real-word field deployment (on the highways in Shenzhen, China) shows that SenseMag significantly outperforms the existing methods in both classification accuracy and the granularity of vehicle types (i. e., 7 types by SenseMag versus 4 types by the existing work in comparisons).
Cross-Vendor CT Image Data Harmonization Using CVH-CT
We propose a novel deep learning approach called CVH-CT for harmonizing CT images captured using scanners from different vendors.
NeuroLKH: Combining Deep Learning Model with Lin-Kernighan-Helsgaun Heuristic for Solving the Traveling Salesman Problem
We present NeuroLKH, a novel algorithm that combines deep learning with the strong traditional heuristic Lin-Kernighan-Helsgaun (LKH) for solving Traveling Salesman Problem.
MGC: A Complex-Valued Graph Convolutional Network for Directed Graphs
We test our model on several graph datasets including directed homogeneous and heterogeneous graphs.
M6-10T: A Sharing-Delinking Paradigm for Efficient Multi-Trillion Parameter Pretraining
Recent expeditious developments in deep learning algorithms, distributed training, and even hardware design for large models have enabled training extreme-scale models, say GPT-3 and Switch Transformer possessing hundreds of billions or even trillions of parameters.
Deep Reinforcement Learning for Solving the Heterogeneous Capacitated Vehicle Routing Problem
To solve those problems, we propose a DRL method based on the attention mechanism with a vehicle selection decoder accounting for the heterogeneous fleet constraint and a node selection decoder accounting for the route construction, which learns to construct a solution by automatically selecting both a vehicle and a node for this vehicle at each step.
Heterogeneous Attentions for Solving Pickup and Delivery Problem via Deep Reinforcement Learning
In particular, the heterogeneous attention mechanism specifically prescribes attentions for each role of the nodes while taking into account the precedence constraint, i. e., the pickup node must precede the pairing delivery node.
ACDC: Online Unsupervised Cross-Domain Adaptation
We consider the problem of online unsupervised cross-domain adaptation, where two independent but related data streams with different feature spaces -- a fully labeled source stream and an unlabeled target stream -- are learned together.
Generative Adversarial Training for Neural Combinatorial Optimization Models
Recent studies show that deep neural networks can be trained to learn good heuristics for various Combinatorial Optimization Problems (COPs).
Learning Scenario Representation for Solving Two-stage Stochastic Integer Programs
Many practical combinatorial optimization problems under uncertainty can be modeled as stochastic integer programs (SIPs), which are extremely challenging to solve due to the high complexity.
Pairwise Emotional Relationship Recognition in Drama Videos: Dataset and Benchmark
We expect the ERATO as well as our proposed SMTA to open up a new way for PERR task in video understanding and further improve the research of multi-modal fusion methodology.
LUAI Challenge 2021 on Learning to Understand Aerial Images
This report summarizes the results of Learning to Understand Aerial Images (LUAI) 2021 challenge held on ICCV 2021, which focuses on object detection and semantic segmentation in aerial images.
Optimisation of two-stage biomass gasification for hydrogen production via artificial neural network
The developed neural network model was then applied for optimising operating conditions of the two-stage gasification for high carbon conversion, high hydrogen yield and low carbon dioxide in nitrogen and carbon dioxide environments.
Meta Gradient Adversarial Attack
In recent years, research on adversarial attacks has become a hot spot.
Exploring Structure Consistency for Deep Model Watermarking
However, little attention has been devoted to the protection of DNNs in image processing tasks.
Poison Ink: Robust and Invisible Backdoor Attack
As the image structure can keep its semantic meaning during the data transformation, such trigger pattern is inherently robust to data transformations.
Locality-aware Channel-wise Dropout for Occluded Face Recognition
Furthermore, by randomly dropping out several feature channels, our method can well simulate the occlusion of larger area.
Supply Chain Digital Twin Framework Design: An Approach of Supply Chain Operations Reference Model and System of Systems
This paper explains what is 'twined' in supply chain digital twin and how to 'twin' them to handle the spatio-temporal dynamic issue.
Impact of Rotary-Wing UAV Wobbling on Millimeter-wave Air-to-Ground Wireless Channel
Our contributions of this paper lie in: i) modeling the wobbling process of a hovering RW UAV; ii) developing an analytical model to derive the channel temporal autocorrelation function (ACF) for the millimeter-wave RW UAV A2G link in a closed-form expression; and iii) investigating how RW UAV wobbling impacts the Doppler effect on the millimeter-wave RW UAV A2G link.
SelfCF: A Simple Framework for Self-supervised Collaborative Filtering
Collaborative filtering (CF) is widely used to learn informative latent representations of users and items from observed interactions.
CT Image Harmonization for Enhancing Radiomics Studies
While remarkable advances have been made in Computed Tomography (CT), capturing CT images with non-standardized protocols causes low reproducibility regarding radiomic features, forming a barrier on CT image analysis in a large scale.
From Personalized Medicine to Population Health: A Survey of mHealth Sensing Techniques
Mobile Sensing Apps have been widely used as a practical approach to collect behavioral and health-related information from individuals and provide timely intervention to promote health and well-beings, such as mental health and chronic cares.
MFR 2021: Masked Face Recognition Competition
These teams successfully submitted 18 valid solutions.
M6-T: Exploring Sparse Expert Models and Beyond
Mixture-of-Experts (MoE) models can achieve promising results with outrageous large amount of parameters but constant computation cost, and thus it has become a trend in model scaling.
HINet: Half Instance Normalization Network for Image Restoration
Specifically, we present a novel block: Half Instance Normalization Block (HIN Block), to boost the performance of image restoration networks.
Ranked #3 on
Single Image Deraining
on Test2800
Reversible Watermarking in Deep Convolutional Neural Networks for Integrity Authentication
Specifically, we present the reversible watermarking problem of deep convolutional neural networks and utilize the pruning theory of model compression technology to construct a host sequence used for embedding watermarking information by histogram shift.
