Search Results for author:
Found 78 papers, 27 papers with code
The NiuTrans’s Submission to the IWSLT22 English-to-Chinese Offline Speech Translation Task
This paper describes NiuTrans’s submission to the IWSLT22 English-to-Chinese (En-Zh) offline speech translation task.
The NiuTrans System for the WMT20 Quality Estimation Shared Task
This paper describes the submissions of the NiuTrans Team to the WMT 2020 Quality Estimation Shared Task.
The NiuTrans Machine Translation Systems for WMT20
This paper describes NiuTrans neural machine translation systems of the WMT20 news translation tasks.
The NiuTrans System for the WMT 2021 Efficiency Task
This paper describes the NiuTrans system for the WMT21 translation efficiency task.
Prior Constraints-based Reward Model Training for Aligning Large Language Models
Reinforcement learning with human feedback for aligning large language models (LLMs) trains a reward model typically using ranking loss with comparison pairs. However, the training procedure suffers from an inherent problem: the uncontrolled scaling of reward scores during reinforcement learning due to the lack of constraints while training the reward model. This paper proposes a Prior Constraints-based Reward Model (namely PCRM) training method to mitigate this problem.
Efficient Prompting Methods for Large Language Models: A Survey
In this paper, we present a comprehensive overview of these methods.
RankPrompt: Step-by-Step Comparisons Make Language Models Better Reasoners
Our experiments across 11 arithmetic and commonsense reasoning tasks show that RankPrompt significantly enhances the reasoning performance of ChatGPT and GPT-4, with improvements of up to 13%.
Large Language Models are Parallel Multilingual Learners
In this study, we reveal an in-context learning (ICL) capability of multilingual large language models (LLMs): by translating the input to several languages, we provide Parallel Input in Multiple Languages (PiM) to LLMs, which significantly enhances their comprehension abilities.
Soft Alignment of Modality Space for End-to-end Speech Translation
End-to-end Speech Translation (ST) aims to convert speech into target text within a unified model.
Introduction to Transformers: an NLP Perspective
Transformers have dominated empirical machine learning models of natural language processing.
Rethinking and Improving Multi-task Learning for End-to-end Speech Translation
Significant improvements in end-to-end speech translation (ST) have been achieved through the application of multi-task learning.
Incorporating Probing Signals into Multimodal Machine Translation via Visual Question-Answering Pairs
This paper presents an in-depth study of multimodal machine translation (MMT), examining the prevailing understanding that MMT systems exhibit decreased sensitivity to visual information when text inputs are complete.
PartialFormer: Modeling Part Instead of Whole
The design choices in Transformer feed-forward neural networks have resulted in significant computational and parameter overhead.
Ranked #23 on
Machine Translation
on WMT2014 English-German
Bridging the Gaps of Both Modality and Language: Synchronous Bilingual CTC for Speech Translation and Speech Recognition
In this study, we present synchronous bilingual Connectionist Temporal Classification (CTC), an innovative framework that leverages dual CTC to bridge the gaps of both modality and language in the speech translation (ST) task.
Learning Evaluation Models from Large Language Models for Sequence Generation
Large language models achieve state-of-the-art performance on sequence generation evaluation, but typically have a large number of parameters.
ESRL: Efficient Sampling-based Reinforcement Learning for Sequence Generation
Applying Reinforcement Learning (RL) to sequence generation models enables the direct optimization of long-term rewards (\textit{e. g.,} BLEU and human feedback), but typically requires large-scale sampling over a space of action sequences.
Towards Robust Aspect-based Sentiment Analysis through Non-counterfactual Augmentations
While state-of-the-art NLP models have demonstrated excellent performance for aspect based sentiment analysis (ABSA), substantial evidence has been presented on their lack of robustness.
Aspect-Based Sentiment Analysis
Aspect-Based Sentiment Analysis (ABSA)
+2
Recent Advances in Direct Speech-to-text Translation
Recently, speech-to-text translation has attracted more and more attention and many studies have emerged rapidly.
Understanding Parameter Sharing in Transformers
Inspired by this, we tune the training hyperparameters related to model convergence in a targeted manner.
Modality Adaption or Regularization? A Case Study on End-to-End Speech Translation
Pre-training and fine-tuning is a paradigm for alleviating the data scarcity problem in end-to-end speech translation (E2E ST).
MobileNMT: Enabling Translation in 15MB and 30ms
With the co-design of model and engine, compared with the existing system, we speed up 47. 0x and save 99. 5% of memory with only 11. 6% loss of BLEU.
Deliberate then Generate: Enhanced Prompting Framework for Text Generation
Large language models (LLMs) have shown remarkable success across a wide range of natural language generation tasks, where proper prompt designs make great impacts.
Augmenting Large Language Model Translators via Translation Memories
Using translation memories (TMs) as prompts is a promising approach to in-context learning of machine translation models.
CTC-based Non-autoregressive Speech Translation
Combining end-to-end speech translation (ST) and non-autoregressive (NAR) generation is promising in language and speech processing for their advantages of less error propagation and low latency.
Bridging the Granularity Gap for Acoustic Modeling
While Transformer has become the de-facto standard for speech, modeling upon the fine-grained frame-level features remains an open challenge of capturing long-distance dependencies and distributing the attention weights.
TranSFormer: Slow-Fast Transformer for Machine Translation
Learning multiscale Transformer models has been evidenced as a viable approach to augmenting machine translation systems.
Multi-Path Transformer is Better: A Case Study on Neural Machine Translation
For years the model performance in machine learning obeyed a power-law relationship with the model size.
Improved Knowledge Distillation for Pre-trained Language Models via Knowledge Selection
Knowledge distillation addresses the problem of transferring knowledge from a teacher model to a student model.
Prompting Neural Machine Translation with Translation Memories
Improving machine translation (MT) systems with translation memories (TMs) is of great interest to practitioners in the MT community.
EIT: Enhanced Interactive Transformer
In this paper, we propose a novel architecture, the Enhanced Interactive Transformer (EIT), to address the issue of head degradation in self-attention mechanisms.
Improving End-to-end Speech Translation by Leveraging Auxiliary Speech and Text Data
We present a method for introducing a text encoder into pre-trained end-to-end speech translation systems.
Learning Multiscale Transformer Models for Sequence Generation
In this work, we define those scales in different linguistic units, including sub-words, words and phrases.
On Vision Features in Multimodal Machine Translation
Previous work on multimodal machine translation (MMT) has focused on the way of incorporating vision features into translation but little attention is on the quality of vision models.
ODE Transformer: An Ordinary Differential Equation-Inspired Model for Sequence Generation
Inspired by this, we design a new architecture, {\it ODE Transformer}, which is analogous to the Runge-Kutta method that is well motivated in ODE.
The NiuTrans Machine Translation Systems for WMT21
This paper describes NiuTrans neural machine translation systems of the WMT 2021 news translation tasks.
The NiuTrans System for the WMT21 Efficiency Task
This paper describes the NiuTrans system for the WMT21 translation efficiency task (http://statmt. org/wmt21/efficiency-task. html).
The NiuTrans System for WNGT 2020 Efficiency Task
This paper describes the submissions of the NiuTrans Team to the WNGT 2020 Efficiency Shared Task.
RankNAS: Efficient Neural Architecture Search by Pairwise Ranking
This paper addresses the efficiency challenge of Neural Architecture Search (NAS) by formulating the task as a ranking problem.
Bag of Tricks for Optimizing Transformer Efficiency
Improving Transformer efficiency has become increasingly attractive recently.
The NiuTrans End-to-End Speech Translation System for IWSLT 2021 Offline Task
This paper describes the submission of the NiuTrans end-to-end speech translation system for the IWSLT 2021 offline task, which translates from the English audio to German text directly without intermediate transcription.
Stacked Acoustic-and-Textual Encoding: Integrating the Pre-trained Models into Speech Translation Encoders
To our knowledge, we are the first to develop an end-to-end ST system that achieves comparable or even better BLEU performance than the cascaded ST counterpart when large-scale ASR and MT data is available.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
ODE Transformer: An Ordinary Differential Equation-Inspired Model for Neural Machine Translation
We show that a residual block of layers in Transformer can be described as a higher-order solution to ODEs.
An Efficient Transformer Decoder with Compressed Sub-layers
The large attention-based encoder-decoder network (Transformer) has become prevailing recently due to its effectiveness.
Learning Light-Weight Translation Models from Deep Transformer
We proposed a novel group-permutation based knowledge distillation approach to compressing the deep Transformer model into a shallow model.
A Simple and Effective Approach to Robust Unsupervised Bilingual Dictionary Induction
Our experiments show that this simple method does not hamper the performance of similar language pairs and achieves an accuracy of 13. 64~55. 53% between English and four distant languages, i. e., Chinese, Japanese, Vietnamese and Thai.
Dynamic Curriculum Learning for Low-Resource Neural Machine Translation
This eases training by highlighting easy samples that the current model has enough competence to learn.
Layer-Wise Multi-View Learning for Neural Machine Translation
In this way, in addition to the topmost encoder layer (referred to as the primary view), we also incorporate an intermediate encoder layer as the auxiliary view.
Training Flexible Depth Model by Multi-Task Learning for Neural Machine Translation
The standard neural machine translation model can only decode with the same depth configuration as training.
Shallow-to-Deep Training for Neural Machine Translation
We find that stacking layers is helpful in improving the representation ability of NMT models and adjacent layers perform similarly.
Weight Distillation: Transferring the Knowledge in Neural Network Parameters
Inspired by this, we investigate methods of model acceleration and compression in another line of research.
Towards Fully 8-bit Integer Inference for the Transformer Model
8-bit integer inference, as a promising direction in reducing both the latency and storage of deep neural networks, has made great progress recently.
Does Multi-Encoder Help? A Case Study on Context-Aware Neural Machine Translation
In encoder-decoder neural models, multiple encoders are in general used to represent the contextual information in addition to the individual sentence.
Learning Architectures from an Extended Search Space for Language Modeling
Neural architecture search (NAS) has advanced significantly in recent years but most NAS systems restrict search to learning architectures of a recurrent or convolutional cell.
Multi-layer Representation Fusion for Neural Machine Translation
In this paper, we propose a multi-layer representation fusion (MLRF) approach to fusing stacked layers.
Neural Machine Translation with Joint Representation
Though early successes of Statistical Machine Translation (SMT) systems are attributed in part to the explicit modelling of the interaction between any two source and target units, e. g., alignment, the recent Neural Machine Translation (NMT) systems resort to the attention which partially encodes the interaction for efficiency.
Improved Differentiable Architecture Search for Language Modeling and Named Entity Recognition
In this paper, we study differentiable neural architecture search (NAS) methods for natural language processing.
Ranked #1 on
Language Modelling
on PTB Diagnostic ECG Database
The NiuTrans Machine Translation Systems for WMT19
We participated in 13 translation directions, including 11 supervised tasks, namely EN↔{ZH, DE, RU, KK, LT}, GU→EN and the unsupervised DE↔CS sub-track.
Sharing Attention Weights for Fast Transformer
This is even 16 times faster than the baseline with no use of the attention cache.
Shared-Private Bilingual Word Embeddings for Neural Machine Translation
For similar source and target words, their embeddings tend to share a part of the features and they cooperatively learn these common representation units.
Learning Deep Transformer Models for Machine Translation
Transformer is the state-of-the-art model in recent machine translation evaluations.
The NiuTrans Machine Translation System for WMT18
This paper describes the submission of the NiuTrans neural machine translation system for the WMT 2018 Chinese ↔ English news translation tasks.
A Simple and Effective Approach to Coverage-Aware Neural Machine Translation
We offer a simple and effective method to seek a better balance between model confidence and length preference for Neural Machine Translation (NMT).
Towards Bidirectional Hierarchical Representations for Attention-Based Neural Machine Translation
This paper proposes a hierarchical attentional neural translation model which focuses on enhancing source-side hierarchical representations by covering both local and global semantic information using a bidirectional tree-based encoder.
