Distilling Linguistic Context for Language Model Compression

EMNLP 2021 Distilling Linguistic Context for Language Model Compression Geondo Park, Gyeongman Kim, Eunho Yang  A computationally expensive and memory intensive neural network lies behind the recent success of language representation learning. Knowledge distillation, a major tech...

Contrastive Learning with Adversarial Perturbations for Conditional Text Generation

ICLR 2021 Contrastive Learning with Adversarial Perturbations for Conditional Text Generation Seanie Lee, Dong Bok Lee and Sung Ju Hwang  Recently, sequence-to-sequence (seq2seq) models with the Transformer architecture have achieved remarkable performance on various conditional ...

Cluster-Promoting Quantization with Bit-Drop for Minimizing Network Quantization Loss

ICCV 2021 Cluster-Promoting Quantization with Bit-Drop for Minimizing Network Quantization Loss Jung Hyun Lee, Jihun Yun, Sung Ju Hwang and Eunho Yang  Network quantization, which aims to reduce the bitlengths of the network weights and activations, has emerged for their deployme...

Clinical Risk Prediction with Temporal Probabilistic Asymmetric Multi-Task Learning

AAAI 2021 Clinical Risk Prediction with Temporal Probabilistic Asymmetric Multi-Task Learning Tuan Nguyen*, Hyewon Jeong*, Eunho Yang and Sung Ju Hwang  Although recent multi-task learning methods have shown to be effective in improving the generalization of deep n...

Adversarial Purification with Score-based Generative Models

ICML 2021 Adversarial Purification with Score-based Generative Models Jongmin Yoon, Sung Ju Hwang and Juho Lee  While adversarial training is considered as a standard defense method against adversarial attacks for image classifiers, adversarial purification, which purifies attack...

Adaptive Proximal Gradient Methods for Structured Neural Networks

NeurIPS 2021 Adaptive Proximal Gradient Methods for Structured Neural Networks Jihun Yun, Aurelie C. Lozano, Eunho Yang  We consider the training of structured neural networks where the regularizer can be non-smooth and possibly non-convex. While popular machine learning librarie...

Accurate Learning of Graph Representations with Graph Multiset Pooling

ICLR 2021 Accurate Learning of Graph Representations with Graph Multiset Pooling Jinheon Baek, Minki Kang and Sung Ju Hwang  Message-passing graph neural networks have been widely used on modeling graph data, achieving impressive results on a number of graph classification and li...

Why Not to Use Zero Imputation? Correcting Sparsity Bias in Training Neural Networks

ICLR 2020 Why Not to Use Zero Imputation? Correcting Sparsity Bias in Training Neural Networks Joonyoung Yi, Juhyuk Lee, Kwang Joon Kim, Sung Ju Hwang, Eunho Yang  Handling missing data is one of the most fundamental problems in machine learning. Among many approaches, ...

Self-supervised Label Augmentation via Input Transformations

ICML 2020 Self-supervised Label Augmentation via Input Transformations Hankook Lee, Sung Ju Hwang, Jinwoo Shin  Self-supervised learning, which learns by constructing artificial labels given only the input signals, has recently gained considerable attention for learning repr...

Scalable and Order-robust Continual Learning with Additive Parameter Decomposition

ICLR 2020 Scalable and Order-robust Continual Learning with Additive Parameter Decomposition Jaehong Yoon, Saehoon Kim, Eunho Yang, Sung Ju Hwang  While recent continual learning methods largely alleviate the catastrophic problem on toy-sized datasets, there are issues that ...