Search Results for author:
Found 48 papers, 20 papers with code
SALICON: Saliency in Context
Saliency in Context (SALICON) is an ongoing effort that aims at understanding and predicting visual attention.
A Paradigm for Building Generalized Models of Human Image Perception Through Data Fusion
(3) It can guide the design of a generalized computational algorithm for multi-dimensional visual perception.
Says Who\ldots? Identification of Expert versus Layman Critics' Reviews of Documentary Films
We extend classic review mining work by building a binary classifier that predicts whether a review of a documentary film was written by an expert or a layman with 90. 70{\%} accuracy (F1 score), and compare the characteristics of the predicted classes.
Learning Visual Attention to Identify People With Autism Spectrum Disorder
This paper presents a novel method for quantitative and objective diagnoses of Autism Spectrum Disorder (ASD) using eye tracking and deep neural networks.
Beyond Trade-off: Accelerate FCN-based Face Detector with Higher Accuracy
Fully convolutional neural network (FCN) has been dominating the game of face detection task for a few years with its congenital capability of sliding-window-searching with shared kernels, which boiled down all the redundant calculation, and most recent state-of-the-art methods such as Faster-RCNN, SSD, YOLO and FPN use FCN as their backbone.
Emotional Attention: A Study of Image Sentiment and Visual Attention
In this paper, we present the first study to focus on the relation between emotional properties of an image and visual attention.
A Neural Network Aided Approach for LDPC Coded DCO-OFDM with Clipping Distortion
In this paper, a neural network-aided bit-interleaved coded modulation (NN-BICM) receiver is designed to mitigate the nonlinear clipping distortion in the LDPC coded direct currentbiased optical orthogonal frequency division multiplexing (DCOOFDM) systems.
Which Has Better Visual Quality: The Clear Blue Sky or a Blurry Animal?
The proposed method, SFA, is compared with nine representative blur-specific NR-IQA methods, two general-purpose NR-IQA methods, and two extra full-reference IQA methods on Gaussian blur images (with and without Gaussian noise/JPEG compression) and realistic blur images from multiple databases, including LIVE, TID2008, TID2013, MLIVE1, MLIVE2, BID, and CLIVE.
Exploiting High-Level Semantics for No-Reference Image Quality Assessment of Realistic Blur Images
To guarantee a satisfying Quality of Experience (QoE) for consumers, it is required to measure image quality efficiently and reliably.
Quality Assessment for Tone-Mapped HDR Images Using Multi-Scale and Multi-Layer Information
So we propose a new no-reference method of tone-mapped image quality assessment based on multi-scale and multi-layer features that are extracted from a pre-trained deep convolutional neural network model.
Blind Image Quality Assessment
No-Reference Image Quality Assessment
Multimedia
Parsing R-CNN for Instance-Level Human Analysis
Models need to distinguish different human instances in the image panel and learn rich features to represent the details of each instance.
Ranked #1 on
Pose Estimation
on DensePose-COCO
Single Image Blind Deblurring Using Multi-Scale Latent Structure Prior
Blind image deblurring is a challenging problem in computer vision, which aims to restore both the blur kernel and the latent sharp image from only a blurry observation.
Quality Assessment of In-the-Wild Videos
We propose an objective no-reference video quality assessment method by integrating both effects into a deep neural network.
Ranked #6 on
Video Quality Assessment
on MSU NR VQA Database
TIGEr: Text-to-Image Grounding for Image Caption Evaluation
This paper presents a new metric called TIGEr for the automatic evaluation of image captioning systems.
REO-Relevance, Extraness, Omission: A Fine-grained Evaluation for Image Captioning
In this study, we present a fine-grained evaluation method REO for automatically measuring the performance of image captioning systems.
LabelFool: A Trick in the Label Space
It is widely known that well-designed perturbations can cause state-of-the-art machine learning classifiers to mis-label an image, with sufficiently small perturbations that are imperceptible to the human eyes.
A Constituency Parsing Tree based Method for Relation Extraction from Abstracts of Scholarly Publications
We present a simple, rule-based method for extracting entity networks from the abstracts of scientific literature.
Lossless Attention in Convolutional Networks for Facial Expression Recognition in the Wild
And we validate our method on the Aff-Wild2 datasets released by the Challenge.
Facial Expression Recognition
Facial Expression Recognition (FER)
Improving Scholarly Knowledge Representation: Evaluating BERT-based Models for Scientific Relation Classification
With the rapid growth of research publications, there is a vast amount of scholarly knowledge that needs to be organized in digital libraries.
Fantastic Answers and Where to Find Them: Immersive Question-Directed Visual Attention
The Immersive Question-directed Visual Attention (IQVA) dataset features visual attention and corresponding task performance (i. e., answer correctness).
Leveraging Bottom-Up and Top-Down Attention for Few-Shot Object Detection
Few-shot object detection aims at detecting objects with few annotated examples, which remains a challenging research problem yet to be explored.
Saliency Prediction with External Knowledge
At the core of the method is a new Graph Semantic Saliency Network (GraSSNet) that constructs a graph that encodes semantic relationships learned from external knowledge.
AiR: Attention with Reasoning Capability
In this work, we propose an Attention with Reasoning capability (AiR) framework that uses attention to understand and improve the process leading to task outcomes.
Norm-in-Norm Loss with Faster Convergence and Better Performance for Image Quality Assessment
Experiments on two relevant datasets (KonIQ-10k and CLIVE) show that, compared to MAE or MSE loss, the new loss enables the IQA model to converge about 10 times faster and the final model achieves better performance.
Ranked #2 on
Image Quality Assessment
on MSU NR VQA Database
Blind Image Quality Assessment
No-Reference Image Quality Assessment
+1
Unified Quality Assessment of In-the-Wild Videos with Mixed Datasets Training
We focus on automatically assessing the quality of in-the-wild videos, which is a challenging problem due to the absence of reference videos, the complexity of distortions, and the diversity of video contents.
Ranked #1 on
Video Quality Assessment
on MSU NR VQA Database
Self-Distillation for Few-Shot Image Captioning
We propose an ensemble-based self-distillation method that allows image captioning models to be trained with unpaired images and captions.
A Portable, Self-Contained Neuroprosthetic Hand with Deep Learning-Based Finger Control
This enables the implementation of the neuroprosthetic hand as a portable and self-contained unit with real-time control of individual finger movements.
Predicting Human Scanpaths in Visual Question Answering
Conditioned on a task guidance map, the proposed model learns question-specific attention patterns to generate scanpaths.
Explicit Knowledge Incorporation for Visual Reasoning
Existing explainable and explicit visual reasoning methods only perform reasoning based on visual evidence but do not take into account knowledge beyond what is in the visual scene.
Leveraging Human Attention in Novel Object Captioning
Image captioning models depend on training with paired image-text corpora, which poses various challenges in describing images containing novel objects absent from the training data.
A Speaker-aware Parallel Hierarchical Attentive Encoder-Decoder Model for Multi-turn Dialogue Generation
Differing from prior work that solely relies on the content of conversation history to generate a response, we argue that capturing relative social relations among utterances (i. e., generated by either the same speaker or different persons) benefits the machine capturing fine-grained context information from a conversation history to improve context coherence in the generated response.
VisualHow: Multimodal Problem Solving
With an overarching goal of developing intelligent systems to assist humans in various daily activities, we propose VisualHow, a free-form and open-ended research that focuses on understanding a real-life problem and deriving its solution by incorporating key components across multiple modalities.
Query and Attention Augmentation for Knowledge-Based Explainable Reasoning
Explainable visual question answering (VQA) models have been developed with neural modules and query-based knowledge incorporation to answer knowledge-requiring questions.
Artificial Intelligence Enables Real-Time and Intuitive Control of Prostheses via Nerve Interface
Objective: The next generation prosthetic hand that moves and feels like a real hand requires a robust neural interconnection between the human minds and machines.
Attention in Reasoning: Dataset, Analysis, and Modeling
In this work, we propose an Attention with Reasoning capability (AiR) framework that uses attention to understand and improve the process leading to task outcomes.
Cross-Modality Gated Attention Fusion for Multimodal Sentiment Analysis
Multimodal sentiment analysis is an important research task to predict the sentiment score based on the different modality data from a specific opinion video.
Proportionate Recursive Maximum Correntropy Criterion Adaptive Filtering Algorithms and their Performance Analysis
Motivated by the success of our recently proposed proportionate recursive least squares (PRLS) algorithm for sparse system identification, we propose to introduce the proportionate updating (PU) mechanism into the RMCC, leading to two sparsity-aware RMCC algorithms: the proportionate recursive MCC (PRMCC) algorithm and the combinational PRMCC (CPRMCC) algorithm.
Evaluating BERT-based Scientific Relation Classifiers for Scholarly Knowledge Graph Construction on Digital Library Collections
To address these limitations, we started by creating OCR-noisy texts based on three clean corpora.
Benchmarking LLM-based Machine Translation on Cultural Awareness
Furthermore, we devise a novel evaluation metric to assess the understandability of translations in a reference-free manner by GPT-4.
Key Gene Mining in Transcriptional Regulation for Specific Biological Processes with Small Sample Sizes Using Multi-network pipeline Transformer
Gene mining is an important topic in the field of life sciences, but traditional machine learning methods cannot consider the regulatory relationships between genes.
What Do Deep Saliency Models Learn about Visual Attention?
In recent years, deep saliency models have made significant progress in predicting human visual attention.
CFBenchmark: Chinese Financial Assistant Benchmark for Large Language Model
Large language models (LLMs) have demonstrated great potential in the financial domain.
CPopQA: Ranking Cultural Concept Popularity by LLMs
In this study, we introduce a novel few-shot question-answering task (CPopQA) that examines LLMs' statistical ranking abilities for long-tail cultural concepts (e. g., holidays), with a specific focus on these concepts' popularity in the United States and the United Kingdom, respectively.
Prompting Large Vision-Language Models for Compositional Reasoning
Vision-language models such as CLIP have shown impressive capabilities in encoding texts and images into aligned embeddings, enabling the retrieval of multimodal data in a shared embedding space.
Ranked #22 on
Visual Reasoning
on Winoground
GAN Based Near-Field Channel Estimation for Extremely Large-Scale MIMO Systems
The mixed line-of-sight (LoS) and non-line-of-sight (NLoS) XL-MIMO near-field channel model is adopted to describe the XL-MIMO near-field channel accurately.
Lookahead Exploration with Neural Radiance Representation for Continuous Vision-Language Navigation
Vision-and-language navigation (VLN) enables the agent to navigate to a remote location following the natural language instruction in 3D environments.
Beyond Average: Individualized Visual Scanpath Prediction
Understanding how attention varies across individuals has significant scientific and societal impacts.
