Search Results for author:
Found 66 papers, 26 papers with code
JMMMU: A Japanese Massive Multi-discipline Multimodal Understanding Benchmark for Culture-aware Evaluation
Accelerating research on Large Multimodal Models (LMMs) in non-English languages is crucial for enhancing user experiences across broader populations.
FoodMLLM-JP: Leveraging Multimodal Large Language Models for Japanese Recipe Generation
Research on food image understanding using recipe data has been a long-standing focus due to the diversity and complexity of the data.
Training-Free Sketch-Guided Diffusion with Latent Optimization
We introduce latent optimization, a method that refines the noisy latent at each intermediate step of the generation process using cross-attention maps to ensure that the generated images closely adhere to the desired structure outlined in the reference sketch.
MangaUB: A Manga Understanding Benchmark for Large Multimodal Models
MangaUB is designed to assess the recognition and understanding of content shown in a single panel as well as conveyed across multiple panels, allowing for a fine-grained analysis of a model's various capabilities required for manga understanding.
Zero-Shot Character Identification and Speaker Prediction in Comics via Iterative Multimodal Fusion
To address this problem, we propose an iterative multimodal framework, the first to employ multimodal information for both character identification and speaker prediction tasks.
Unsolvable Problem Detection: Evaluating Trustworthiness of Vision Language Models
This paper introduces a novel and significant challenge for Vision Language Models (VLMs), termed Unsolvable Problem Detection (UPD).
Entity-NeRF: Detecting and Removing Moving Entities in Urban Scenes
Recent advancements in the study of Neural Radiance Fields (NeRF) for dynamic scenes often involve explicit modeling of scene dynamics.
Cross-Lingual Learning in Multilingual Scene Text Recognition
We aim to find the condition that exploits knowledge from high-resource languages for improving performance in low-resource languages.
The Lottery Ticket Hypothesis in Denoising: Towards Semantic-Driven Initialization
Text-to-image diffusion models allow users control over the content of generated images.
Retrieval-Augmented Layout Transformer for Content-Aware Layout Generation
We show that a simple retrieval augmentation can significantly improve the generation quality.
Can Pre-trained Networks Detect Familiar Out-of-Distribution Data?
We consider that such data may significantly affect the performance of large pre-trained networks because the discriminability of these OOD data depends on the pre-training algorithm.
Out-of-Distribution Detection
Out of Distribution (OOD) Detection
Open-Set Domain Adaptation with Visual-Language Foundation Models
Unsupervised domain adaptation (UDA) has proven to be very effective in transferring knowledge obtained from a source domain with labeled data to a target domain with unlabeled data.
Manga109Dialog: A Large-scale Dialogue Dataset for Comics Speaker Detection
For further understanding of comics, an automated approach is needed to link text in comics to characters speaking the words.
LoCoOp: Few-Shot Out-of-Distribution Detection via Prompt Learning
CLIP's local features have a lot of ID-irrelevant nuisances (e. g., backgrounds), and by learning to push them away from the ID class text embeddings, we can remove the nuisances in the ID class text embeddings and enhance the separation between ID and OOD.
Out-of-Distribution Detection
Out of Distribution (OOD) Detection
Guided Image Synthesis via Initial Image Editing in Diffusion Model
Diffusion models have the ability to generate high quality images by denoising pure Gaussian noise images.
GL-MCM: Global and Local Maximum Concept Matching for Zero-Shot Out-of-Distribution Detection
Zero-shot out-of-distribution (OOD) detection is a task that detects OOD images during inference with only in-distribution (ID) class names.
Comprehensive Comparisons of Uniform Quantization in Deep Image Compression
The experimental results reveal that the best approximated quantization differs by the network architectures, and the best approximations of the three are different from the original ones used for the architectures.
Non-uniform Sampling Strategies for NeRF on 360{\textdegree} images
We propose two non-uniform ray sampling schemes for NeRF to suit 360{\textdegree} images - distortion-aware ray sampling and content-aware ray sampling.
Universal Deep Image Compression via Content-Adaptive Optimization with Adapters
This task aims to compress images belonging to arbitrary domains, such as natural images, line drawings, and comics.
Rethinking Rotation in Self-Supervised Contrastive Learning: Adaptive Positive or Negative Data Augmentation
The semantics of an image can be rotation-invariant or rotation-variant, so whether the rotated image is treated as positive or negative should be determined based on the content of the image.
Saliency-based Multiple Region of Interest Detection from a Single 360° image
360{\deg} images are informative -- it contains omnidirectional visual information around the camera.
Evaluating the Stability of Deep Image Quality Assessment With Respect to Image Scaling
We comprehensively evaluate four deep IQAs on the same five datasets, and the experimental results show that image scale significantly influences IQA performance.
COO: Comic Onomatopoeia Dataset for Recognizing Arbitrary or Truncated Texts
To encourage research on this topic, we provide a novel comic onomatopoeia dataset (COO), which consists of onomatopoeia texts in Japanese comics.
SVG Vector Font Generation for Chinese Characters with Transformer
Designing fonts for Chinese characters is highly labor-intensive and time-consuming.
Intersection Prediction from Single 360° Image via Deep Detection of Possible Direction of Travel
We constructed a large-scale 360{\deg} Image Intersection Identification (iii360) dataset for training and evaluation where 360{\deg} videos were collected from various areas such as school campus, downtown, suburb, and china town and demonstrate that our PDoT-based method achieves 88\% accuracy, which is significantly better than that achieved by the direct naive binary classification based method.
Distortion-Aware Self-Supervised 360° Depth Estimation from A Single Equirectangular Projection Image
We propose a framework of direct use of ERP with coordinate conversion of correspondences and distortion-aware upsampling module to deal with the ERP related problems and extend a self-supervised learning method for open environments.
Field-of-View IoU for Object Detection in 360° Images
360{\deg} cameras have gained popularity over the last few years.
Noisy Annotation Refinement for Object Detection
In this study, we propose a new problem setting of training object detectors on datasets with entangled noises of annotations of class labels and bounding boxes.
NTIRE 2021 Challenge on Perceptual Image Quality Assessment
This paper reports on the NTIRE 2021 challenge on perceptual image quality assessment (IQA), held in conjunction with the New Trends in Image Restoration and Enhancement workshop (NTIRE) workshop at CVPR 2021.
A Novel Perspective for Positive-Unlabeled Learning via Noisy Labels
Positive-unlabeled learning refers to the process of training a binary classifier using only positive and unlabeled data.
What If We Only Use Real Datasets for Scene Text Recognition? Toward Scene Text Recognition With Fewer Labels
To the best of our knowledge, this is the first study that 1) shows sufficient performance by only using real labels and 2) introduces semi- and self-supervised methods into STR with fewer labels.
Building Movie Map -- A Tool for Exploring Areas in a City -- and its Evaluation
Frames of the video are localized on the map, intersections are detected, and videos are segmented.
Few-Shot Font Generation with Deep Metric Learning
Designing fonts for languages with a large number of characters, such as Japanese and Chinese, is an extremely labor-intensive and time-consuming task.
The Aleatoric Uncertainty Estimation Using a Separate Formulation with Virtual Residuals
We propose a new optimization framework for aleatoric uncertainty estimation in regression problems.
SLGAN: Style- and Latent-guided Generative Adversarial Network for Desirable Makeup Transfer and Removal
Furthermore, we show that our proposal can interpolate facial makeup images to determine the unique features, compare existing methods, and help users find desirable makeup configurations.
Multi-Task Curriculum Framework for Open-Set Semi-Supervised Learning
Semi-supervised learning (SSL) has been proposed to leverage unlabeled data for training powerful models when only limited labeled data is available.
Channel-Level Variable Quantization Network for Deep Image Compression
In this paper, we propose a channel-level variable quantization network to dynamically allocate more bitrates for significant channels and withdraw bitrates for negligible channels.
Building a Manga Dataset "Manga109" with Annotations for Multimedia Applications
Manga, or comics, which are a type of multimodal artwork, have been left behind in the recent trend of deep learning applications because of the lack of a proper dataset.
Distance Surface for Event-Based Optical Flow
We propose DistSurf-OF, a novel optical flow method for neuromorphic cameras.
Unsupervised Out-of-Distribution Detection by Maximum Classifier Discrepancy
Unlike previous methods, we also utilize unlabeled data for unsupervised training and we use these unlabeled data to maximize the discrepancy between the decision boundaries of two classifiers to push OOD samples outside the manifold of the in-distribution (ID) samples, which enables us to detect OOD samples that are far from the support of the ID samples.
Out-of-Distribution Detection
Out of Distribution (OOD) Detection
Object-Aware Instance Labeling for Weakly Supervised Object Detection
Weakly supervised object detection (WSOD), where a detector is trained with only image-level annotations, is attracting more and more attention.
TriDepth: Triangular Patch-based Deep Depth Prediction
We propose a novel and efficient representation for single-view depth estimation using Convolutional Neural Networks (CNNs).
Computational Attention System for Children, Adults and Elderly
Observer's of different age-group have shown different scene viewing tendencies independent to the class of the image viewed.
Recognition of Multiple Food Items in a Single Photo for Use in a Buffet-Style Restaurant
We investigate image recognition of multiple food items in a single photo, focusing on a buffet restaurant application, where menu changes at every meal, and only a few images per class are available.
Context-Patch Face Hallucination Based on Thresholding Locality-constrained Representation and Reproducing Learning
To this end, this study incorporates the contextual information of image patch and proposes a powerful and efficient context-patch based face hallucination approach, namely Thresholding Locality-constrained Representation and Reproducing learning (TLcR-RL).
Scale Drift Correction of Camera Geo-Localization using Geo-Tagged Images
Camera geo-localization from a monocular video is a fundamental task for video analysis and autonomous navigation.
Local and Global Optimization Techniques in Graph-Based Clustering
We propose a local optimization method, which is widely applicable to graph-based clustering cost functions.
Fast and Robust Estimation for Unit-Norm Constrained Linear Fitting Problems
M-estimator using iteratively reweighted least squares (IRLS) is one of the best-known methods for robust estimation.
Category-Based Deep CCA for Fine-Grained Venue Discovery from Multimodal Data
Given a photo as input, this model performs (i) exact venue search (find the venue where the photo was taken), and (ii) group venue search (find relevant venues with the same category as that of the photo), by the cross-modal correlation between the input photo and textual description of venues.
Personalized Classifier for Food Image Recognition
In this paper, we address the personalization problem, which involves adapting to the user's domain incrementally using a very limited number of samples.
Cross-Domain Weakly-Supervised Object Detection through Progressive Domain Adaptation
Can we detect common objects in a variety of image domains without instance-level annotations?
Ranked #5 on
Weakly Supervised Object Detection
on Watercolor2k
(using extra training data)
Parallel Grid Pooling for Data Augmentation
Convolutional neural network (CNN) architectures utilize downsampling layers, which restrict the subsequent layers to learn spatially invariant features while reducing computational costs.
Joint Optimization Framework for Learning with Noisy Labels
Deep neural networks (DNNs) trained on large-scale datasets have exhibited significant performance in image classification.
Ranked #39 on
Image Classification
on Clothing1M
Object Detection for Comics using Manga109 Annotations
We annotated an existing image dataset of comics and created the largest annotation dataset, named Manga109-annotations.
Significance of Softmax-based Features in Comparison to Distance Metric Learning-based Features
However, in these DML studies, there were no equitable comparisons between features extracted from a DML-based network and those from a softmax-based network.
PQk-means: Billion-scale Clustering for Product-quantized Codes
Data clustering is a fundamental operation in data analysis.
Spatio-Temporal Vector of Locally Max Pooled Features for Action Recognition in Videos
The proposed method addresses an important problem of video understanding: how to build a video representation that incorporates the CNN features over the entire video.
Action Recognition In Videos
Temporal Action Localization
+1
cGAN-based Manga Colorization Using a Single Training Image
The final results are sharp, clear, and in high resolution, and stay true to the character's original color scheme.
Residual Expansion Algorithm: Fast and Effective Optimization for Nonconvex Least Squares Problems
We propose the residual expansion (RE) algorithm: a global (or near-global) optimization method for nonconvex least squares problems.
Gaze Distribution Analysis and Saliency Prediction Across Age Groups
In this paper, we investigated how visual scene processing changes with age and we propose an age-adapted framework that helps to develop a computational model that can predict saliency across different age groups.
PQTable: Non-exhaustive Fast Search for Product-quantized Codes using Hash Tables
In this paper, we propose a product quantization table (PQTable); a fast search method for product-quantized codes via hash-tables.
Uncalibrated Photometric Stereo by Stepwise Optimization Using Principal Components of Isotropic BRDFs
We propose a model that represents various isotropic reflectance functions by using the principal components of items in a dataset, and formulate the uncalibrated photometric stereo as a regression problem.
PQTable: Fast Exact Asymmetric Distance Neighbor Search for Product Quantization Using Hash Tables
We propose the product quantization table (PQTable), a product quantization-based hash table that is fast and requires neither parameter tuning nor training steps.
Sketch-based Manga Retrieval using Manga109 Dataset
From the experiments, we verified that: (1) the retrieval accuracy of the proposed method is higher than those of previous methods; (2) the proposed method can localize an object instance with reasonable runtime and accuracy; and (3) sketch querying is useful for manga search.
Photometric Stereo using Constrained Bivariate Regression for General Isotropic Surfaces
This paper presents a photometric stereo method that is purely pixelwise and handles general isotropic surfaces in a stable manner.
