Search Results for author:
Found 61 papers, 28 papers with code
STIV: Scalable Text and Image Conditioned Video Generation
The field of video generation has made remarkable advancements, yet there remains a pressing need for a clear, systematic recipe that can guide the development of robust and scalable models.
Chasing Better Deep Image Priors between Over- and Under-parameterization
Besides, we also extend LIP to compressive sensing image reconstruction, where a pre-trained GAN generator is used as the prior (in contrast to untrained DIP or deep decoder), and confirm its validity in this setting too.
Cavia: Camera-controllable Multi-view Video Diffusion with View-Integrated Attention
Recent studies have attempted to incorporate camera control into the generation process, but their results are often limited to simple trajectories or lack the ability to generate consistent videos from multiple distinct camera paths for the same scene.
RED QUEEN: Safeguarding Large Language Models against Concealed Multi-Turn Jailbreaking
Our experiments reveal that all LLMs are vulnerable to RED QUEEN ATTACK, reaching 87. 62% attack success rate on GPT-4o and 75. 4% on Llama3-70B.
COLUMBUS: Evaluating COgnitive Lateral Understanding through Multiple-choice reBUSes
Then, we develop COLUMBUS, a synthetic benchmark that applies the task pipeline to create QA sets with text and icon rebus puzzles based on publicly available collections of compounds and common phrases.
Error-controlled non-additive interaction discovery in machine learning models
We further address the challenges of using off-the-shelf interaction importance measures by proposing a calibration procedure that refines these measures to maintain the desired FDR.
Energy-Aware UAV-Enabled Target Tracking: Online Optimization with Location Constraints
Specifically, our approach designs the UAV trajectory by solving two subproblems: the candidate trajectory optimization problem and the energy-aware backup trajectory optimization problem.
Sustainable Wireless Networks via Reconfigurable Intelligent Surfaces (RISs): Overview of the ETSI ISG RIS
Reconfigurable Intelligent Surfaces (RISs) are a novel form of ultra-low power devices that are capable to increase the communication data rates as well as the cell coverage in a cost- and energy-efficient way.
SemEval-2024 Task 9: BRAINTEASER: A Novel Task Defying Common Sense
In this paper, we split the original benchmark to also support fine-tuning setting and present SemEval Task 9: BRAIN-TEASER(S), the first task at this competition designed to test the system's reasoning and lateral thinking ability.
MARVEL: Multidimensional Abstraction and Reasoning through Visual Evaluation and Learning
Further analysis of perception questions reveals that MLLMs struggle to comprehend the visual features (near-random performance) and even count the panels in the puzzle ( <45%), hindering their ability for abstract reasoning.
TrafPS: A Shapley-based Visual Analytics Approach to Interpret Traffic
Recent achievements in deep learning (DL) have shown its potential for predicting traffic flows.
The Curious Case of Nonverbal Abstract Reasoning with Multi-Modal Large Language Models
While large language models (LLMs) are still being adopted to new domains and utilized in novel applications, we are experiencing an influx of the new generation of foundation models, namely multi-modal large language models (MLLMs).
UAV-enabled Integrated Sensing and Communication: Tracking Design and Optimization
Specifically, a weighted sum of predicted posterior Cram\'er-Rao bound (PCRB) for object relative position and velocity estimation is minimized by optimizing the UAV trajectory, where an efficient solution is obtained based on the successive convex approximation method.
VASE: Object-Centric Appearance and Shape Manipulation of Real Videos
Recently, several works tackled the video editing task fostered by the success of large-scale text-to-image generative models.
PAIR Diffusion: A Comprehensive Multimodal Object-Level Image Editor
We propose PAIR Diffusion a generic framework that enables a diffusion model to control the structure and appearance properties of each object in the image.
Efficient-NeRF2NeRF: Streamlining Text-Driven 3D Editing with Multiview Correspondence-Enhanced Diffusion Models
In this work, we propose that by incorporating correspondence regularization into diffusion models, the process of 3D editing can be significantly accelerated.
FSGS: Real-Time Few-shot View Synthesis using Gaussian Splatting
Novel view synthesis from limited observations remains an important and persistent task.
BRAINTEASER: Lateral Thinking Puzzles for Large Language Models
The success of language models has inspired the NLP community to attend to tasks that require implicit and complex reasoning, relying on human-like commonsense mechanisms.
Pose-Free Generalizable Rendering Transformer
To address this challenge, we introduce PF-GRT, a new Pose-Free framework for Generalizable Rendering Transformer, eliminating the need for pre-computed camera poses and instead leveraging feature-matching learned directly from data.
Efficient-3DiM: Learning a Generalizable Single-image Novel-view Synthesizer in One Day
Although the fidelity and generalizability are greatly improved, training such a powerful diffusion model requires a vast volume of training data and model parameters, resulting in a notoriously long time and high computational costs.
ARN: Analogical Reasoning on Narratives
However, while cognitive theories of analogy often focus on narratives and study the distinction between surface, relational, and system similarities, existing work in natural language processing has a narrower focus as far as relational analogies between word pairs.
FIRE: Food Image to REcipe generation
Food computing has emerged as a prominent multidisciplinary field of research in recent years.
Wasserstein distributional robustness of neural networks
To evaluate the distributional robustness of neural networks, we propose a first-order AA algorithm and its multi-step version.
POPE: 6-DoF Promptable Pose Estimation of Any Object, in Any Scene, with One Reference
To mitigate this issue, we propose a general paradigm for object pose estimation, called Promptable Object Pose Estimation (POPE).
The Weighted Möbius Score: A Unified Framework for Feature Attribution
Feature attribution aims to explain the reasoning behind a black-box model's prediction by identifying the impact of each feature on the prediction.
In-Context Learning Unlocked for Diffusion Models
We present Prompt Diffusion, a framework for enabling in-context learning in diffusion-based generative models.
Transferring Procedural Knowledge across Commonsense Tasks
Stories about everyday situations are an essential part of human communication, motivating the need to develop AI agents that can reliably understand these stories.
Patch Diffusion: Faster and More Data-Efficient Training of Diffusion Models
Patch Diffusion meanwhile improves the performance of diffusion models trained on relatively small datasets, $e. g.$, as few as 5, 000 images to train from scratch.
PAIR-Diffusion: A Comprehensive Multimodal Object-Level Image Editor
We propose PAIR Diffusion, a generic framework that can enable a diffusion model to control the structure and appearance properties of each object in the image.
Spatial-temporal Transformer-guided Diffusion based Data Augmentation for Efficient Skeleton-based Action Recognition
In this paper, we introduce a novel data augmentation method for skeleton-based action recognition tasks, which can effectively generate high-quality and diverse sequential actions.
NeuralLift-360: Lifting an In-the-Wild 2D Photo to a 3D Object With 360deg Views
In this work, we study the challenging task of lifting a single image to a 3D object and, for the first time, demonstrate the ability to generate a plausible 3D object with 360deg views that corresponds well with the given reference image.
NeuralLift-360: Lifting An In-the-wild 2D Photo to A 3D Object with 360° Views
In this work, we study the challenging task of lifting a single image to a 3D object and, for the first time, demonstrate the ability to generate a plausible 3D object with 360{\deg} views that correspond well with the given reference image.
AligNeRF: High-Fidelity Neural Radiance Fields via Alignment-Aware Training
Neural Radiance Fields (NeRFs) are a powerful representation for modeling a 3D scene as a continuous function.
Signal Processing for Implicit Neural Representations
We answer this question by proposing an implicit neural signal processing network, dubbed INSP-Net, via differential operators on INR.
Pose-Guided Graph Convolutional Networks for Skeleton-Based Action Recognition
Graph convolutional networks (GCNs), which can model the human body skeletons as spatial and temporal graphs, have shown remarkable potential in skeleton-based action recognition.
NeRF-SOS: Any-View Self-supervised Object Segmentation on Complex Scenes
Our framework, called NeRF with Self-supervised Object Segmentation NeRF-SOS, couples object segmentation and neural radiance field to segment objects in any view within a scene.
Testing Pre-trained Language Models' Understanding of Distributivity via Causal Mediation Analysis
To what extent do pre-trained language models grasp semantic knowledge regarding the phenomenon of distributivity?
Grasping the Arrow of Time from the Singularity: Decoding Micromotion in Low-dimensional Latent Spaces from StyleGAN
To study the motion features in the latent space of StyleGAN, in this paper, we hypothesize and demonstrate that a series of meaningful, natural, and versatile small, local movements (referred to as "micromotion", such as expression, head movement, and aging effect) can be represented in low-rank spaces extracted from the latent space of a conventionally pre-trained StyleGAN-v2 model for face generation, with the guidance of proper "anchors" in the form of either short text or video clips.
Unified Implicit Neural Stylization
Representing visual signals by implicit representation (e. g., a coordinate based deep network) has prevailed among many vision tasks.
SinNeRF: Training Neural Radiance Fields on Complex Scenes from a Single Image
Despite the rapid development of Neural Radiance Field (NeRF), the necessity of dense covers largely prohibits its wider applications.
TrafPS: A Visual Analysis System Interpreting Traffic Prediction in Shapley
TrafPS consists of three layers, from data process to results computation and visualization.
VAQF: Fully Automatic Software-Hardware Co-Design Framework for Low-Bit Vision Transformer
To the best of our knowledge, this is the first time quantization has been incorporated into ViT acceleration on FPGAs with the help of a fully automatic framework to guide the quantization strategy on the software side and the accelerator implementations on the hardware side given the target frame rate.
Fast and High-Quality Image Denoising via Malleable Convolutions
These spatially-varying kernels are produced by an efficient predictor network running on a downsampled input, making them much more efficient to compute than per-pixel kernels produced by a full-resolution image, and also enlarging the network's receptive field compared with static kernels.
Auto-X3D: Ultra-Efficient Video Understanding via Finer-Grained Neural Architecture Search
Efficient video architecture is the key to deploying video recognition systems on devices with limited computing resources.
Action Recognition with Domain Invariant Features of Skeleton Image
Due to the fast processing-speed and robustness it can achieve, skeleton-based action recognition has recently received the attention of the computer vision community.
A Teacher-Student Framework with Fourier Augmentation for COVID-19 Infection Segmentation in CT Images
Automatic segmentation of infected regions in computed tomography (CT) images is necessary for the initial diagnosis of COVID-19.
Lottery Image Prior
Drawing inspirations from the recently prosperous research on lottery ticket hypothesis (LTH), we conjecture and study a novel “lottery image prior” (LIP), stated as: given an (untrained or trained) DNN-based image prior, it will have a sparse subnetwork that can be training in isolation, to match the original DNN’s performance when being applied as a prior to various image inverse problems.
SSH: A Self-Supervised Framework for Image Harmonization
Image harmonization aims to improve the quality of image compositing by matching the "appearance" (\eg, color tone, brightness and contrast) between foreground and background images.
CERL: A Unified Optimization Framework for Light Enhancement with Realistic Noise
We present \underline{C}oordinated \underline{E}nhancement for \underline{R}eal-world \underline{L}ow-light Noisy Images (CERL), that seamlessly integrates light enhancement and noise suppression parts into a unified and physics-grounded optimization framework.
IA-RED$^2$: Interpretability-Aware Redundancy Reduction for Vision Transformers
The self-attention-based model, transformer, is recently becoming the leading backbone in the field of computer vision.
Ranked #29 on
Efficient ViTs
on ImageNet-1K (with DeiT-S)
InstantNet: Automated Generation and Deployment of Instantaneously Switchable-Precision Networks
The promise of Deep Neural Network (DNN) powered Internet of Thing (IoT) devices has motivated a tremendous demand for automated solutions to enable fast development and deployment of efficient (1) DNNs equipped with instantaneous accuracy-efficiency trade-off capability to accommodate the time-varying resources at IoT devices and (2) dataflows to optimize DNNs' execution efficiency on different devices.
A Design Space Study for LISTA and Beyond
Unrolling is believed to incorporate the model-based prior with the learning capacity of deep learning.
TransGAN: Two Pure Transformers Can Make One Strong GAN, and That Can Scale Up
Our vanilla GAN architecture, dubbed TransGAN, consists of a memory-friendly transformer-based generator that progressively increases feature resolution, and correspondingly a multi-scale discriminator to capture simultaneously semantic contexts and low-level textures.
Ranked #14 on
Image Generation
on STL-10
Few-shot Learning for CT Scan based COVID-19 Diagnosis
To compensate for the sparseness of labeled data, the proposed method utilizes a large amount of synthetic COVID-19 CT images and adjusts the networks from the source domain (synthetic data) to the target domain (real data) with a cross-domain training mechanism.
Unsupervised domain adaptation based COVID-19 CT infection segmentation network
In this paper, we propose an unsupervised domain adaptation based segmentation network to improve the segmentation performance of the infection areas in COVID-19 CT images.
AutoPose: Searching Multi-Scale Branch Aggregation for Pose Estimation
Such simplification limits the fusion of information at different scales and fails to maintain high-resolution representations.
COVID-19 CT Image Synthesis with a Conditional Generative Adversarial Network
However, training a deep-learning model requires large volumes of data, and medical staff faces a high risk when collecting COVID-19 CT data due to the high infectivity of the disease.
AutoGAN: Neural Architecture Search for Generative Adversarial Networks
Neural architecture search (NAS) has witnessed prevailing success in image classification and (very recently) segmentation tasks.
Ranked #22 on
Image Generation
on STL-10
EnlightenGAN: Deep Light Enhancement without Paired Supervision
Deep learning-based methods have achieved remarkable success in image restoration and enhancement, but are they still competitive when there is a lack of paired training data?
StrokeNet: A Neural Painting Environment
In this paper we try to address the discrete nature of software environment with an intermediate, differentiable simulation.
Pedestrian-Synthesis-GAN: Generating Pedestrian Data in Real Scene and Beyond
The results show that our framework can smoothly synthesize pedestrians on background images of variations and different levels of details.
Ranked #2 on
Scene Text Recognition
on MSDA
