Search Results for author:
Found 32 papers, 13 papers with code
Scenethesis: A Language and Vision Agentic Framework for 3D Scene Generation
Our key insight is that vision perception can bridge this gap by providing realistic spatial guidance that LLMs lack.
Describe Anything: Detailed Localized Image and Video Captioning
We introduce the Describe Anything Model (DAM), a model designed for detailed localized captioning (DLC).
Cosmos-Transfer1: Conditional World Generation with Adaptive Multimodal Control
We introduce Cosmos-Transfer, a conditional world generation model that can generate world simulations based on multiple spatial control inputs of various modalities such as segmentation, depth, and edge.
Cosmos World Foundation Model Platform for Physical AI
In this paper, we present the Cosmos World Foundation Model Platform to help developers build customized world models for their Physical AI setups.
freePruner: A Training-free Approach for Large Multimodal Model Acceleration
Large Multimodal Models (LMMs) have demonstrated impressive capabilities in visual-language tasks but face significant deployment challenges due to their high computational demands.
Edify Image: High-Quality Image Generation with Pixel Space Laplacian Diffusion Models
We introduce Edify Image, a family of diffusion models capable of generating photorealistic image content with pixel-perfect accuracy.
Edify 3D: Scalable High-Quality 3D Asset Generation
We introduce Edify 3D, an advanced solution designed for high-quality 3D asset generation.
BEHAVIOR Vision Suite: Customizable Dataset Generation via Simulation
We introduce the BEHAVIOR Vision Suite (BVS), a set of tools and assets to generate fully customized synthetic data for systematic evaluation of computer vision models, based on the newly developed embodied AI benchmark, BEHAVIOR-1K.
Visual Fact Checker: Enabling High-Fidelity Detailed Caption Generation
VFC consists of three steps: 1) proposal, where image-to-text captioning models propose multiple initial captions; 2) verification, where a large language model (LLM) utilizes tools such as object detection and VQA models to fact-check proposed captions; 3) captioning, where an LLM generates the final caption by summarizing caption proposals and the fact check verification results.
DreamDistribution: Learning Prompt Distribution for Diverse In-distribution Generation
We introduce a solution that allows a pretrained T2I diffusion model to learn a set of soft prompts, enabling the generation of novel images by sampling prompts from the learned distribution.
3D Copy-Paste: Physically Plausible Object Insertion for Monocular 3D Detection
A major challenge in monocular 3D object detection is the limited diversity and quantity of objects in real datasets.
Evaluating Pretrained models for Deployable Lifelong Learning
We create a novel benchmark for evaluating a Deployable Lifelong Learning system for Visual Reinforcement Learning (RL) that is pretrained on a curated dataset, and propose a novel Scalable Lifelong Learning system capable of retaining knowledge from the previously learnt RL tasks.
Beyond Generation: Harnessing Text to Image Models for Object Detection and Segmentation
A foreground-background segmentation algorithm is then used to generate foreground object masks.
CLR: Channel-wise Lightweight Reprogramming for Continual Learning
Reprogramming parameters are task-specific and exclusive to each task, which makes our method immune to catastrophic forgetting.
Robustness Analysis on Foundational Segmentation Models
In this work, we perform a robustness analysis of Visual Foundation Models (VFMs) for segmentation tasks and focus on robustness against real-world distribution shift inspired perturbations.
Building One-class Detector for Anything: Open-vocabulary Zero-shot OOD Detection Using Text-image Models
Despite considerable effort, the problem remains significantly challenging in deep learning models due to their propensity to output over-confident predictions for OOD inputs.
Lightweight Learner for Shared Knowledge Lifelong Learning
We propose a new Shared Knowledge Lifelong Learning (SKILL) challenge, which deploys a decentralized population of LL agents that each sequentially learn different tasks, with all agents operating independently and in parallel.
EM-Paste: EM-guided Cut-Paste with DALL-E Augmentation for Image-level Weakly Supervised Instance Segmentation
Finally, the third component creates a large-scale pseudo-labeled instance segmentation training dataset by compositing the foreground object masks onto the original and generated background images.
Improving Zero-shot Generalization and Robustness of Multi-modal Models
We also show that our method improves across ImageNet shifted datasets, four other datasets, and other model architectures such as LiT.
Neural-Sim: Learning to Generate Training Data with NeRF
However, existing approaches either require human experts to manually tune each scene property or use automatic methods that provide little to no control; this requires rendering large amounts of random data variations, which is slow and is often suboptimal for the target domain.
Contributions of Shape, Texture, and Color in Visual Recognition
We use human experiments to confirm that both HVE and humans predominantly use some specific features to support the classification of specific classes (e. g., texture is the dominant feature to distinguish a zebra from other quadrupeds, both for humans and HVE).
DALL-E for Detection: Language-driven Compositional Image Synthesis for Object Detection
For foreground object mask generation, we use a simple textual template with object class name as input to DALL-E to generate a diverse set of foreground images.
Invariant Structure Learning for Better Generalization and Causal Explainability
ISL splits the data into different environments, and learns a structure that is invariant to the target across different environments by imposing a consistency constraint.
Encouraging Disentangled and Convex Representation with Controllable Interpolation Regularization
(2) They lack convexity constraints, which is important for meaningfully manipulating specific attributes for downstream tasks.
Towards Generic Interface for Human-Neural Network Knowledge Exchange
Take image classification as an example, HNI visualizes the reasoning logic of a NN with class-specific Structural Concept Graphs (c-SCG), which are human-interpretable.
A Peek Into the Reasoning of Neural Networks: Interpreting with Structural Visual Concepts
Despite substantial progress in applying neural networks (NN) to a wide variety of areas, they still largely suffer from a lack of transparency and interpretability.
Graph Autoencoder for Graph Compression and Representation Learning
We consider the problem of graph data compression and representation.
Generative Auto-Encoder: Non-adversarial Controllable Synthesis with Disentangled Exploration
DEAE can become a generative model and synthesis semantic controllable samples by interpolating latent code, which can even synthesis novel attribute value never is shown in the original dataset.
Beneficial Perturbation Network for designing general adaptive artificial intelligence systems
Continual learning of multiple tasks in artificial neural networks using gradient descent leads to catastrophic forgetting, whereby a previously learned mapping of an old task is erased when learning new mappings for new tasks.
Zero-shot Synthesis with Group-Supervised Learning
Visual cognition of primates is superior to that of artificial neural networks in its ability to 'envision' a visual object, even a newly-introduced one, in different attributes including pose, position, color, texture, etc.
Pose Augmentation: Class-agnostic Object Pose Transformation for Object Recognition
After training on unbalanced discrete poses (5 classes with 6 poses per object instance, plus 5 classes with only 2 poses), we show that OPT-Net can synthesize balanced continuous new poses along yaw and pitch axes with high quality.
Synthesis and Inpainting-Based MR-CT Registration for Image-Guided Thermal Ablation of Liver Tumors
Then, an unsupervised registration network is used to efficiently align the pre-procedural CT (pCT) with the inpainted iCT (inpCT) image.
