Search Results for author:
Found 23 papers, 8 papers with code
REPA-E: Unlocking VAE for End-to-End Tuning of Latent Diffusion Transformers
We show that while diffusion loss is ineffective, end-to-end training can be unlocked through the representation-alignment (REPA) loss -- allowing both VAE and diffusion model to be jointly tuned during the training process.
Ranked #2 on
Image Generation
on ImageNet 256x256
REPA-E: Unlocking VAE for End-to-End Tuning with Latent Diffusion Transformers
We show that while diffusion loss is ineffective, end-to-end training can be unlocked through the representation-alignment (REPA) loss -- allowing both VAE and diffusion model to be jointly tuned during the training process.
R2E-Gym: Procedural Environments and Hybrid Verifiers for Scaling Open-Weights SWE Agents
Improving open-source models on real-world SWE tasks (solving GITHUB issues) faces two key challenges: 1) scalable curation of execution environments to train these models, and, 2) optimal scaling of test-time compute.
Storybooth: Training-free Multi-Subject Consistency for Improved Visual Storytelling
In this work, we first analyze the reason for these limitations.
Entropy-Assisted Quality Pattern Identification in Finance
Short-term patterns in financial time series form the cornerstone of many algorithmic trading strategies, yet extracting these patterns reliably from noisy market data remains a formidable challenge.
Negative Token Merging: Image-based Adversarial Feature Guidance
We introduce negative token merging (NegToMe), a simple but effective training-free approach which performs adversarial guidance through images by selectively pushing apart matching visual features between reference and generated images during the reverse diffusion process.
On the Impact of White-box Deployment Strategies for Edge AI on Latency and Model Performance
Among the non-hybrid operators, the Distilled operator is a better alternative in both mobile and edge tiers for lower latency performance at the cost of small to medium accuracy loss.
Vec2Face: Scaling Face Dataset Generation with Loosely Constrained Vectors
However, existing works 1) are typically limited in how many well-separated identities can be generated and 2) either neglect or use a separate editing model for attribute augmentation.
OpenHands: An Open Platform for AI Software Developers as Generalist Agents
OpenDevin), a platform for the development of powerful and flexible AI agents that interact with the world in similar ways to those of a human developer: by writing code, interacting with a command line, and browsing the web.
Towards Hierarchical Multi-Agent Workflows for Zero-Shot Prompt Optimization
Specifically, we include a hierarchy of LLMs, first constructing a prompt with precise instructions and accurate wording in a hierarchical manner, and then using this prompt to generate the final answer to the user query.
On the Impact of Black-box Deployment Strategies for Edge AI on Latency and Model Performance
In this paper, we conduct inference experiments involving 3 deployment operators (i. e., Partitioning, Quantization, Early Exit), 3 deployment tiers (i. e., Mobile, Edge, Cloud) and their combinations on four widely used Computer-Vision models to investigate the optimal strategies from the point of view of MLOps developers.
SmartMask: Context Aware High-Fidelity Mask Generation for Fine-grained Object Insertion and Layout Control
To overcome these limitations, we introduce SmartMask, which allows any novice user to create detailed masks for precise object insertion.
IMPUS: Image Morphing with Perceptually-Uniform Sampling Using Diffusion Models
We present a diffusion-based image morphing approach with perceptually-uniform sampling (IMPUS) that produces smooth, direct and realistic interpolations given an image pair.
Divide, Evaluate, and Refine: Evaluating and Improving Text-to-Image Alignment with Iterative VQA Feedback
Furthermore, we also find that the assertion level alignment scores provide a useful feedback which can then be used in a simple iterative procedure to gradually increase the expression of different assertions in the final image outputs.
Probabilistic and Semantic Descriptions of Image Manifolds and Their Applications
This paper begins with a description of methods for estimating image probability density functions that reflects the observation that such data is usually constrained to lie in restricted regions of the high-dimensional image space-not every pattern of pixels is an image.
High-Fidelity Guided Image Synthesis with Latent Diffusion Models
The user scribbles control the color composition while the text prompt provides control over the overall image semantics.
UAV-based Visual Remote Sensing for Automated Building Inspection
Unmanned Aerial Vehicle (UAV) based remote sensing system incorporated with computer vision has demonstrated potential for assisting building construction and in disaster management like damage assessment during earthquakes.
Paint2Pix: Interactive Painting based Progressive Image Synthesis and Editing
In particular, we propose a novel approach paint2pix, which learns to predict (and adapt) "what a user wants to draw" from rudimentary brushstroke inputs, by learning a mapping from the manifold of incomplete human paintings to their realistic renderings.
Intelli-Paint: Towards Developing Human-like Painting Agents
However, current research in this direction is often reliant on a progressive grid-based division strategy wherein the agent divides the overall image into successively finer grids, and then proceeds to paint each of them in parallel.
Sparse Attention Guided Dynamic Value Estimation for Single-Task Multi-Scene Reinforcement Learning
However, we argue that the sample variance for a multi-scene environment is best minimized by treating each scene as a distinct MDP, and then learning a joint value function V(s, M) dependent on both state s and MDP M. We further demonstrate that the true joint value function for a multi-scene environment, follows a multi-modal distribution which is not captured by traditional CNN / LSTM based critic networks.
Enhanced Scene Specificity with Sparse Dynamic Value Estimation
Recently, Singh et al. [1] tried to address this by proposing a dynamic value estimation approach that models the true joint value function distribution as a Gaussian mixture model (GMM).
Combining Semantic Guidance and Deep Reinforcement Learning For Generating Human Level Paintings
2) We also introduce invariance to the position and scale of the foreground object through a neural alignment model, which combines object localization and spatial transformer networks in an end to end manner, to zoom into a particular semantic instance.
Deep Reinforcement Learning
Model-based Reinforcement Learning
+4
Dynamic Value Estimation for Single-Task Multi-Scene Reinforcement Learning
Training deep reinforcement learning agents on environments with multiple levels / scenes / conditions from the same task, has become essential for many applications aiming to achieve generalization and domain transfer from simulation to the real world.
