Search Results for author:
Found 46 papers, 12 papers with code
Learning Structural Similarity of User Interface Layouts using Graph Networks
We propose a novel representation learning technique for measuring the similarity of user interface designs.
Thinking Outside the BBox: Unconstrained Generative Object Compositing
Compositing an object into an image involves multiple non-trivial sub-tasks such as object placement and scaling, color/lighting harmonization, viewpoint/geometry adjustment, and shadow/reflection generation.
Self-Improving Diffusion Models with Synthetic Data
Unfortunately, training new generative models with synthetic data from current or past generation models creates an autophagous (self-consuming) loop that degrades the quality and/or diversity of the synthetic data in what has been termed model autophagy disorder (MAD) and model collapse.
Ranked #1 on
Image Generation
on CIFAR-10
FINEMATCH: Aspect-based Fine-grained Image and Text Mismatch Detection and Correction
To address this, we propose FineMatch, a new aspect-based fine-grained text and image matching benchmark, focusing on text and image mismatch detection and correction.
ProMark: Proactive Diffusion Watermarking for Causal Attribution
ProMark can maintain image quality whilst outperforming correlation-based attribution.
VIXEN: Visual Text Comparison Network for Image Difference Captioning
We present VIXEN - a technique that succinctly summarizes in text the visual differences between a pair of images in order to highlight any content manipulation present.
TrustMark: Universal Watermarking for Arbitrary Resolution Images
We propose TrustMark - a GAN-based watermarking method with novel design in architecture and spatio-spectra losses to balance the trade-off between watermarked image quality with the watermark recovery accuracy.
DECORAIT -- DECentralized Opt-in/out Registry for AI Training
We report a prototype of DECORAIT, which explores hierarchical clustering and a combination of on/off-chain storage to create a scalable decentralized registry to trace the provenance of GenAI training data in order to determine training consent and reward creatives who contribute that data.
DIFF-NST: Diffusion Interleaving For deFormable Neural Style Transfer
Neural Style Transfer (NST) is the field of study applying neural techniques to modify the artistic appearance of a content image to match the style of a reference style image.
ALADIN-NST: Self-supervised disentangled representation learning of artistic style through Neural Style Transfer
Representation learning aims to discover individual salient features of a domain in a compact and descriptive form that strongly identifies the unique characteristics of a given sample respective to its domain.
NeAT: Neural Artistic Tracing for Beautiful Style Transfer
As a component of curating this data, we present a novel model able to classify if an image is stylistic.
EKILA: Synthetic Media Provenance and Attribution for Generative Art
We present EKILA; a decentralized framework that enables creatives to receive recognition and reward for their contributions to generative AI (GenAI).
RoSteALS: Robust Steganography using Autoencoder Latent Space
Data hiding such as steganography and invisible watermarking has important applications in copyright protection, privacy-preserved communication and content provenance.
VADER: Video Alignment Differencing and Retrieval
We propose VADER, a spatio-temporal matching, alignment, and change summarization method to help fight misinformation spread via manipulated videos.
PARASOL: Parametric Style Control for Diffusion Image Synthesis
We propose PARASOL, a multi-modal synthesis model that enables disentangled, parametric control of the visual style of the image by jointly conditioning synthesis on both content and a fine-grained visual style embedding.
Audio-Visual Contrastive Learning with Temporal Self-Supervision
We propose a self-supervised learning approach for videos that learns representations of both the RGB frames and the accompanying audio without human supervision.
SceneComposer: Any-Level Semantic Image Synthesis
We propose a new framework for conditional image synthesis from semantic layouts of any precision levels, ranging from pure text to a 2D semantic canvas with precise shapes.
HyperNST: Hyper-Networks for Neural Style Transfer
We present HyperNST; a neural style transfer (NST) technique for the artistic stylization of images, based on Hyper-networks and the StyleGAN2 architecture.
Video Manipulations Beyond Faces: A Dataset with Human-Machine Analysis
To this end, we present VideoSham, a dataset consisting of 826 videos (413 real and 413 manipulated).
RepMix: Representation Mixing for Robust Attribution of Synthesized Images
Uniquely, we present a solution to this task capable of 1) matching images invariant to their semantic content; 2) robust to benign transformations (changes in quality, resolution, shape, etc.)
SImProv: Scalable Image Provenance Framework for Robust Content Attribution
We present SImProv - a scalable image provenance framework to match a query image back to a trusted database of originals and identify possible manipulations on the query.
CoGS: Controllable Generation and Search from Sketch and Style
CoGS enables exploration of diverse appearance possibilities for a given sketched object, enabling decoupled control over the structure and the appearance of the output.
StyleBabel: Artistic Style Tagging and Captioning
We present StyleBabel, a unique open access dataset of natural language captions and free-form tags describing the artistic style of over 135K digital artworks, collected via a novel participatory method from experts studying at specialist art and design schools.
ARIA: Adversarially Robust Image Attribution for Content Provenance
Finally, we show how to train an adversarially robust image comparator model for detecting editorial changes in matched images.
VPN: Video Provenance Network for Robust Content Attribution
We present VPN - a content attribution method for recovering provenance information from videos shared online.
Scene Designer: a Unified Model for Scene Search and Synthesis from Sketch
Scene Designer is a novel method for searching and generating images using free-hand sketches of scene compositions; i. e. drawings that describe both the appearance and relative positions of objects.
OSCAR-Net: Object-centric Scene Graph Attention for Image Attribution
Our key contribution is OSCAR-Net (Object-centric Scene Graph Attention for Image Attribution Network); a robust image hashing model inspired by recent successes of Transformers in the visual domain.
Compositional Sketch Search
We present an algorithm for searching image collections using free-hand sketches that describe the appearance and relative positions of multiple objects.
Magic Layouts: Structural Prior for Component Detection in User Interface Designs
We present Magic Layouts; a method for parsing screenshots or hand-drawn sketches of user interface (UI) layouts.
ALADIN: All Layer Adaptive Instance Normalization for Fine-grained Style Similarity
We present ALADIN (All Layer AdaIN); a novel architecture for searching images based on the similarity of their artistic style.
Geo-PIFu: Geometry and Pixel Aligned Implicit Functions for Single-view Human Reconstruction
We propose Geo-PIFu, a method to recover a 3D mesh from a monocular color image of a clothed person.
Sketchformer: Transformer-based Representation for Sketched Structure
Sketchformer is a novel transformer-based representation for encoding free-hand sketches input in a vector form, i. e. as a sequence of strokes.
Neural Architecture Search for Deep Image Prior
We present a neural architecture search (NAS) technique to enhance the performance of unsupervised image de-noising, in-painting and super-resolution under the recently proposed Deep Image Prior (DIP).
An Internal Learning Approach to Video Inpainting
We propose a novel video inpainting algorithm that simultaneously hallucinates missing appearance and motion (optical flow) information, building upon the recent 'Deep Image Prior' (DIP) that exploits convolutional network architectures to enforce plausible texture in static images.
Semantic Estimation of 3D Body Shape and Pose using Minimal Cameras
We aim to simultaneously estimate the 3D articulated pose and high fidelity volumetric occupancy of human performance, from multiple viewpoint video (MVV) with as few as two views.
Ranked #176 on
3D Human Pose Estimation
on Human3.6M
Robust Synthesis of Adversarial Visual Examples Using a Deep Image Prior
We present a novel method for generating robust adversarial image examples building upon the recent `deep image prior' (DIP) that exploits convolutional network architectures to enforce plausible texture in image synthesis.
TAPESTRY: A Blockchain based Service for Trusted Interaction Online
We present a novel blockchain based service for proving the provenance of online digital identity, exposed as an assistive tool to help non-expert users make better decisions about whom to trust online.
ARCHANGEL: Tamper-proofing Video Archives using Temporal Content Hashes on the Blockchain
We present ARCHANGEL; a novel distributed ledger based system for assuring the long-term integrity of digital video archives.
LiveSketch: Query Perturbations for Guided Sketch-based Visual Search
LiveSketch is a novel algorithm for searching large image collections using hand-sketched queries.
Volumetric performance capture from minimal camera viewpoints
We present a convolutional autoencoder that enables high fidelity volumetric reconstructions of human performance to be captured from multi-view video comprising only a small set of camera views.
Deep Autoencoder for Combined Human Pose Estimation and body Model Upscaling
We present a method for simultaneously estimating 3D human pose and body shape from a sparse set of wide-baseline camera views.
Ranked #9 on
3D Human Pose Estimation
on Total Capture
Disentangling Structure and Aesthetics for Style-Aware Image Completion
Content-aware image completion or in-painting is a fundamental tool for the correction of defects or removal of objects in images.
Sketching With Style: Visual Search With Sketches and Aesthetic Context
We propose a novel measure of visual similarity for image retrieval that incorporates both structural and aesthetic (style) constraints.
BAM! The Behance Artistic Media Dataset for Recognition Beyond Photography
Furthermore, we carry out baseline experiments to show the value of this dataset for artistic style prediction, for improving the generality of existing object classifiers, and for the study of visual domain adaptation.
Generalisation and Sharing in Triplet Convnets for Sketch based Visual Search
We propose and evaluate several triplet CNN architectures for measuring the similarity between sketches and photographs, within the context of the sketch based image retrieval (SBIR) task.
