Search Results for author:
Found 53 papers, 18 papers with code
MMedAgent: Learning to Use Medical Tools with Multi-modal Agent
We curate an instruction-tuning dataset comprising six medical tools solving seven tasks across five modalities, enabling the agent to choose the most suitable tools for a given task.
TinyLLaVA Factory: A Modularized Codebase for Small-scale Large Multimodal Models
We present TinyLLaVA Factory, an open-source modular codebase for small-scale large multimodal models (LMMs) with a focus on simplicity of code implementations, extensibility of new features, and reproducibility of training results.
An empirical study of LLaMA3 quantization: from LLMs to MLLMs
To uncover the capabilities of low-bit quantized MLLM, we assessed the performance of the LLaMA3-based LLaVA-Next-8B model under 2-4 ultra-low bits with post-training quantization methods.
Lamarckian Inheritance Improves Robot Evolution in Dynamic Environments
This study explores the integration of Lamarckian system into evolutionary robotics (ER), comparing it with the traditional Darwinian model across various environments.
A Fourier Transform Framework for Domain Adaptation
By using unsupervised domain adaptation (UDA), knowledge can be transferred from a label-rich source domain to a target domain that contains relevant information but lacks labels.
TinyLLaVA: A Framework of Small-scale Large Multimodal Models
We present the TinyLLaVA framework that provides a unified perspective in designing and analyzing the small-scale Large Multimodal Models (LMMs).
Ranked #182 on
Visual Question Answering
on MM-Vet
Accurate LoRA-Finetuning Quantization of LLMs via Information Retention
This paper proposes a novel IR-QLoRA for pushing quantized LLMs with LoRA to be highly accurate through information retention.
Ambient-Pix2PixGAN for Translating Medical Images from Noisy Data
Recently, an augmented GAN architecture named AmbientGAN has been proposed that can be trained on noisy measurement data to synthesize high-quality clean medical images.
Unsupervised Generation of Pseudo Normal PET from MRI with Diffusion Model for Epileptic Focus Localization
In this study, we investigated unsupervised learning methods for unpaired MRI to PET translation for generating pseudo normal FDG PET for epileptic focus localization.
Exploring Robot Morphology Spaces through Breadth-First Search and Random Query
Using two different query mechanisms, Breadth-First Search (BFS) and Random Query, within the context of evolving robot morphologies using CPPNs and robot controllers using tensors, and testing them in two evolutionary frameworks, Lamarckian and Darwinian systems, this study investigates their influence on evolutionary outcomes and performance.
A comparison of controller architectures and learning mechanisms for arbitrary robot morphologies
The main question this paper addresses is: What combination of a robot controller and a learning method should be used, if the morphology of the learning robot is not known in advance?
Lamarck's Revenge: Inheritance of Learned Traits Can Make Robot Evolution Better
We research this issue through simulations with an evolutionary robot framework where morphologies (bodies) and controllers (brains) of robots are evolvable and robots also can improve their controllers through learning during their lifetime.
Radiology-Llama2: Best-in-Class Large Language Model for Radiology
This paper introduces Radiology-Llama2, a large language model specialized for radiology through a process known as instruction tuning.
ESP: Exploiting Symmetry Prior for Multi-Agent Reinforcement Learning
In addition, the proposed framework is model-agnostic and can be applied to most of the current MARL algorithms.
Modulate Your Spectrum in Self-Supervised Learning
In this work, we introduce Spectral Transformation (ST), a framework to modulate the spectrum of embedding and to seek for functions beyond whitening that can avoid dimensional collapse.
SlicerTMS: Real-Time Visualization of Transcranial Magnetic Stimulation for Mental Health Treatment
We present a real-time visualization system for Transcranial Magnetic Stimulation (TMS), a non-invasive neuromodulation technique for treating various brain disorders and mental health diseases.
BiFSMNv2: Pushing Binary Neural Networks for Keyword Spotting to Real-Network Performance
We highlight that benefiting from the compact architecture and optimized hardware kernel, BiFSMNv2 can achieve an impressive 25. 1x speedup and 20. 2x storage-saving on edge hardware.
Environment induced emergence of collective behaviour in evolving swarms with limited sensing
Designing controllers for robot swarms is challenging, because human developers have typically no good understanding of the link between the details of a controller that governs individual robots and the swarm behavior that is an indirect result of the interactions between swarm members and the environment.
Delving into the Estimation Shift of Batch Normalization in a Network
We define the estimation shift magnitude of BN to quantitatively measure the difference between its estimated population statistics and expected ones.
BiFSMN: Binary Neural Network for Keyword Spotting
Then, to allow the instant and adaptive accuracy-efficiency trade-offs at runtime, we also propose a Thinnable Binarization Architecture to further liberate the acceleration potential of the binarized network from the topology perspective.
GPU optimization of the 3D Scale-invariant Feature Transform Algorithm and a Novel BRIEF-inspired 3D Fast Descriptor
This work details a highly efficient implementation of the 3D scale-invariant feature transform (SIFT) algorithm, for the purpose of machine learning from large sets of volumetric medical image data.
The Effects of Learning in Morphologically Evolving Robot Systems
Simultaneously evolving morphologies (bodies) and controllers (brains) of robots can cause a mismatch between the inherited body and brain in the offspring.
Rot-Pro: Modeling Transitivity by Projection in Knowledge Graph Embedding
In this paper, we first theoretically show that the transitive relations can be modeled with projections.
Ranked #13 on
Link Prediction
on YAGO3-10
All-Around Real Label Supervision: Cyclic Prototype Consistency Learning for Semi-supervised Medical Image Segmentation
Observing this, we ask an unexplored but interesting question: can we exploit the unlabeled data via explicit real label supervision for semi-supervised training?
Gait-learning with morphologically evolving robots generated by L-system
The second approach is evolution plus learning which means the brain of a child is inherited as well, but additionally is developed by a learning algorithm - RevDEknn.
Double-Uncertainty Guided Spatial and Temporal Consistency Regularization Weighting for Learning-based Abdominal Registration
Such convention has two limitations: (i) Besides the laborious grid search for the optimal fixed weight, the regularization strength of a specific image pair should be associated with the content of the images, thus the "one value fits all" training scheme is not ideal; (ii) Only spatially regularizing the transformation may neglect some informative clues related to the ill-posedness.
Trust It or Not: Confidence-Guided Automatic Radiology Report Generation
Experimental results have demonstrated that the proposed method for model uncertainty characterization and estimation can produce more reliable confidence scores for radiology report generation, and the modified loss function, which takes into account the uncertainties, leads to better model performance on two public radiology report datasets.
Noisy Labels are Treasure: Mean-Teacher-Assisted Confident Learning for Hepatic Vessel Segmentation
Manually segmenting the hepatic vessels from Computer Tomography (CT) is far more expertise-demanding and laborious than other structures due to the low-contrast and complex morphology of vessels, resulting in the extreme lack of high-quality labeled data.
Deep-Learning-Enabled Inverse Engineering of Multi-Wavelength Invisibility-to-Superscattering Switching with Phase-Change Materials
Inverse design of nanoparticles for desired scattering spectra and dynamic switching between the two opposite scattering anomalies, i. e. superscattering and invisibility, is important in realizing cloaking, sensing and functional devices.
Optics
Unsupervised Multimodal Image Registration with Adaptative Gradient Guidance
Multimodal image registration (MIR) is a fundamental procedure in many image-guided therapies.
Unimodal Cyclic Regularization for Training Multimodal Image Registration Networks
The loss function of an unsupervised multimodal image registration framework has two terms, i. e., a metric for similarity measure and regularization.
F3RNet: Full-Resolution Residual Registration Network for Deformable Image Registration
In this paper, we propose a novel unsupervised registration network, namely the Full-Resolution Residual Registration Network (F3RNet), for deformable registration of severely deformed organs.
Adversarial Uni- and Multi-modal Stream Networks for Multimodal Image Registration
Deformable image registration between Computed Tomography (CT) images and Magnetic Resonance (MR) imaging is essential for many image-guided therapies.
Do Public Datasets Assure Unbiased Comparisons for Registration Evaluation?
In this study, we use the variogram to screen the manually annotated landmarks in two datasets used to benchmark registration in image-guided neurosurgeries.
Unbiased Scene Graph Generation via Rich and Fair Semantic Extraction
Extracting graph representation of visual scenes in image is a challenging task in computer vision.
An Investigation of Feature-based Nonrigid Image Registration using Gaussian Process
For a wide range of clinical applications, such as adaptive treatment planning or intraoperative image update, feature-based deformable registration (FDR) approaches are widely employed because of their simplicity and low computational complexity.
Distant Supervised Relation Extraction with Separate Head-Tail CNN
Distant supervised relation extraction is an efficient and effective strategy to find relations between entities in texts.
I Know You'll Be Back: Interpretable New User Clustering and Churn Prediction on a Mobile Social Application
Then we design a novel deep learning pipeline based on LSTM and attention to accurately predict user churn with very limited initial behavior data, by leveraging the correlations among users' multi-dimensional activities and the underlying user types.
Are Registration Uncertainty and Error Monotonically Associated
Probabilistic image registration (PIR) methods provide measures of registration uncertainty, which could be a surrogate for assessing the registration error.
Temporal Registration in Application to In-utero MRI Time Series
To achieve accurate and robust alignment, we make a Markov assumption on the nature of motion and take advantage of the temporal smoothness in the image data.
Deep Information Theoretic Registration
This paper establishes an information theoretic framework for deep metric based image registration techniques.
Deep learning based automatic segmentation of lumbosacral nerves on non-contrast CT for radiographic evaluation: a pilot study
Pixel accuracy, IoU, and Dice score are used to assess the segmentation performance of lumbosacral structures.
Synthesizing dynamic MRI using long-term recurrent convolutional networks
A method is proposed for converting raw ultrasound signals of respiratory organ motion into high frame rate dynamic MRI using a long-term recurrent convolutional neural network.
Semi-Supervised Deep Metrics for Image Registration
In this paper, we propose a strategy for learning such metrics from roughly aligned training data.
A Feature-Driven Active Framework for Ultrasound-Based Brain Shift Compensation
Kernels of the GP are estimated by using variograms and a discrete grid search method.
On the Applicability of Registration Uncertainty
For probabilistic image registration (PIR), the predominant way to quantify the registration uncertainty is using summary statistics of the distribution of transformation parameters.
mvn2vec: Preservation and Collaboration in Multi-View Network Embedding
With experiments on a series of synthetic datasets, a large-scale internal Snapchat dataset, and two public datasets, we confirm the validity and importance of preservation and collaboration as two objectives for multi-view network embedding.
Misdirected Registration Uncertainty
Since the transformation is such an essential component of registration, most existing researches conventionally quantify the registration uncertainty, which is the confidence in the estimated spatial correspondences, by the transformation uncertainty.
Temporal Registration in In-Utero Volumetric MRI Time Series
We present a robust method to correct for motion and deformations for in-utero volumetric MRI time series.
Reinterpreting the Transformation Posterior in Probabilistic Image Registration
Meanwhile, summary statistics of the posterior are employed to evaluate the registration uncertainty, that is the trustworthiness of the registered image.
Learning from Candidate Labeling Sets
In this paper, we propose a semi-supervised framework to model this kind of problems.
Who’s Doing What: Joint Modeling of Names and Verbs for Simultaneous Face and Pose Annotation
Given a corpus of news items consisting of images accompanied by text captions, we want to find out ``whos doing what, i. e. associate names and action verbs in the captions to the face and body pose of the persons in the images.
Beyond Novelty Detection: Incongruent Events, when General and Specific Classifiers Disagree
We define a formal framework for the representation and processing of incongruent events: starting from the notion of label hierarchy, we show how partial order on labels can be deduced from such hierarchies.
