Isaac Gerg

Isaac D. Gerg, Ph.D.

AI Research Scientist building frontier generative and world models: physics-based, multimodal, grounded in real sensor data

I develop and rigorously evaluate generative and predictive models for complex, noisy sensor data: diffusion and GAN-based image restoration and synthesis, self-supervised representation learning, learned compression, and the signal processing that makes them work. I do my best work heads-down on a hard technical problem: building an approach, testing it rigorously, understanding why it fails, and charting the next direction. A single thread runs through this work: I build the physics and domain structure of the problem into the model, scattering behavior, coherent-imaging constraints, and signal priors, rather than relying on scale alone. Where I am headed: building frontier generative and world models, physics-based and multimodal learned simulators that fuse real sensor observations with physical structure into representations you can probe, stress-test, and build on.

Before that I spent over a decade pushing deep learning into one of the hardest sensing domains there is: complex-valued, low-data synthetic aperture sonar. Along the way I published 50+ papers, earned two best-paper finalists, and led multi-year research programs and set their technical direction. I earned my Ph.D. at Penn State while working full time.

Open to remote research roles building frontier generative and world models, and advancing the methods behind them: generative modeling, physics-based and domain-enriched ML, and multimodal sensing. Most interested in developing and rigorously evaluating new methods and setting technical direction. US-based, fully remote.

Operating thesis
Intelligence = Model-making + Search + Evaluation

All three are load-bearing. Drop any one and a method that keeps all three wins out. Building that complete system is the direction I work in.


Core Competencies

Physics-based & model-based deep learning (scattering, coherent imaging, priors) Generative models (diffusion, GANs) Foundation models & self-supervision World models & spatiotemporal dynamics Signal & sensor ML (complex-valued, multimodal, acoustic) Large-scale model training & GPU acceleration (CUDA) Rigorous evaluation, benchmarking & ablation PyTorch, JAX, Python Research program leadership (PI: DARPA, ONR, NAVSEA)

Education & Career

2000 2004 2008 2012 2016 2020 2025
Education
Schreyer (PSU)
B.S.
Aug 2004
Penn State
M.S.
2008
Penn State (PSU)
Ph.D.
2022
Career
Raytheon
Penn State ARL
Kitware
Navy Nuclear Lab (DOE)
Raytheon
Penn State ARL
NATO CMRE (sabbatical)
Kitware
ClimateAI

Research Interests

Physics-Based & Domain-Enriched ML

Building the measurement model and domain structure into the network instead of relying on scale alone: acoustic scattering, coherent-imaging and phase constraints, rendering forward models, and signal priors. This model-based approach keeps learning sample-efficient, interpretable, and robust in the noisy, low-data regimes where black-box models break down.

Earth-System AI & Earth Observation

Learning from satellite and Earth-observation data toward models of Earth systems: representation, bias, and the building blocks of physics-based world models.

Remote Sensing & AI

Applying machine learning to extract actionable intelligence from sonar, radar, and other sensor modalities.

Signal Processing

Integrating classical signal processing theory with modern deep learning for robust sensor data analysis.

Robustness & Interpretability

How deep networks represent, generalize, and fail, studied through metamers, perceptual priors, and out-of-distribution behavior, with inspiration from biological perception.

Computer Vision

Detection, recognition, and understanding of objects and scenes in challenging imaging environments.


Selected Publications

Annotated synthetic aperture sonar target showing highlight and acoustic shadow
IEEE IGARSS · 2026
Prompted, Not Trained: On Zero-Shot Classification of Synthetic Aperture Imagery with Vision-Language Models
I. D. Gerg
Weather Data in Consumer Spending Prediction
J. Retailing & Consumer Services · 2025
The Substantial Role of Weather Data in Consumer Spending Prediction: A Robust Machine Learning Assessment
I. D. Gerg, A. M. Tashie, A. Patanaik, E. Koester, H. Gupta, D. J. Farnham
Low-Shot Learning for SAS
IEEE OCEANS · 2025
Low-Shot Learning for Synthetic Aperture Sonar Image Classification Using Hierarchical Pretraining & AirSAS
I. D. Gerg, A. Lynch, T. E. Blanford
Phase Retrieval for Acoustic Classification
IEEE OCEANS · 2025
Are Magnitude-Only Spectrograms Sufficient for Acoustic Classification in PAM? A Phase Retrieval Perspective
I. D. Gerg, P. McAfee, T. E. Blanford
Synthetic aperture sonar imagery before and after DAPL autofocus, with point-target zoom insets
IEEE JSTARS · 2024
Deep Adaptive Phase Learning: Enhancing Synthetic Aperture Sonar Imagery Through Learned Coherent Autofocus
I. D. Gerg, D. A. Cook, V. Monga
Progressive Diffusion Autofocus
IGARSS · 2024
Progressive Diffusion Autofocus for Synthetic Aperture Sonar Imagery
B. V. Bingol, I. D. Gerg, V. Monga
Uncovering Bias in Building Damage Assessment
IGARSS · 2024
Uncovering Bias in Building Damage Assessment from Satellite Imagery
D. Melamed, C. Johnson, I. D. Gerg, et al.
Metamers in Deep Learning for SAS
Acoustical Society of America · 2024
Metamers in Deep Learning for Synthetic Aperture Sonar Image Classification & Their Significance in Understanding Deep Networks
I. D. Gerg, C. Cotner
Erroneous Labels on SAS Classifier Performance
Institute of Acoustics · 2023
Exploring the Impact of Erroneous Labels on Synthetic Aperture Sonar Classifier Performance
I. D. Gerg, B. E. Cowen
Structural Prior Driven Regularized Deep Learning
IEEE TGRS · 2022
Structural Prior Driven Regularized Deep Learning for Sonar Image Classification
I. D. Gerg, V. Monga
Deep Multi-Look Sequence Processing for SAS Segmentation
IEEE TGRS · 2022
Deep Multi-Look Sequence Processing for Synthetic Aperture Sonar Image Segmentation
I. D. Gerg, V. Monga
Real-Time Deep SAS Autofocus
Best Paper Finalist
IGARSS · 2021
Real-Time, Deep Synthetic Aperture Sonar (SAS) Autofocus
I. D. Gerg, V. Monga
Learnable Image Compression for SAS
Best Paper Finalist
OCEANS · 2021
A Learnable Image Compression Scheme for Synthetic Aperture Sonar Imagery
I. D. Gerg
GPU Acceleration for SAS Image Reconstruction
Global OCEANS · 2020
GPU Acceleration for SAS Image Reconstruction
I. D. Gerg
Coupling Rendering and GANs for Artificial SAS Image Generation
OCEANS · 2019
Coupling Rendering and GANs for Artificial SAS Image Generation
I. D. Gerg

Personal Projects

ApertureLab
Simulation Workbench
ApertureLab: Synthetic Aperture Sonar Simulator
A fully coherent synthetic aperture sonar simulator and beamforming workbench that models the whole chain from raw IQ time series to a fully formed image. Built to create world models of the ocean floor for autonomous underwater AI.
Nittany Nights
LLM + Computer Vision
Nittany Nights: an autonomous LLM + computer-vision data pipeline
A production pipeline that uses LLMs and state-of-the-art computer vision to read venue menus, flyers, and social posts, then automatically structures a live database of happy hours, specials, food, and events across 60+ venues. A consumer guide to State College nightlife on the surface; an end-to-end applied-GenAI system underneath.
REMUS RLF Reader
Python · GitHub
REMUS-100 AUV Run Log File Reader
Python parser for REMUS-100 AUV binary run log files, reverse-engineered from the Makua Beach dataset. Parses RLF, ADCP, GPS, and RMF formats.
SAS Autofocus
Python · 18 stars · GitHub
Synthetic Aperture Sonar Autofocus
Compares standard Phase Gradient Autofocus (PGA) with Shadow PGA for SAS imagery, focusing on shadow regions rather than bright reflectors.
MATLAB Hyperspectral Toolbox
MATLAB · 115 stars · GitHub
MATLAB Hyperspectral Toolbox
Comprehensive toolbox for hyperspectral image processing and analysis, originally developed for Penn State thesis work on endmember extraction algorithms.
uBiome Longitudinal Analysis
Python · 13 stars · GitHub
uBiome Longitudinal Microbiome Analysis
Longitudinal analysis and visualization of gut microbiome composition over time using uBiome sequencing data, including heatmaps, diversity indices, and cluster analysis.


Contact

isaac.gerg@gergltd.com