Ongoing Research

Investigating Grokking as a Phase Transition under the SETOL framework and Latent Dunning-Kruger Dynamics

(WIP) 1. Introduction Grokking is a delayed yet sudden leap in generalization performance, often observed in neural networks long after they have perfectly memorized the training set. Recent studies indicate two compelling perspectives: 1. Spectral Phase Transition: Grokking arises from heavy-tailed self-regularization (HTSR), where the network’s eigenvalue spectrum evolves

Ongoing Research

Converting 2D Architectural Drawings into Hierarchical Scene Graph Representations

In this ongoing research, we outline a theoretical framework for transforming architectural 2D drawings into hierarchical scene graph representations. Traditional floorplans carry far more than rectangular outlines: they contain rich spatial relationships, complex room shapes, and functional constraints. Our approach bridges graph-theoretic floorplan analysis, hierarchical scene modeling, and constraint reasoning

Ongoing Research

Exploring the "Latent Dunning-Kruger" Effect for Grokking Meta-Optimizers

In this ongoing research, we propose that the Dunning-Kruger effect, a statistically demonstrated phenomenon in human learning, has a striking analogy in deep learning. Specifically, when we look at models undergoing grokking (an abrupt shift from memorization to generalization), we observe hysteresis-like curves reminiscent of both: 1. Dunning-Kruger curves, where

Ongoing Research

SETOL-based LLM Mechanistic Interpretability of Domain Knowledge

In this ongoing research study, we collaborate with WeightWatcher to build a new LLM mechanistic interpretability framework grounded in the SETOL (Semi-Empirical Theory of Deep Learning) perspective. This exploration aims to combine insights from spectral analysis of weights with energy-based and physics-inspired approaches to understanding how large language models (LLMs)