[Idea] Cubic Step: We Have More Info, Lets Use It!
Current gradient based optimization ignores 0th order information. In this post I want to motivate others to try utilizing it.
Welcome to my Machine Learning blog. Ill divide posts into three sections: Fun/Random [Rand], Research/Ideas [Idea], and Personal Posts [Pers]. I hope you enjoy.
Recent posts
[Idea] Super-Masks: Are They Good Initializations?
In this post I discuss a Super Masks and their potential as initializations, similar to the lottery ticket hypothesis
[Pers] Hack Umass Mentor: How to Debug
This past weekend I had the pleasure of being a mentor at Hack Umass. In this post I discuss how the experience was along with common debugging ideas that ar...
[Idea] Focal Gradient Loss: Are we looking at focal loss correctly?
In this post I discuss a new loss function for single stage object detectors that like retinanets focal loss is adaptive, but works at the gradient level rat...
[Idea] Intermediate Loss Sampling: A Differentiable Categorical Sampler
I propose a novel sampling approach, leveraging an intermediate loss function to differentiate through a categorical draw. I compare this method to the commo...
[Rand] ANNagram: Neural Nets for Anagrams
ANNagram is a neural network to take a set of letters and output a possible word. I use a pointer network with a transformer backbone for this task. In the p...