KentonMurray/ProxGradPytorch
PyTorch implementation of Proximal Gradient Algorithms a la Parikh and Boyd (2014). Useful for Auto-Sizing (Murray and Chiang 2015, Murray et al. 2019). (⭐ 42)
Sampling in Unit Time with Kernel Fisher-Rao Flow
We introduce a new mean-field ODE and corresponding interacting particle systems (IPS) for sampling from an unnormalized target density. The IPS are gradient-free, available in closed form, and only require the ability to sample from a reference dens...
Frank-Wolfe Style Algorithms for Large Scale Optimization
We introduce a few variants on Frank-Wolfe style algorithms suitable for large scale optimization. We show how to modify the standard Frank-Wolfe algorithm using stochastic gradients, approximate subproblem solutions, and sketched decision variables...
johnkorzhuk/grabient
Cosine gradient generator & Palette finder (⭐ 2007)
keijiro/CosineGradient
Cosine gradient generator for Unity (⭐ 401)
Remember and Forget for Experience Replay
Experience replay (ER) is a fundamental component of off-policy deep reinforcement learning (RL). ER recalls experiences from past iterations to compute gradient estimates for the current policy, increasing data-efficiency. However, the accuracy of s...
Dr TIM: Ray-tracer TIM, with additional specialist scientific capabilities
We describe several extensions to TIM, a raytracing program for ray-optics research. These include relativistic raytracing; simulation of the external appearance of Eaton lenses, Luneburg lenses and generalized focusing gradient-index (GGRIN) lenses,...
Abhinandan-Kushwaha/react-native-gifted-charts
The most loved library for Bar, Line, Area, Pie, Donut, Stacked Bar, Population Pyramid, Radar, Bubble and Scatter charts in React Native. Allows 2D, 3D, gradient, animations and live data updates. (⭐ 1280)
Sharp $L^2$ estimates for the drift heat equation on shrinking Ricci solitons
We prove an $L^2$ estimate for the drift heat equation on a complete gradient shrinking Ricci soliton. This estimate has a time-dependent weight which is Gaussian in its spatial asymptotics. When transferred and scaled to an estimate for the heat equ...
Efficient Training of Convolutional Neural Nets on Large Distributed Systems
Deep Neural Networks (DNNs) have achieved im- pressive accuracy in many application domains including im- age classification. Training of DNNs is an extremely compute- intensive process and is solved using variants of the stochastic gradient descent...
A Letter on Convergence of In-Parameter-Linear Nonlinear Neural Architectures with Gradien…
This letter summarizes and proves the concept of bounded-input bounded-state (BIBS) stability for weight convergence of a broad family of in-parameter-linear nonlinear neural architectures as it generally applies to a broad family of incremental grad...
Learning Resource Scheduling with High Priority Users using Deep Deterministic Policy Grad…
Advances in mobile communication capabilities open the door for closer integration of pre-hospital and in-hospital care processes. For example, medical specialists can be enabled to guide on-site paramedics and can, in turn, be supplied with live vit...
Impact of correlated observation errors on the convergence of the conjugate gradient algor…
An important class of nonlinear weighted least-squares problems arises from the assimilation of observations in atmospheric and ocean models. In variational data assimilation, inverse error covariance matrices define the weighting matrices of the lea...
Backpropagation - Wikipedia
In machine learning, backpropagation is a gradient computation method commonly used for training a neural network in computing parameter updates. It is
An Approximate Newton Method for Markov Decision Processes
Gradient-based algorithms are one of the methods of choice for the optimisation of Markov Decision Processes. In this article we will present a novel approximate Newton algorithm for the optimisation of such models. The algorithm has various desirabl...
A Retrospective Approximation Approach for Smooth Stochastic Optimization
Stochastic Gradient (SG) is the defacto iterative technique to solve stochastic optimization (SO) problems with a smooth (non-convex) objective $f$ and a stochastic first-order oracle. SG's attractiveness is due in part to its simplicity of executing...
Lion Cub: Minimizing Communication Overhead in Distributed Lion
Communication overhead is a key challenge in distributed deep learning, especially on slower Ethernet interconnects, and given current hardware trends, communication is likely to become a major bottleneck. While gradient compression techniques have b...
Gradient-adjusted Incremental Target Propagation Provides Effective Credit Assignment in D…
Many of the recent advances in the field of artificial intelligence have been fueled by the highly successful backpropagation of error (BP) algorithm, which efficiently solves the credit assignment problem in artificial neural networks. However, it i...
SING: SDE Inference via Natural Gradients
Latent stochastic differential equation (SDE) models are important tools for the unsupervised discovery of dynamical systems from data, with applications ranging from engineering to neuroscience. In these complex domains, exact posterior inference of...
SING: A Plug-and-Play DNN Learning Technique
We propose SING (StabIlized and Normalized Gradient), a plug-and-play technique that improves the stability and generalization of the Adam(W) optimizer. SING is straightforward to implement and has minimal computational overhead, requiring only a lay...