Works
Below is a list of my major projects. Most of them are open-sourced. Please feel free to share your comments or contribute directly on GitHub.
The code for every project here was written from scratch, by hand, without AI assistance.
001/011
RetNet
Independent implementation of RetNet, the Transformer successor with parallel training and low-cost inference, including xPos relative positional encoding.
Discover
002/011
Training Binary Neural Networks
Reproduction of the Bayesian Learning Rule for BNNs (BayesBiNN vs. STE) with an end-to-end trainer. ML Reproducibility Challenge 2020, published in ReScience C.
Discover
003/011
Byte-level BPE
From-scratch C++ recreation of Byte Pair Encoding and byte-level BPE, the tokenizers behind modern language models.
Discover
004/011
Pagerank with MapReduce
Google's official PageRank algorithm implementation for distributed machines.
Discover
005/011
Generative Networks
An open-source general implementation of some common generative models (GANs and VAEs).
Discover
006/011
MAML in Pytorch
An open-source implementation of MAML algorithm for supervised learning in Pytorch.
Discover
007/011
Tic-Tac-Toe MCTS
An AI bot for the game Tic-Tac-Toe using modified Monte Carlo Tree Search.
Discover
008/011
Vector Space Retrieval
End-to-End retrieval system to perform toy-scale evaluation of Vector-space models.
Discover
009/011
Matrix game analysis
Implementation and analysis of Matrix Game, a toy problem for analyzing Multi-Agent RL.
Discover
