The Daily Ink — AI Research, Simply Explained

tl;dr — Machine Learning research is booming right now. SOTA records are broken every day, and some of the world’s smartest people are working to advance the field. Machine learning research is also opaque, inaccessible, and filled with unwritten institutional knowledge. The Daily Ink is my attempt to change this. The latest in AI/ML research, explained in simple language, in your inbox every Monday/Wednesday/Friday. Check out the posts from the last couple of months, and subscribe if this sounds up your alley :)

Why Start Writing?#

I think Philip J Guo’s “The Ph.D. Grind” describes the process of getting into research very well. Imagine: Your advisor has been in this playground for multiple decades, you’ve just been tossed in, and all you can do is read a lot and try to stay afloat. Much like philosophy, all research references itself. Going back to the fundamentals— the Platos of ML (that’s probably the classic LeNet or MLP papers) feel archaic and hard to place in context. Modern research feels extremely tangled up. It’s very hard to start! I was thankful to have mentors who held my hand through this process.

But not everyone has the fortune to go to a world-class research institution, make friends with kind and welcoming graduate students, or have the time to invest in struggling with reading research until it clicks. This does not mean they shouldn’t have access to it, however — because research might seem like a scary beast, but some of the most complex concepts are quite elegant and simple at their core. I think everybody even mildly interested in ML should have access to understanding this core.

ML Research is overwhelming in quantity#

Here’s something crazy: more than 10,000 papers are published in ML every year. That’s roughly 30 per day. A normal human with a full-time job has no hope of keeping up with this velocity. Part of my mission with the Daily Ink is to be a filter in the noise of research, similar to @_akhaliq and @DAIRInstitute on Twitter. The Daily Ink will cover the latest hottest papers (pre-print, conferences, and otherwise) as well as some of the classics of NLP/ML once in a while.

ML Research is often alchemy#

One of my professors at Berkeley once likened ML to alchemy. I find the analogy quite appropriate. A lot of times, things in ML are empirical and “just work”, even if we don’t have an underlying theory for why. Understanding these trends and heuristics is distinct from understanding the papers — you can do the latter without the former. Through writing about papers, I intend to introduce people to the “insider knowledge” that is often brushed away in research papers.

ML Research is the hottest thing on the fucking planet#

There are a lot of unsolved problems in ML and LLMs right now. It is exciting and enthralling to see small improvements make massive downstream effects. For example, a reduction in I/O cost led to the first Language Model breakthrough in extending context length. The more equitable we make machine learning, the more people can solve these problems, and the better we can make the models we are creating. Making cutting-edge ML accessible is essential to the future of ML.

What’s Next?#

I’ve always loved teaching — I did a lot of it at Berkeley, and I intend to keep doing a lot more. Beyond the paper reviews, here’s what you should expect in the next few months: 1. Deep dives into fundamental concepts like embeddings or feature engineering 2. Tutorials to get started with making your own models 3. Other people!!

It’s a really exciting time to be in ML. I hope you’ll consider joining me on this journey. I’ll see you the next odd day of the week, as we get ever closer to creating AGI together ;)