Here are some small AI / ML experiments I’ve done in the last few years as I work on my coding and implementation skills and test out some of the ideas bubbling around in my head. Much of the work has been in the time since leaving my job to dig into AI full time.

An explicit goal of these last years has been to dabble. When I started I was wary of getting pulled into a major effort that would suck up all my time, so aimed to spend no more than a month or two on individual projects. I also wanted to get my hands dirty in a number of different areas. The result, for what it is worth, is a collection of small projects, some of them unfinished—and several of which I would like to pick back up again if I have the opportunity.

When I embark on one of these experiments, my objective (aside from having a good learning experience) is to find a large effect (like 5 or 10%) or go home. I’m trying to ruthlessly prioritize work—and usually anxious to move on to another idea—so if there’s only a 1% effect, then I’m generally not interested enough to invest more time. I’ll write up just enough to indicate what it is that I was looking into.

As a result, in many cases what you’re seeing is the equivalent of me scribbling some ideas onto paper, realizing I’m heading down a useless path, then wadding up the paper and tossing it in the bin. Live and learn.

My machine has a little ole GPX 1070Ti (8 GiB VRAM). Due to this limitation I’m usually looking for interesting things to do using small models (e.g., GPTNeo-124M) and modest datasets (e.g., subsets of TinyStories).

There’s nothing too exciting here (no breakthrough results), but I want to document some of what I’ve been up to. I also want to get better about doing it in public (an entirely safe link, btw).