I’m grateful for John Schulman’s Opinionated Guide to ML Research, if only for articulating the difference between idea-driven and goal-driven research. This is something that took me the first year and a half of my Ph.D. to figure out, and I’m still trying to pin down my own “research taste” and long-term goals.

I’ve decided to post a few of my goals here, forcing myself to organize and critique my thoughts while simultaneously putting myself on the public record (which will be fun to look back at in a few years). They’re vaguely ordered by the likeliness that I’ll spend time on them.

1. Develop more compression methods with provable approximation guarantees.

2. Make AI Dungeon 2 more semantically coherent and game-like.

3. Achieve BERT-level performance from scratch for less than $100 of compute. (Currently$7,000).
4. Align the compression community on comparable metrics.

5. Solve MNIST with a quantum computer simulator
6. Improve machine translation in low-resource domains.
• Why does domain adaptation help?
• What is a principled way of doing domain adaptation?
• How is this related to fine-tuning pre-trained models?

7. Bring theory back to DL (not my field)

Writing this list revealed how very few real or novel problems I am aware of. Most problems I know about (especially #6 and #7) come from reading the literature. They’re idea-driven rather than goal-driven research, almost by definition. I’m excited to get back out into the real world this summer and get some exposure to more novel problems.