Here’s a laundry list of research topics I’m interested in:
- Democratization, local LLMs and other GenAI, and doing more with low resources
- LoRA and QLoRA
- Efficient learning: TinyStories, curriculum learning
- Model merging
- Adaptive / conditional computation
- Diffusion models
- Various architectures
- Transformers
- JEPA
- SSMs, Mamba, MoE-Mamba, BlackMamba
- Mixture of Experts (MoE)
- Text diffusion, Vec2Text
- Reservoir computing
- GLOM / capsule networks
- Alignment
- Mechanistic interpretability
- Safety and bias considerations
- Addressing shortcomings of existing systems
- Time as a first-class citizen: multimodal, spatiotemporal representation of knowledge / understanding
- World models, reasoning, system 2 thinking and planning
- Sample efficiency
- Context length
- Biological connections and inspirations / AI and neuroscience intersections
- Predictive coding
- The free energy principle
- Fovea and saccades
- Ventral and dorsal vision streams
- Spiking neural networks (SNNs), neuromorphic computing