Machine Learning

• Machine Learning Methods generally require the following four components:

  • Data
  • Models
  • Objective
  • Optimization Algorithms in Machine Learning.

• The three classic paradigms are suited for different problems:

  • Supervised Learning - for finding functional mappings between features and labels.
  • Unsupervised Learning - for understanding the underlying structure of the data .
  • Reinforcement Learning - for developing policies and behaviors.

• Machine Learning models have inductive biases - assumptions that the model uses to perform a prediction task wherein the model prefers a certain class of solutions.

• The No Free Lunch Theorem - any learning algorithm generalizes better on data with certain distributions and worse with other distributions.

• Machine Learning models can capture patterns within systems.

Topics §

Theory §

• Model Performance

• Multimodal Models

• Pathologies of Deep Learning

• Geometric Deep Learning

• [[Low Level of Training a Model]

Neural Network Architectures §

• Neural Network
• Convolutional Neural Network
• Recurrent Neural Network
• Transformer Model
• Generative Adversarial Network
• Diffusion Network
• Mamba Model

Domains and Applications §

• AI in Education
• Machine Learning and Mathematical Reasoning
• Computer Vision
• Natural Language Processing

Tools §

• HuggingFace- an open source platform provider of machine learning technologies. They also provide datasets.
• PyTorch - a machine learning framework. Opinon: More developer friendly than TensorFlow
• sentencepiece- a tokenizer that supports byte pair encoding. It allows ofr a purely end-to-end system that does not depend on language-specific pre/post-processing
• tiktoken- a fast byte pair encoding tokenzier

Links §

• Dive into Deep Learning by Zhang, Lipton, Li and Smola - An excellent introductory book about deep learning.

• Running Neural Networks on Meshes of Light

• AAAI - contains a collection of conference papers about AI and ML

• ICML proceedings - also has Tutorials.

• ICLR proceedings

• IJCAI proceedings

• NeuroIPS - another trustedconference for AI/ML matters.

• Notes on AI - a published obsidian vault containing notes on ML

• Lil’Log - a knowledge vault containing detailed notes on ML from a member of the OpenAI safety team

• paperswithcode - A site that archives different ML research papers.