Course Overview

An intensive, project-driven exploration of foundational Artificial Intelligence algorithms. The coursework utilized the classic Pacman environment to bridge theoretical AI concepts with practical implementations, covering autonomous planning, decision-making under uncertainty, probabilistic tracking, and modern deep learning architectures.

Core Project 1: State-Space Search & Adversarial Planning

Engineered autonomous agents capable of navigating complex deterministic mazes and surviving stochastic, adversarial environments.

  • Heuristic Search Strategies: Implemented uninformed search (DFS, BFS, UCS) and informed search (A* Search). Designed highly optimized, custom admissible and consistent heuristics to solve complex state-space problems, such as finding the optimal path to consume all food dots and navigating through specific corner coordinates.
  • Adversarial Game Tree Search: Modeled adversarial ghost behaviors by developing multi-agent search algorithms. Implemented Minimax with Alpha-Beta Pruning to drastically reduce the expanded search space, and Expectimax to account for the stochastic, sub-optimal nature of random ghost movements. Designed custom evaluation functions to dynamically weigh game states.

Core Project 2: Reinforcement Learning & MDPs

Modeled the Pacman environment as a Markov Decision Process (MDP) to develop agents that learn optimal policies through trial and error.

  • Offline Planning: Implemented Value Iteration and an optimized Prioritized Sweeping Value Iteration to compute precise state values and extract optimal policies in environments with known transition models.
  • Model-Free Q-Learning: Developed an online, model-free agent using Q-Learning with an $\epsilon$-greedy exploration strategy to balance exploration and exploitation.
  • Approximate Q-Learning: Scaled the agent to handle massive, computationally intractable state spaces by implementing feature-based Approximate Q-Learning, utilizing linear weight extraction to generalize learning across similar states.

Core Project 3: Probabilistic Inference (Ghostbusters)

Architected a probabilistic tracking system to locate invisible ghosts based on noisy, time-dependent sensor readings using Bayesian Networks and Hidden Markov Models (HMMs).

  • Exact Inference: Implemented generic Variable Elimination (joining and eliminating factors) to compute exact marginal probabilities in complex Bayesian Networks.
  • Hidden Markov Models: Engineered a dynamic tracking system using the Forward algorithm. Fused noisy sensor observation data (Observation Probability) with ghost transition models (Time Elapse) to maintain an accurate belief distribution of ghost locations.
  • Approximate Inference: Overcame the computational bottlenecks of exact inference in large state spaces by implementing Particle Filtering, maintaining a set of discrete samples to approximate complex continuous probability distributions over time.

Core Project 4: Deep Learning Foundations

Constructed and trained various neural network architectures from scratch using PyTorch to solve classification and sequence generation tasks.

  • Feedforward & Convolutional Networks: Built and optimized non-linear regression models and multi-class classifiers for digit recognition. Designed and trained Convolutional Neural Networks (CNNs) to extract spatial hierarchies from image data.
  • Sequence Modeling & Attention: Addressed variable-length data processing by implementing sequence models for Language Identification. Explored modern transformer foundations by implementing custom Attention mechanisms to build a character-level generative model (character-GPT).

Technical Stack

  • Languages: Python
  • Concepts: A* Search, Admissible Heuristics, Alpha-Beta Pruning, Markov Decision Processes (MDP), Q-Learning, Hidden Markov Models (HMM), Variable Elimination, Particle Filtering.
  • Tools: PyTorch, NumPy.