In this blog post, we will explore the importance of stochastic models in the context of unsupervised learning. We will start with k-means clustering, which deterministically clusters points based on heuristics, and build up to Expectation Maximization (EM), which can use any parametrized probabilistic distribution to cluster data. k-means clustering k-means clustering is a method […]
Category: Assignments
Mastering Markov Decision Processes: A Practical RL Journey with OpenAI Gym
Introduction Reinforcement Learning (RL) is a powerful subset of machine learning where agents interact with an environment to hone their decision-making skills. At the core of RL lie Markov Decision Processes (MDPs), providing a mathematical structure to define states, actions, rewards, and the dynamics of how an environment transitions over time. OpenAI Gym has become […]
No Straight Lines Here: The Wacky World of Non-Linear Manifold Learning
In this era of machine learning and data analysis, the quest to understand complex relationships within high-dimensional data like images or videos is not simple and often requires techniques beyond simple ones. The patterns are complex, twisted and intertwined, defying the simplicity of straight lines. This is where non-linear manifold learning algorithms step in. They […]
Evolution & Taxonomy of Clustering Algorithms
History of Clustering The history of clustering algorithms dates back to the early 20th century, foundationally originating in the realms of anthropology and psychology in the 1930s [1, 2] . It was introduced in anthropology by Driver and Kroeber in 1932 [3] to simplify the ambiguity of empirically based typologies of cultures and individuals. In […]
How to Evaluate Features after Dimensionality Reduction?
Introduction Dimensionality reduction can be a critical preprocessing step that transforms a dataset’s features with high dimensions in input space to much lower dimensions in some latent space. It can bring us multiple benefits when training the model including avoiding the curse of dimensionality issues, reducing the risk of model overfitting, and lowering the computation […]
Navigating the Optimization Landscape: The Do’s and Don’ts of choosing an Optimization Problem
Selecting the right optimization problem is crucial for solving complex challenges, involving the adjustment of model parameters to optimize an objective function in machine learning. Mathematical and computational techniques aim to find the best solution from a set of feasible ones, focusing on objective functions, decision variables, and constraints. Optimization enhances machine learning models through training, hyperparameter tuning, feature selection, and cost function minimization, directly affecting accuracy and performance. This process necessitates an understanding of problem specifics, appropriate metric selection, and computational complexity consideration, while avoiding pitfalls like unclear objectives and overlooking real-world constraints.
Comparative Analysis of Random Search Algorithms
Introduction Local Search Algorithms play a crucial role in Machine Learning by addressing a wide range of optimization problems, as noted by Solis and Wets [1]. These algorithms are especially useful for tasks like hyperparameter optimization or optimizing loss functions. Search algorithms are particularly beneficial in situations where computational resources are limited or the problem […]
Simulated Annealing : Methods and Real-World Applications
Optimization techniques play a critical role in numerous challenges within machine learning and signal processing spaces.This blog specifically focuses on a significant class of methods for global optimization known as Simulated Annealing (SA). We cover the motivation, procedures and types of simulated annealing that have been used over the years. Finally, we look at some real world applications of simulated annealing, not limited to the realms of Machine Learning, demonstrating the power of this technique.
Tutorial on Hyperparameter Tuning Using scikit-learn
Introduction Hyperparameter tuning is a method for finding the best parameters to use for a machine learning model. There are a few different methods for hyperparameter tuning such as Grid Search, Random Search, and Bayesian Search. Grid Search is a search algorithm that performs an exhaustive search over a user-defined discrete hyperparameter space [1, 3]. […]
Introduction to Classification Model Comparison Methods
Introduction In the field of machine learning, much of the focus and tutorials available are on the creation of a specific model and going from start to finish as if the process were a linear path. In reality, the process is more like a hike where you may find yourself exploring different paths just to […]