Classification is one of the main areas of study today, due to increased emphasis on developing technologies that resemble human behavior. With advancements in the study of Artificial Intelligence, Supervised Machine Learning has always gained attention due to simulating behavior with that to the humans. For this, many classification techniques have been proposed out of which classifying the data with Support Vector Machine (SVM) has made a significant contribution in the field of classification. However, the researchers are skeptic about the performance of SVM due to problems like over-fitting, pair-wise classification and regularization of parameters. For such regularization, a set of algorithms called, the Meta-heuristic algorithms can reach a solution by iteratively updating the candidate solution and finding an optimal solution to a problem, by optimizing the objective function. In this paper, the parameters of SVM are optimized with the help of Firefly algorithm (FFA), which by evaluating its performance, is deduced to outperform the performance of other meta-heuristic algorithms named Particle Swarm Optimization (PSO) and Accelerated PSO (APSO). Experiments have been conducted on a variety of datasets, collected from the UCI repository.