Local Search Algorithm in Artificial Intelligence

Local Search Algorithms in Artificial Intelligence are optimization techniques that improve a solution by repeatedly moving to a better neighbouring state. Instead of exploring every possible path, they focus on finding efficient and practical solutions for complex problems.

• Improve solutions through neighbouring states
• Useful for optimization and decision-making problems
• Commonly used in scheduling, routing, and machine learning tasks

Basic Terminologies

• State: A possible solution to the problem
• Current State: The solution currently being evaluated
• Neighbour State: A solution formed by making small changes to the current state
• Objective Function: A function used to measure the quality of a solution
• Local Optimum: The best solution among nearby states
• Global Optimum: The best possible solution in the entire search space

Working

1. Pick a starting point: Start with a possible solution which is often random but sometimes based on rule.

2. Find the Neighbours:

• Neighbours are similar solutions we can get by making small, simple changes to the current one.
• For example, in a puzzle, swapping two pieces creates a neighbour.

3. Compare: Look around at all neighbors to see if any are better.

4. Move: If a better neighbor exists, move to it, making it our new “current” solution.

5. Repeat: Keep searching from the new point, following the same steps.

6. Stop: When none of the neighbors are better or after enough tries.

Types of Local Search Algorithms

1. Hill-Climbing Search Algorithm

Hill-Climbing search algorithm is a simple local search algorithm that continuously moves toward a better neighboring solution until no improvement is possible.

Process:

• Start: Begin with an initial solution.
• Evaluate: Assess the neighboring solutions.
• Move: Transition to the neighbor with the highest objective function value if it improves the current solution.
• Repeat: Continue this process until no better neighboring solution exists.

Pros:

• Easy to implement.
• Works well in small or smooth search spaces.

Cons:

• May get stuck in local optima.
• Limited exploration of the search space.

import random


def f(x):
    return - (x - 3)**2 + 5


def hill_climb():
    current_x = random.uniform(0, 6)
    step_size = 0.1
    max_iterations = 100
    for i in range(max_iterations):
        neighbors = [current_x + step_size, current_x - step_size]
        neighbors = [x for x in neighbors if 0 <= x <= 6]
        neighbor_scores = [f(x) for x in neighbors]
        best_neighbor_idx = neighbor_scores.index(max(neighbor_scores))
        best_neighbor = neighbors[best_neighbor_idx]
        if f(best_neighbor) > f(current_x):
            current_x = best_neighbor
        else:
            break
    return current_x, f(current_x)


result_x, result_value = hill_climb()
print(f"Found maximum at x = {result_x:.2f}, value = {result_value:.2f}")

Output: Found maximum at x = 3.02, value = 5.00

2. Simulated Annealing

Simulated Annealing is a local search algorithm inspired by the heating and cooling process in metallurgy. It occasionally accepts worse solutions to escape local optima, with the acceptance probability decreasing over time.

Process:

• Start: Begin with an initial solution and an initial temperature.
• Move: Transition to a neighboring solution with a certain probability.
• Cooling Schedule: Gradually reduce the temperature over time.
• Probability Function: Accept worse solutions with decreasing probability as temperature lowers.

Pros:

• Helps escape local optima due to probabilistic acceptance of worse solutions.
• Explores the search space more effectively.

Cons:

• Requires careful parameter tuning.
• Computationally expensive due to repeated evaluations.

import math
import random


def f(x):
    return - (x - 3)**2 + 5


def get_neighbor(x, step_size=0.1):
    return x + random.uniform(-step_size, step_size)


def simulated_annealing():
    current_x = random.uniform(0, 6)
    best_x = current_x
    best_eval = f(current_x)
    temp = 10
    max_iterations = 1000
    for i in range(max_iterations):
        t = temp / float(i + 1)
        candidate = get_neighbor(current_x)
        candidate = max(0, min(6, candidate))
        candidate_eval = f(candidate)
        if candidate_eval > best_eval or random.random() < math.exp((candidate_eval - best_eval) / t):
            current_x = candidate
            best_eval = candidate_eval
            best_x = current_x
    return best_x, f(best_x)


result_x, result_value = simulated_annealing()
print(f"Best found x = {result_x:.2f}, value = {result_value:.2f}")

Output: Best found x = 3.02, value = 4.96

3. Genetic Algorithms

Genetic Algorithms (GAs) are inspired by the process of natural selection and evolution. They work with a population of solutions and evolve them over time using genetic operators like selection, crossover and mutation.

Process:

• Initialize: Start with a population of random solutions.
• Evaluate: Assess the fitness of each solution.
• Select: Choose the best solutions for reproduction based on their fitness.
• Crossover: Combine pairs of solutions to produce new offspring.
• Mutate: Apply random changes to offspring to maintain diversity.
• Replace: Form a new population by selecting which solutions to keep.

Pros:

• Can explore a broad solution space and find high-quality solutions.
• Suitable for complex problems with large search spaces.

Cons:

• Can be computationally expensive
• Requires tuning of various parameters like population size and mutation rate.

import random


def f(x):
    return - (x - 3)**2 + 5


def genetic_algorithm():
    population = [random.uniform(0, 6) for _ in range(20)]
    max_generations = 50
    for _ in range(max_generations):
        scores = [f(x) for x in population]
        best = population[scores.index(max(scores))]
        new_population = [best]  # keep best
        while len(new_population) < len(population):
            parents = random.sample(population, 2)
            child = (parents[0] + parents[1]) / 2
            # mutation: small random step
            if random.random() < 0.3:
                child += random.uniform(-0.2, 0.2)
                child = max(0, min(6, child))
            new_population.append(child)
        population = new_population
    scores = [f(x) for x in population]
    best = population[scores.index(max(scores))]
    return best, f(best)


result_x, result_value = genetic_algorithm()
print(f"Best found x = {result_x:.2f}, value = {result_value:.2f}")

Output: Best found x = 3.00, value = 5.00

4. Tabu Search

Tabu Search enhances local search by using a memory structure called the tabu list to avoid revisiting previously explored solutions. This helps to prevent cycling back to local optima and encourages exploration of new areas.

Process:

• Start: Begin with an initial solution and initialize the tabu list.
• Move: Transition to a neighboring solution while considering the tabu list.
• Update: Add the current solution to the tabu list and potentially remove older entries.
• Aspiration Criteria: Allow moves that lead to better solutions even if they are in the tabu list.

Pros:

• Reduces the chance of getting stuck in local optima.
• Effective in exploring large and complex search spaces.

Cons:

• Requires careful management of the tabu list and aspiration criteria.
• Computational complexity can be high.

import random


def f(x):
    return - (x - 3)**2 + 5


def tabu_search():
    current_x = random.uniform(0, 6)
    tabu_list = []
    tabu_size = 5
    step_size = 0.1
    max_iterations = 100
    best_x = current_x
    best_eval = f(current_x)
    for _ in range(max_iterations):
        neighbors = [current_x + step_size, current_x - step_size]
        neighbors = [x for x in neighbors if 0 <=
                     x <= 6 and x not in tabu_list]
        if not neighbors:
            break
        neighbor_scores = [f(x) for x in neighbors]
        best_neighbor_idx = neighbor_scores.index(max(neighbor_scores))
        best_neighbor = neighbors[best_neighbor_idx]
        if f(best_neighbor) > best_eval:
            best_x, best_eval = best_neighbor, f(best_neighbor)
        tabu_list.append(current_x)
        if len(tabu_list) > tabu_size:
            tabu_list.pop(0)
        current_x = best_neighbor
    return best_x, f(best_x)


result_x, result_value = tabu_search()
print(f"Best found x = {result_x:.2f}, value = {result_value:.2f}")

Output: Best found x = 3.02, value = 5.00

Comparison of Local Search Algorithms

Applications

• Scheduling: Creating timetables for schools, jobs, or exams while avoiding conflicts
• Routing: Finding efficient paths for delivery and travel problems such as the Traveling Salesperson Problem
• Resource Allocation: Assigning limited resources like machines, rooms, or staff efficiently
• Games and AI: Making fast decisions and strategic moves in complex games
• Machine Learning: Tuning model parameters to improve performance

Advantages

• Require less memory compared to exhaustive search methods
• Work efficiently for large and complex search spaces
• Can quickly find good or near-optimal solutions
• Useful for real-world optimization problems

Limitations

• May get stuck in local optima
• Do not always guarantee the best solution
• Performance can depend on the initial state
• Some algorithms require careful parameter tuning