goroutines

Goroutines

Goroutines allow us to write concurrent programs in Go. Things like web servers handling thousands of requests or a website rendering new pages while also concurrently making network requests are a few example of concurrency.

In Go, each of these concurrent tasks are called Goroutines.

Best Practices for Using Goroutines in Go

Control the Number of Goroutines: Use worker pools or rate-limiting to prevent resource exhaustion.
Communicate via Channels: Use channels to safely pass data between goroutines and avoid shared memory.
Manage Goroutine Lifecycles: Use context for cancellation and defer for cleanup.
Prevent Goroutine Leaks: Ensure all goroutines exit properly and channels are closed.
Synchronize with WaitGroup: Use sync.WaitGroup to wait for multiple goroutines to complete.
Handle Panics: Recover from panics within goroutines to avoid crashes.
Optimize Scheduling: Avoid unnecessary blocking in critical sections and yield to the scheduler when needed.
Test Concurrent Code: Use the race detector and stress-test your code to ensure robustness.

1. Limit the Number of Goroutines

Avoid Spawning Uncontrolled Goroutines: Spawning too many goroutines can exhaust system resources, leading to performance degradation. Use worker pools or rate-limiting techniques to control the number of active goroutines.

Example: Use a worker pool pattern to limit the number of concurrent workers.

package main

import (
    "fmt"
    "sync"
)

func worker(id int, jobs <-chan int, wg *sync.WaitGroup) {
    defer wg.Done()
    for job := range jobs {
        fmt.Printf("Worker %d processing job %d\n", id, job)
    }
}

func main() {
    const numWorkers = 3
    jobs := make(chan int, 5)
    var wg sync.WaitGroup

    for i := 1; i <= numWorkers; i++ {
        wg.Add(1)
        go worker(i, jobs, &wg)
    }

    for j := 1; j <= 5; j++ {
        jobs <- j
    }
    close(jobs)

    wg.Wait() // Wait for all workers to finish
}

2. Use Channels for Communication and Synchronization

Leverage Channels: Channels are the preferred way to synchronize and communicate between goroutines. Avoid using shared memory or global variables to exchange data between goroutines, as this can lead to race conditions.

Example: Use channels to pass data between goroutines safely.

package main

import (
    "fmt"
    "sync"
)

func sum(nums []int, ch chan<- int, wg *sync.WaitGroup) {
    defer wg.Done()
    total := 0
    for _, num := range nums {
        total += num
    }
    ch <- total
}

func main() {
    nums := []int{1, 2, 3, 4, 5, 6}
    ch := make(chan int)
    var wg sync.WaitGroup

    wg.Add(1)
    go sum(nums[:len(nums)/2], ch, &wg)

    wg.Add(1)
    go sum(nums[len(nums)/2:], ch, &wg)

    go func() {
        wg.Wait()
        close(ch)
    }()

    result := 0
    for partial := range ch {
        result += partial
    }

    fmt.Println("Total sum:", result)
}

3. Properly Handle Goroutine Lifecycles

Always Clean Up Resources: Use defer to ensure that resources like channels, files, or locks are properly released when a goroutine exits.
Use Contexts for Cancellation: Utilize the context package to manage the lifecycle of goroutines, particularly for cancellation and timeouts. This allows you to stop goroutines gracefully.

Example: Use context to cancel goroutines.

package main

import (
    "context"
    "fmt"
    "time"
)

func worker(ctx context.Context, id int) {
    for {
        select {
        case <-ctx.Done():
            fmt.Printf("Worker %d stopping\n", id)
            return
        default:
            fmt.Printf("Worker %d working\n", id)
            time.Sleep(500 * time.Millisecond)
        }
    }
}

func main() {
    ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
    defer cancel()

    for i := 1; i <= 3; i++ {
        go worker(ctx, i)
    }

    time.Sleep(3 * time.Second)
    fmt.Println("Main function done")
}

4. Avoid Goroutine Leaks

Ensure Goroutines Exit: Always ensure that your goroutines exit when they are supposed to. This can be achieved by checking for signals or using channels and context.
Close Channels: If a goroutine is waiting on a channel, ensure that the channel is closed when no more data will be sent. This prevents goroutines from blocking indefinitely.

Example: Prevent leaks by ensuring proper channel closure.

package main

import (
    "fmt"
    "time"
)

func worker(ch <-chan int) {
    for job := range ch {
        fmt.Println("Processing job", job)
    }
    fmt.Println("Worker finished, channel closed")
}

func main() {
    ch := make(chan int)
    go worker(ch)

    for i := 1; i <= 5; i++ {
        ch <- i
    }
    close(ch) // Close the channel to signal completion

    time.Sleep(1 * time.Second)
}

5. Use Sync.WaitGroup for Waiting

Sync Multiple Goroutines: When you need to wait for multiple goroutines to finish, use sync.WaitGroup. This ensures that your main goroutine waits until all spawned goroutines complete their work.

Example: Use sync.WaitGroup to wait for goroutines to finish.

package main

import (
    "fmt"
    "sync"
)

func worker(id int, wg *sync.WaitGroup) {
    defer wg.Done()
    fmt.Printf("Worker %d starting\n", id)
    // Simulate work
    fmt.Printf("Worker %d done\n", id)
}

func main() {
    var wg sync.WaitGroup

    for i := 1; i <= 3; i++ {
        wg.Add(1)
        go worker(i, &wg)
    }

    wg.Wait() // Wait for all goroutines to finish
    fmt.Println("All workers done")
}

6. Handle Panics Inside Goroutines

Recover from Panics: Panics inside goroutines can terminate your program if not handled properly. Use defer and recover to catch and handle panics within goroutines to avoid crashes.

Example: Use recover to handle panics in goroutines.

package main

import (
    "fmt"
    "sync"
)

func worker(id int, wg *sync.WaitGroup) {
    defer wg.Done()
    defer func() {
        if r := recover(); r != nil {
            fmt.Printf("Worker %d recovered from panic: %v\n", id, r)
        }
    }()

    fmt.Printf("Worker %d starting\n", id)
    // Simulate a panic
    if id == 2 {
        panic("something went wrong")
    }
    fmt.Printf("Worker %d done\n", id)
}

func main() {
    var wg sync.WaitGroup

    for i := 1; i <= 3; i++ {
        wg.Add(1)
        go worker(i, &wg)
    }

    wg.Wait()
    fmt.Println("All workers done")
}

7. Consider Goroutine Scheduling

Avoid Blocking Operations: Avoid blocking operations (e.g., I/O, time.Sleep) in critical sections of your goroutines, as they can delay the execution of other goroutines and reduce concurrency.
Yield to Scheduler: If you have a tight loop in a goroutine, consider adding a short sleep or runtime.Gosched() to yield to the Go scheduler, allowing other goroutines to run.

8. Test Concurrent Code Thoroughly

Use Race Detector: The Go race detector (go run -race) can identify race conditions in your code. Always run it when working with concurrent code.
Test with Different Loads: Test your concurrent code under different loads and scenarios to ensure it behaves correctly under stress.