Before we even begin to understand what "concurrency" is in Go, we need to make sure we understand that concurrency is not parallelism. Parallelism is means parallel execution of processes where as concurrency is about design. With concurrency you can:
- Design your program as a collection of independent processes.
- Design these processes to eventually run in parallel.
- Design your code so that the outcome is always the same.
Do not worry if you don't understand the above explanation, we will take a deep dive into concurrency patterns in this blog with real examples.
What can we achieve with Concurrency?
- We can have multiple groups of code(workers) running independent tasks.
- We can eliminate race conditions.
- We can eliminate deadlocks.
- The more workers we have, the faster the execution is.
Goroutines
To understand concurrency, we must understand what Goroutines are. A Goroutine is a function or a method which executes independently and simultaneously in connection with any other Goroutines present in your program.
So we could also say, every concurrently running process in Go language is known as a Goroutine.
func main (){
go makeHttpCall("http://xyz.com")
// π makes this method a GoRoutine
}
func makeHttpCall(link string) {
_, err := http.Get(link)
if err == nil {
fmt.Printf("The link %v is up", link)
}
}
Code explanation:
The main function invokes the makeHttpCall() function with a keyword go in front of it. This makes this function run on in a Goroutine. You can consider a Goroutine like a light weighted thread. The cost of creating Goroutines is very small as compared to a thread.
The main function also has its own routine which we do not have to define, known as main Goroutine. All the other Goroutines are working under the main Goroutine. If the main Goroutine is terminated, all the other Goroutines present in the program are also terminated.
Where are we going with this?
Let us say we want to fetch data from multiple links without using Goroutines.
func main() {
// creating a slice of links
links := []string{
"http://abc.com",
"http://pqr.com",
"http://xyz.com",
}
// fetching data from each link
for _, link := range links {
makeHttpCall(link)
}
}
func makeHttpCall(link string) {
_, err := http.Get(link)
if err == nil {
fmt.Printf("The link %v is up", link)
}
}
Code explanation:
This code will iterate over the slice of links and sequentially make http calls on each of the links. In case on of the links takes x seconds to retrieve the response, the other main go routine will block for x seconds before it sends the next request.
We can easily make use of Goroutines to fix this. Let us see the code and then we will discuss how it helps us.
func main() {
// creating a slice of links
links := []string{
"http://abc.com",
"http://pqr.com",
"http://xyz.com",
}
// fetching data from each link
for _, link := range links {
go makeHttpCall(link) // π
}
}
func makeHttpCall(link string) {
_, err := http.Get(link)
if err == nil {
fmt.Printf("The link %v is up", link)
}
}
Code explanation:
We made use of the same go keyword we discuss earlier to run the makeHttpCall function in its own Goroutine. Now the main Goroutine will not wait for the request to be resolve, rather it will keep on iterating and creating separate Goroutine for each invocation of makeHttpCall.
Here's the catch!
We might think that this solves the issue, right? But it doesn't. The output of this function would be:
π Absolutely nothing. Why is that? #[1]
This is because, the main go routine (tied to the main function by default) creates the Goroutine for each makeHttpCall function invocation. Once the iteration is completed, the main function continues to run and reaches the end. It doesn't wait for the other Goroutines to finish.
Remember, we discussed a few minutes ago "If the main Goroutine is terminated, all the other Goroutines present in the program are also terminated". This is exactly what happened. So how do we solve it?
To solve this, we need a way to communicate between these Goroutines and the main Goroutine. And to communicate between multiple Goroutines, we make use of something called channels.
Channels
Go provides a mechanism called a channel that is used to share data between goroutines. Channels act as a pipe between the goroutines and provide a mechanism that guarantees a synchronous exchange.
There are two types of channels:
- Unbuffered channels (which we will be using for the example)
- Buffered channels
Unbuffered channels are used to perform synchronous communication within the goroutines. These provide a guarantee that an exchange of the data is performed at the instant it is sent.
In go we declare the channels and we also must specify the data-type at the time of the channel declaration. The data-type is the type of the data that will be shared through the channel.
myChannel := make(chan string)
This means, this channel can be used to only share the data of type string.
Buffered channels are used to perform asynchronous communication within the goroutines.
myBufferedChannel := make(chan string, 10)
Buffered channels (you may skip this section)
In the buffered channels there is a capacity to hold one or more values before they're received. The sending and receiving is not performed synchronously and immediately.
The blocking cases:
- The receive will block when there is no value in the channel to receive.
- The send will block when there is no
available bufferto place the value being sent.
Code example:
package main
import (
"fmt"
"io/ioutil"
"log"
"net/http"
"sync"
)
func main() {
// initializing a WaitGroup
var wg sync.WaitGroup
// adding 3 counts/buffer to the WaitGroup
wg.Add(3)
fmt.Println("Start Goroutines")
go responseSize("https://www.golangprograms.com", &wg)
go responseSize("https://stackoverflow.com", &wg)
go responseSize("https://coderwall.com", &wg)
// wait for goroutines to finish
wg.Wait()
fmt.Println("Terminating the main program")
}
// just prints the response size of the body returned
func responseSize(url string, wg *sync.WaitGroup) {
// schedule the Done() call when the goroutine is finished
defer wg.Done()
fmt.Println("Step1: ", url)
response, err := http.Get(url)
if err != nil {
log.Fatal(err)
}
fmt.Println("Step2: ", url)
body, err := ioutil.ReadAll(response.Body)
if err != nil {
log.Fatal(err)
}
fmt.Println("Step3: ", len(body))
}
The output for the above program would be:
Start Goroutines
Step1: https://coderwall.com
Step1: https://www.golangprograms.com
Step1: https://stackoverflow.com
Step2: https://stackoverflow.com
Step2: https://www.golangprograms.com
Step3: 31857
Step3: 207321
Step2: https://coderwall.com
Step3: 189752
Terminating the main program
This is quite intuitive how it worked. For the people who did not understand this, three different goroutines were spun up and the main program, waited (using wait group) for the all the goroutines to finish before terminating the program.
Back to the main topic π
We learned about channels and how they act as pipes to communicate data within the goroutines. Let us continue with the issue at hand #[1], that is the how to stop code to from terminating before goroutines are finished executing and get access to which link is down or not.
So in this code, we will make use of channels and communicate the main goroutine (running by default in the main function) and the goroutine spun up during the helper function invocation:
func main() {
// creating a channel to share string type data
myChanel := make(chan string)
// creating a slice of links
links := []string{
"http://abc.com",
"http://pqr.com",
"http://xyz.com",
}
// fetching data from each link
for _, link := range links {
go makeHttpCall(link, myChanel)
}
// listening for three messages coming from the chanel
for i := 0; i <3; i++ {
fmt.Printf("Link %v is up \n", <- myChanel)
}
}
func makeHttpCall(link string, myChanel chan string) {
_, err := http.Get(link)
if err == nil {
// sending the link name to the chanel π
myChanel <- link
}
}
The output of the above program is:
Link http://abc.com is up
Link http://xyz.com is up
Link http://pqr.com is up
Code explanation:
So here, we listened (3 times) for the message to receive (synchronously) from the goroutines from the channel.
In other words, the main function's goroutine waited for the chanel to receive a message, printed out the print statement, and then went on to receive the next message from the channel and repeated itself three times.
That's all there is to know the basics of goroutines and concurrency patterns. There is a lot more to learn. So, in the future blogs, we will take a deep dive on how everything is being handled behind the scenes. Until then, peace β
Top comments (2)
This is a great explanation of a topic I'm just getting familiar with in Go. Thanks!
I will write more blogs on channels and backend microservices development v soon.