In-depth implementation of the Go language defer principle
Table of contents.
This article explains the rules for executing defer and introduces the defer type. It explains how defer function calls are done, mainly through heap allocation.
Introduction
Defer execution rules, the order of execution of multiple defers is “last in first out lifo ".
In the above example, the string Naveen is traversed using a for loop and then defer is called. These defer calls act as if they were stacked, and the last defer call pushed onto the stack is pulled out and executed first.
The output is as follows.
The defer declaration will first calculate the value of the parameter
In this example, the variable i is determined when defer is called, not when defer is executed, so the output of the above statement is 0.
defer can modify the return value of a named return value function
As officially stated.
For instance, if the deferred function is a function literal and the surrounding function has named result parameters that are in scope within the literal, the deferred function may access and modify the result parameters before they are returned.
An example is as follows.
However, it should be noted that only the named return value (named result parameters) function can be modified, and the anonymous return value function cannot be modified, as follows.
Because anonymous return-valued functions are declared when return is executed, only named return-valued functions can be accessed in the defer statement, not anonymous return-valued functions directly.
Types of defer
Go made two optimizations to defer in versions 1.13 and 1.14, which significantly reduced the performance overhead of defer in most scenarios.
Allocation on the heap
Prior to Go 1.13 all defer s were allocated on the heap, a mechanism that at compile time.
- inserting runtime.deferproc at the location of the defer statement, which, when executed, saves the defer call as a runtime._defer structure to the top of the _defer chain of Goroutine.
- runtime.deferreturn is inserted at the position before the function returns, and when executed, the top runtime._defer is retrieved from Goroutine’s _defer chain and executed sequentially.
Allocation on the stack
New in Go 1.13, deferprocStack implements on-stack allocation of defer . Compared to heap allocation, on-stack allocation frees _defer after the function returns, eliminating the performance overhead of memory allocation and requiring only proper maintenance of the chain of _defer . According to the official documentation, this improves performance by about 30%.
Except for the difference in allocation location, there is no fundamental difference between allocating on the stack and allocating on the heap.
It is worth noting that not all defer s can be allocated on the stack in version 1.13. A defer in a loop, whether it is a display for loop or an implicit loop formed by goto , can only use heap allocation, even if it loops once.
Open coding
Go 1.14 added open coding, a mechanism that inserts defer calls directly into functions before they return, eliminating the need for deferproc or deferprocStack operations at runtime. This optimization reduces the overhead of defer calls from ~35ns in version 1.13 to ~6ns or so.
However, certain conditions need to be met in order to trigger.
- the compiler optimization is not disabled, i.e. -gcflags "-N" is not set.
- the number of defer s in the function does not exceed 8 and the product of the return statements and the number of defer statements does not exceed 15.
- the defer keyword of the function cannot be executed in a loop.
defer structure
The parameters to note above are siz , heap , fn , link , openDefer which will be covered in the following analysis.
In this article, we will start with the heap allocation, we will talk about why the execution rules of defer are as described at the beginning, and then we will talk about the stack allocation of defer and the development coding related content.
The analysis starts with a function call as the entry point.
Named function return value calls
Let’s start with the example mentioned above and look at heap allocation from function calls. Note that running the following example on 1.15 does not allocate directly to the heap, but requires you to recompile the Go source code to force the defer to allocate to the heap.
File location: src/cmd/compile/internal/gc/ssa.go
Print the assembly using the command.
First of all, let’s look at the main function, there is nothing to say, it is a very simple call to the f function.
The following subparagraph looks at the calls to the f function.
Since allocation on the defer heap calls the runtime.deferproc function, what is shown in this assembly is an assembly before the runtime.deferproc function is called, which is still very simple to understand.
Because the argument to the runtime.deferproc function is two arguments, as follows.
In the function call process, the parameters are passed from the right to the left of the parameter list stack , so the top of the stack is pressed into the constant 8, in the 8(SP) position is pressed into the second parameter f.func1-f function address.
See here may have a question, in the pressure into the constant 8 when the size is int32 occupies 4 bytes size, why the second parameter does not start from 4 (SP), but to start from 8 (SP), this is because the need to do memory alignment caused.
In addition to the parameters, it should also be noted that the 16(SP) position is pressed into the 40(SP) address value. So the entire pre-call stack structure should look like the following.
Let’s look at runtime.deferproc :
File location: src/runtime/panic.go
When calling the deferproc function, we know that the argument siz is passed in as the value at the top of the stack representing the argument size of 8 and the address corresponding to the 8(SP) passed in as the argument fn.
So the two sentences above are actually a combination of the address value we saved in 16(SP) above into the next 8bytes block of memory immediately below defer as the argument to defer . A simple diagram would look like the following, where the argp immediately below defer actually stores the address value saved in 16(SP).
Note that here the argp value is copied by a copy operation, so the argument is already determined when defer is called , not when it is executed, but here the value of an address is copied.
And we know that when allocated on the heap, defer is stored in the current Goroutine as a chain, so if there are 3 defer s called separately, the last one called will be at the top of the chain.
For the newdefer function, the general idea is to fetch from P’s local cache pool, and if not, fetch half of defer from sched’s global cache pool to fill P’s local resource pool, and if there is still no available cache, allocate new defer and args directly from the heap. The memory allocation here is roughly the same as the memory allocator allocation, so we won’t analyze it again, but you can see for yourself if you are interested.
Let’s go back to the assembly of the f function.
Here it is very simple, write constant 6 directly to 40(SP) as the return value and then call runtime.deferreturn to execute defer .
Let’s look at runtime.deferreturn :
First, note that the argument arg0 passed in here is actually the value at the top of the caller’s stack, so the following assignment actually copies the defer argument to the top of the caller’s stack.
*(*uintptr)(deferArgs(d)) What is stored here is actually the address value saved by the caller 16(SP). Then the caller’s stack frame is shown below.
Go to runtime.jmpdefer to see how this is done.
Location: src/runtime/asm_amd64.s
This assembly is very interesting, the jmpdefer function, since it was called by runtime.deferreturn , now has the following call stack frame
The arguments passed to the jmpdefer function are 0(FP) for the fn function address, and 8(FP) for the SP of the call stack of the f function.
So the following sentence represents the return address of the runtime.deferreturn call stack written to SP.
Then -8(SP) represents the Base Pointer of the runtime.deferreturn call stack.
We will focus on explaining why the value of the SP pointer minus 5 is used to obtain the address value of runtime.deferreturn .
We return to the assembly of the f function call.
Since the runtime.deferreturn function needs to return to the 0x45defd address after the call, the return address in the stack frame corresponding to the runtime.deferreturn function is actually 0x45defd.
In the jmpdefer function, the value corresponding to (SP) is the return address of the runtime.deferreturn call stack, so subtracting 5 from 0x45defd will give you 0x45def8, which is the value of the runtime.deferreturn function. address.
Then when we finally jump to the f.func1 function, the call stack is as follows.
The location of the call stack (SP) actually holds a pointer to the deferreturn function, so after the f.func1 function is called, it returns to the deferreturn function until there is no data in the _defer chain.
Here’s another short look at the f.func1 function call.
The call here is very simple: get the data pointed to by the 8(SP) address value and do the arithmetic, then write the result to the stack and return.
Here we have basically shown you the whole process of calling defer functions through heap allocation. The answer is that the defer argument passed during the defer call is a pointer to the return value, so the return value is modified when defer is finally executed.
Anonymous function return value calls
So what if anonymous return value functions are passed? For example, something like the following.
Print the compilation below.
In the output above, we can see that the anonymous return value function call first writes the constant 100 to 24(SP), then writes the address value of 24(SP) to 16(SP), and then writes the return value to 48(SP) with the MOVQ instruction, which means that the value is copied, not the pointer, and so the return value is not modified.
Here is a diagram comparing the two after calling runtime.deferreturn stack frames.
It is clear that the famous return value function stores the address of the return value at 16(SP), while the anonymous return value function stores the address of 24(SP) at 16(SP).
The above sequence of analysis also answers a few questions in passing.
how does defer pass arguments? We found in the above analysis that when executing the deferproc function, the argument value is first copied to the location immediately adjacent to the defer memory address value as the argument, if it is a pointer pass it will directly copy the pointer, and a value pass will directly copy the value to the location of the defer argument.
Then when the deferreturn function is executed, it copies the parameter values to the stack and then calls jmpdefer for execution.
How are multiple defer statements executed?
When the deferproc function is called to register a defer , the new element is inserted at the head of the table, and execution is done by getting the head of the chain in order.
What is the order of execution of defer, return, and return value?
To answer this question, let’s take the assembly of the output in the above example and examine it.
From this assembly, we know that for
- it is the first to set the return value to the constant 6.
- then runtime.deferreturn will be called to execute the defer chain.
- executing the RET instruction to jump to the caller function.
Stack allocation
As mentioned at the beginning, defer on-stack allocation was added after Go version 1.13, so one difference from heap allocation is that defer is created on the stack via deferprocStack .
Go goes through the SSA stage at compile time, and if it’s a stack allocation, then it needs to use the compiler to initialize the _defer record directly on the function call frame and pass it as an argument to deferprocStack . The rest of the execution process is no different from heap allocation.
For the deferprocStack function let’s look briefly at.
The main function is to assign a value to the _defer structure and return it.
The Go language was optimized in 1.14 by inlining code so that calls to the defer function are made directly at the end of the function, with little additional overhead. In the build phase of SSA buildssa will insert open coding based on a check to see if the condition is met. Since the code in the build phase of SSA is not well understood, only the basics are given below and no code analysis is involved.
We can compile a printout of the example for the allocation on the heap.
We can see in the assembly output above that the defer function is inserted directly into the end of the function to be called.
This example above is easy to optimize, but what if a defer is in a conditional statement that must not be determined until runtime?
The defer bit delay bit is also used in open coding to determine whether a conditional branch should be executed or not. This delay bit is an 8-bit binary code, so only a maximum of 8 defer s can be used in this optimization, including the defer in the conditionals. Each bit is set to 1 to determine if the delay statement is set at runtime, and if so, the call occurs. Otherwise, it is not called.
For example, an example is explained in the following article.
https://go.googlesource.com/proposal/+/refs/heads/master/design/34481-opencoded-defers.md
At the stage of creating a deferred call, it is first recorded which defer with conditions are triggered by a specific location of the deferred bits.
Before the function returns and exits, the exit function creates a check code for the delayed bits in reverse order:
Before the function exits, it determines whether the position is 1 by taking the delayed bits with the corresponding position, and if it is 1, then the defer function can be executed.
This article explains the execution rules of defer and introduces the defer type. The main purpose of this article is to explain how defer function calls are made through heap allocation, such as: function calls to understand “ defer argument passing”, “how multiple defer statements are executed”, “and what is the order of execution of defer, return, and return value”, and other issues. Through this analysis, we hope you can have a deeper understanding of defer.
Go Defer Simplified with Practical Visuals
Learn about golang’s defer statement with various usage examples..
Inanc Gumus
Learn Go Programming
What is defer?
It takes a func and executes it just before * the surrounding func returns whether there is a panic or not.
👉 Go doesn’t need destructors because it has no built-in constructors. This is a good balance.
👉 Defer resembles to “ finally ” however the defer belongs to the surrounding “func” while the finally belongs to an exception “block”.
👊 Bonus: See my comment here about the internals of defer if you’re curious about how it works. It’s actually been documented pragmatically as “it runs — after — the surrounding func returns”, however, there are some other inner details.
Releasing acquired resources
Defer funcs are often used to release the acquired resources inside a func.
The func closes the opened file whether there is an error or not on all returns — marked with the star.
Save us from panic
Defer can recover from a panic to prevent the termination of a program if the panic emitted from the same goroutine.
recover() returns the value provided to panic() which lets you decide what you’d do with it. You can also pass an error or other types of values to panic, then you can check whether the panic was caused by the value you’re looking for. More here .
Deferred closure
A deferred func can be of any type of func . So, when used with an anonymous func — obviously — the func becomes aware of its surroundings.
Notice that it sees the latest state of the surrounding values, check out:
Params evaluation
Go runtime will save any passed params to the deferred func at the time of registering the defer— not when it runs .
Declare a dummy func that registers a deferred closure. It also uses a named result value “n” to increase the passed number for the second time :
What happened?
In some situations defer can help you to change the result value before the return by using the named result values as seen in the example.
Multiple defers
Multiple defers are saved in a stack list. So, the last registered defer will run as the first. Beware: Using multiple defers may hinder the readability.
Watch how it works
Deferred methods.
You can also use methods with defer. However, there’s a quirk. Watch.
Without pointers
With pointers, what’s going on.
Remember that the passed params to a deferred func are saved aside immediately without waiting for the deferred func to be run.
So, when a method with a value-receiver is used with defer, the receiver will be copied ( in this case Car) at the time of registering and the changes to it wouldn’t be visible ( Car.model ). Because the receiver is also an input param and evaluated immediately to “DeLorean DMC-12” when it’s registered with the defer.
On the other hand, when the receiver is a pointer when it’s called with defer, a new pointer is created but the address it points to would be the same with the “c” pointer above. So, any changes to it would be reflected flawlessly.
Alright, that’s all for now. Thank you for reading so far.
Let’s stay in touch:
- 📩 Join my newsletter
- 🐦 Follow me on twitter
- 📦 Get my Go repository for free tutorials, examples, and exercises
- 📺 Learn Go with my Go Bootcamp Course
- ❤️ Do you want to help? Please clap and share the article. Let other people also learn from this article.
Published in Learn Go Programming
Visual, concise and detailed tutorials, tips and tricks about Go (aka Golang).
Written by Inanc Gumus
Coder. Gopher. Maker. Stoic. Since 1992.
More from Inanc Gumus and Learn Go Programming
5 Gotchas of Defer in Go (Golang) — Part I
Protect yourself from basic defer gotchas..
★ Ultimate Visual Guide to Go Enums ★
Golang enums & iota guide—full of tips and tricks with visuals and runnable code examples..
Organizing your code with Go packages — Master Tricks
Learn the master tricks about how to properly design your system with go packages..
About Go Language — An Overview
Learn about the go ecosystem and the language’s overview. as well as its advantages and disadvantages., recommended from medium.
Jessica Stillman
Jeff Bezos Says the 1-Hour Rule Makes Him Smarter. New Neuroscience Says He’s Right
Jeff bezos’s morning routine has long included the one-hour rule. new neuroscience says yours probably should too..
ZhangJie (Kn)
Design Patterns in Go: Visitor
Behavioral patterns target issues related to communication and interaction between objects. they focus on defining protocols for….
General Coding Knowledge
Coding & Development
Stories to Help You Grow as a Software Developer
Why I’m Switching from Go to Python
Hey everyone so, after spending years writing go code (and loving it, honestly), i’ve decided it’s time for a change. yup, i’m switching….
Tarka Labs Blog
Shamil Siddique
Leveraging DTO pattern in Go-based web apps
I’ll start by confessing a certain bias — i love statically-typed languages. the level of control and certainty they offer compared to….
8 Golang Performance Tips I Discovered After Years of Coding
These have saved me a lot of headaches, and i think they’ll help you too. don’t forget to bookmark them for later.
Stackademic
Abdur Rahman
Python is No More The King of Data Science
5 reasons why python is losing its crown.
Text to speech
Welcome to tutorial no. 29 in Golang tutorial series .
What is Defer?
Defer statement is used to execute a function call just before the surrounding function where the defer statement is present returns. The definition might seem complex but it’s pretty simple to understand by means of an example.
Run in playground
The above is a simple program which illustrates the use of defer . In the above program, defer is used to find out the total time taken for the execution of the test() function. The start time of the test() function execution is passed as argument to defer totalTime(start) in line no. 14. This defer call is executed just before test() returns. totalTime prints the difference between start and the current time using time.Since in line no. 9. To simulate some computation happening in test() , a 2 second sleep is added in line no. 15.
Running this program will print
The output correlates to the 2 second sleep added. Before the test() function returns, totalTime is called and it prints the total time taken for test() to execute.
Arguments evaluation
The arguments of a deferred function are evaluated when the defer statement is executed and not when the actual function call is done.
Let’s understand this by means of an example.
In the program above a initially has a value of 5 in line no. 11. When the defer statement is executed in line no. 12, the value of a is 5 and hence this will be the argument to the displayValue function which is deferred. We change the value of a to 10 in line no. 13. The next line prints the value of a . This program outputs,
From the above output it can be understood that although the value of a changes to 10 after the defer statement is executed, the actual deferred function call displayValue(a) still prints 5 .
Deferred methods
Defer is not restricted only to functions . It is perfectly legal to defer a method call too. Let’s write a small program to test this.
In the above program we have deferred a method call in line no. 21. The rest of the program is self explanatory. This program outputs,
Multiple defer calls are placed in stack
When a function has multiple defer calls, they are pushed to a stack and executed in Last In First Out (LIFO) order.
We will write a small program which prints a string in reverse using a stack of defers.
In the program above, the for range loop in line no. 11, iterates the string and calls defer fmt.Printf("%c", v) in line no. 12. These deferred calls will be added to a stack.
The above image represents the content of the stack after the defer calls are added. The stack is a last in first out datastructure. The defer call that is pushed to the stack last will be popped out and executed first. In this case defer fmt.Printf("%c", 'n') will be executed first and hence the string will be printed in reverse order.
This program will print
Practical use of defer
In this section we will look into some more practical uses of defer.
Defer is used in places where a function call should be executed irrespective of the code flow. Let’s understand this with the example of a program which makes use of WaitGroup . We will first write the program without using defer and then we will modify it to use defer and understand how useful defer is.
In the program above, we have created a rect struct in line no. 8 and a method area on rect in line no. 13 which calculates the area of the rectangle. This method checks whether the length and width of the rectangle is less than zero. If it is so, it prints a corresponding error message else it prints the area of the rectangle.
The main function creates 3 variables r1 , r2 and r3 of type rect . They are then added to the rects slice in line no. 34. This slice is then iterated using a for range loop and the area method is called as a concurrent Goroutine in line no. 37. The WaitGroup wg is used to ensure that the main function is waiting until all Goroutines finish executing. This WaitGroup is passed to the area method as an argument and the area method calls wg.Done() in line nos. 16, 21 and 26 to notify the main function that the Goroutine is done with its job. If you notice closely, you can see that these calls happen just before the area method returns. wg.Done() should be called before the method returns irrespective of the path the code flow takes and hence these calls can be effectively replaced by a single defer call.
Let’s rewrite the above program using defer.
In the program below, we have removed the 3 wg.Done() calls in the above program and replaced it with a single defer wg.Done() call in line no. 14. This makes the code more simple and readable.
This program outputs,
There is one more advantage of using defer in the above program. Let’s say we add another return path to the area method using a new if condition. If the call to wg.Done() was not deferred, we have to be careful and ensure that we call wg.Done() in this new return path. But since the call to wg.Done() is defered, we need not worry about adding new return paths to this method.
This brings us to the end of this tutorial. I hope you liked it. Please leave your feedback and comments. Please consider sharing this tutorial on twitter and LinkedIn . Have a good day.
Next tutorial - Error Handling
IMAGES
VIDEO