From 702b430d18e0d6a98f935a1056d8d5763b296c17 Mon Sep 17 00:00:00 2001 From: Igor Bogoslavskyi Date: Sun, 1 Sep 2024 23:46:15 +0200 Subject: [PATCH] Finish lecture about memory --- lectures/memory_and_smart_pointers.md | 109 +++++++++++++++++--------- readme.md | 34 ++++++++ 2 files changed, 105 insertions(+), 38 deletions(-) diff --git a/lectures/memory_and_smart_pointers.md b/lectures/memory_and_smart_pointers.md index d334335..f529864 100644 --- a/lectures/memory_and_smart_pointers.md +++ b/lectures/memory_and_smart_pointers.md @@ -1,8 +1,7 @@ Memory management and smart pointers -- -

- Video + Video

- [Memory management and smart pointers](#memory-management-and-smart-pointers) @@ -14,16 +13,18 @@ Memory management and smart pointers - [Why not keep persistent data on the stack](#why-not-keep-persistent-data-on-the-stack) - [The heap](#the-heap) - [Operators `new` and `delete`](#operators-new-and-delete) - - [Typical pitfalls with data allocated on the heap](#typical-pitfalls-with-data-allocated-on-the-heap) +- [Typical pitfalls with data allocated on the heap](#typical-pitfalls-with-data-allocated-on-the-heap) - [Forgetting to call `delete`](#forgetting-to-call-delete) - [Performing shallow copy by mistake](#performing-shallow-copy-by-mistake) - - [Performing shallow assignment by mistake](#performing-shallow-assignment-by-mistake) - - [Calling a wrong `delete`](#calling-a-wrong-delete) - - [Returning owning pointers from functions](#returning-owning-pointers-from-functions) - - [Best practices for memory safety](#best-practices-for-memory-safety) -- [Smart pointers to the rescue!](#smart-pointers-to-the-rescue) - - [`std::unique_ptr`](#stdunique_ptr) - - [`std::shared_ptr`](#stdshared_ptr) + - [Performing shallow assignment by mistake](#performing-shallow-assignment-by-mistake) + - [Calling a wrong `delete`](#calling-a-wrong-delete) + - [Returning owning pointers from functions](#returning-owning-pointers-from-functions) +- [RAII for memory safety](#raii-for-memory-safety) + - [STL classes use RAII](#stl-classes-use-raii) + - [Smart pointers to the rescue!](#smart-pointers-to-the-rescue) + - [`std::unique_ptr`](#stdunique_ptr) + - [`std::shared_ptr`](#stdshared_ptr) + - [Prefer `std::unique_ptr`](#prefer-stdunique_ptr) - [Smart pointers are polymorphic](#smart-pointers-are-polymorphic) - [Summary](#summary) @@ -48,27 +49,27 @@ We won't focus too much on garbage collection here though, but if you'd like a m While such a garbage collection system is convenient in terms of not needing to think about memory on our side, using it takes away from the performance of our program. These systems must perform a scan for unused memory repeatedly at runtime which costs time. Furthermore it is hard to predict when these scans will happen and how long they will take, so garbage collected languages are not well-suited for safety-critical applications where we need to know exactly when each operation takes place and that they all fall within a certain time budget. ### The C++ way -What makes C++ suited for such safety-critical applications is that it stays away from garbage collection based memory management altogether. Instead it takes a two-tier approach: it manages memory for local variables with limited lifetime very efficiently without any unpredictable actions and provides us with the tools to manage the more complex cases of memory allocation on our own. And while it might not sounds like a benefit to you, the combination of these two systems and especially the fact that we can decide how and when to allocate and free our memory is really what makes C++ so loved. The beauty of C++ is that the way it is designed, alongside with its Standard Template Library (STL), allows us to write extremely efficient code while making sure that the memory is allocated and freed correctly. +What makes C++ suited for such safety-critical applications is that it stays away from garbage collection based memory management altogether. Instead it takes a two-tier approach. -## Memory allocation under the hood -So let us talk about these systems in-depth! +On one hand it makes a good use of scopes and makes sure that any simple small variable allocated withing a scope gets freed by the end of it in an extremely efficient manner. -And before we talk about the tools we have at our disposal to manage memory in modern C++, let's briefly focus on what happens under the hood when we want to create or destroy a variable. And, while common, this task is anything but trivial. +On the other hand, for some variables that need to allocate a large chunk of memory or that have to be still available after the end of the scope in which they were allocated, it allows us to allocate and release memory manually at our discretion. Which in turn allows us implement any behavior of arbitrary complexity. - -Remember, we want to find a perfect place in memory for every variable we want to use. These variables can have vastly different sizes and lifetime durations which influences their optimal placement in memory. Furthermore, we usually want to avoid "fragmentation" of our memory, i.e., we don't want to have many small "holes" between allocated variables. If we have those, it might be that we have a lot of free memory in the absolute sense, but we are still unable to find a continuous chunk of memory to allocate some larger variable. This means that ideally we want to find for each variable being allocated the smallest free memory slot it fits into. +Here I have to mention that with great power comes great responsibility and, historically, most of the fear of C++ among the beginners came from this ability to manage memory manually. It was complicated to think about how this manually allocated memory should be managed, and especially when it should be freed. There was also a lack of good tools and guidance that would be universal for each and every situation, which made learning how to work with memory safely hard. - -Solving this problem in general is very hard! Think about it, we can't scan **all** the memory at our disposal exhaustively every time we want to allocate a variable, that would take too long! But we _can_ find effective algorithms for finding *not perfect*, but _good-enough_ spots, granted that we have some more information about our data. +But thankfully those days are in the past and that's where **modern C++** really comes into play. You see, today we *have* these tools within the Standard Template Library (STL) as well as the accompanying guidance on how to use them that work for nearly any situation that we might encounter! Using these tools and following the best practices allows us thinking more abstractly about what we do: not when to allocate and free memory but what **entities** we want to create, who owns them and how their ownership should be transferred while the program runs. This makes reasoning about our programs much easier while keeping our code efficient and memory-safe by default. -For example, we might notice that we allocate data with **automatic storage duration**, i.e., variables that live within scopes, much more often than data that persist throughout the whole program lifetime. Furthermore, the memory for such data gets freed very soon after these variables were allocated. And we can even observe that such variables are mostly on the smaller size in terms of how much memory they require. +## Memory allocation under the hood +But before we talk about these tools we have at our disposal to manage memory in modern C++, let's briefly focus on the basics: on what happens under the hood when we want to create or destroy a variable. And, while common, this task is anything but trivial. -This simplifies our task immensely! So let's focus on these variables first and deal with those that must persist for a longer time later. +Remember, we want to find a perfect place in memory for every variable we want to use. These variables can have vastly different sizes and lifetime durations which influences their optimal placement in memory. Furthermore, we usually want to avoid "fragmentation" of our memory, i.e., we don't want to have many small "holes" between allocated variables. If we have those, it might be that we have a lot of free memory in the absolute sense, but we are still unable to find a continuous chunk of memory to allocate some larger variable. This means that ideally we want to find for each variable being allocated the smallest free memory slot it fits into. + +Solving this problem in general is very hard! Think about it, we can't scan **all** the memory at our disposal exhaustively every time we want to allocate a variable, that would take too long! But we _can_ find effective algorithms for finding *not perfect*, but _good-enough_ spots, granted that we have some more information about our data. ### The stack -Because the number of our variables within any scope is relatively small and the variables are themselves also small in size we argue that for any program all of these scoped variables would fit in a relatively small continuous chunk of memory. +For example, we might notice that a lot of scope-local variables we use are really small variables like `int`s or `float`s. They are allocated and freed within their scopes very often which means that, ideally, they should be allocated and freed very quickly. Considering that these variables are mostly small in size and that our scopes are mostly quite short (at least they should be), it is a feasible assumption that we should be able to fit all of such small local variables into a relatively small continuous chunk of memory. -So we designate a small part of our memory, typically 8MB on Linux and MacOS, to managing such local variables. Furthermore, because the variables get de-allocated at the end of the scope, we use a stack-like data structure for managing where to allocate them. +So we designate a small part of our memory, typically around 8MB on Linux and MacOS, to managing such local variables. Furthermore, because these variables are always allocated sequentially and get de-allocated in the reverse order of their allocation at the end of the scope, we use a stack-like data structure for managing where to place them in memory. Garbage collection @@ -105,12 +106,28 @@ int main() { ``` This way of dealing with allocating and freeing variables is amazing for local data. As long as the data does not need to persist beyond the end of the scope, this data structure is very efficient! We always know exactly where to allocate new memory and which memory to free **at constant time**, no additional runtime operations needed! - - ### Why not keep persistent data on the stack -Many things change the moment we want our data to persist beyond the end of the scope. Pause for a moment and think if we can keep such data on the stack too! ⏱️ +Many things change the moment we want to allocate bigger chunk of data at once or for our data to persist beyond the end of the scope. Pause for a moment and think if we can keep such data on the stack too! ⏱️ + +In the case of bigger chunk of data the answer is obvious: once the data stops fitting into the stack, we can't allocate it. If we try to allocate progressively more data in a loop, the program will terminate with `SIGSEGV`, which means that we tried accessing memory that is not allowed for us to access when trying to allocate beyond 8MB of data: +```cpp +#include + +int main() { + const int megabyte{1'000'000}; + int i{}; + while(true) { + // 😱 Don't use C-style arrays in real code! + std::byte numbers[i++ * megabyte]; + std::cerr << "Allocating " << i << " * MB\n"; + } + return 0; +} +``` + +Now, what about persistent data? -And if we think long enough about it we will come up with a couple of issues! Let's for a moment assume that we could keep a variable in our stack for a longer time. And let's also assume that we allocated it in the middle of some scope, with some normal stack variables allocated before and after it. By the end of the scope we must free all memory of those variables. Which essentially means that we would need to pop all the variables above our persistent variable, then copy our variable somewhere, pop the rest of the normal variables and copy our persistent variable back. Not only this is not elegant but we also had to copy a potentially large chunk of memory around, which is slow. +Let's for a moment assume that we could keep a variable in our stack for a longer time. And let's also assume that we allocated it in the middle of some scope, with some normal stack variables allocated before and after it. By the end of the scope we must free all memory of those variables. Which essentially means that we would need to pop all the variables above our persistent variable, then copy our variable somewhere, pop the rest of the normal variables and copy our persistent variable back. Not only this is not elegant but we also had to copy a potentially large chunk of memory around, which is slow. The situation is similarly bad if we would want to free the memory of our persistent variable at some manually chosen point. This would mean that we would need to copy everything that we put on top of it in the stack, remove the persistent data we want, then copy everything back on top of the stack. Also slow! @@ -131,7 +148,9 @@ Anyway, the easiest way to think about the heap is an intuitive one - imagine a Garbage collection -So when we need to create a new variable on the heap, we look through our heap to find a coin that represents space big enough for our variable and store it there. This obviously takes some time at runtime but once such a place is found the variable can stay in that memory until we don't need it anymore. So, allocating on the heap is definitely less efficient than allocating on the stack but it has a benefit of being able to allocate or de-allocate these data at any time with less consideration on how packed the rest of the memory is. +So when we need to create a new variable on the heap, we look through our heap to find a coin that represents space big enough for our variable and store our variable in that found slot. In our analogy this effectively removes the found coin from the heap maybe adding a smaller coin in its place. + +This obviously takes some time at runtime but once such a place is found the variable can stay in that memory until we don't need it anymore. So, allocating on the heap is definitely less efficient than allocating on the stack but it has a benefit of being able to allocate or de-allocate these data at any time with less consideration on how packed the rest of the memory is. Please note that this is a **very inaccurate analogy** as there is much more stuff happening under the hood and the actual implementations of the heap allocators are well-optimized and, as a result, quite complex, but thinking about a heap of coins gives a decent enough intuition. And that is all I'm aiming for here. @@ -140,7 +159,7 @@ We allocate memory on the heap manually in C++. For that we use `new` and `new[] ```cpp int main() { int* ptr_1 = new int{42}; // Allocate single variable. - int* ptr_2 = new int[23]; // Allocate array. + int* ptr_2 = new int[3]{1, 2, 3}; // Allocate array. delete ptr_1; delete[] ptr_2; return 0; @@ -176,7 +195,7 @@ int main() { ``` We start by pushing the `size` and the `ptr` to the stack and enter the inner scope just as before. What is not as before is that we now use the `new[]` operator to allocate our array on the heap. We only allocate memory for it, without initializing the stored values, so initially it stores garbage data. Note that the **pointer to these data**, that we here still call `array`, is **still stored on the stack**! Now we update the values stored in our array and set the `ptr` to point to the same address as the `array` which allows us to print the values using the `ptr` variable. Different from the example before, when we leave the scope, only the `array` variable is cleaned-up from the stack, but the `ptr` variable still points to our data, which still happily lives on the heap. So we can still print all the values we stored in our array using the `ptr` pointer without any undefined behavior. Finally, we can explicitly free the memory using the `delete[]` operator on our `ptr` variable and at the end of the program the stack empties itself. -### Typical pitfalls with data allocated on the heap +## Typical pitfalls with data allocated on the heap There is a number of common pitfalls when using heap-allocated data. And you could probably have already guessed this from the amount of screaming smileys in my code snippets. If you followed my lectures for a while, you know that I am not a fan of taking care of things manually, I don't really trust myself on that :wink:. If we use `new` and `delete` operators manually like in the example before, we have to be very careful with them! #### Forgetting to call `delete` @@ -197,13 +216,16 @@ int main() { return 0; } ``` -But this **does not copy the data**, this only copies the pointer! So instead we now have two pointers that point to the same data! If we `delete` both of them we will get a runtime error `double free or corruption` that hints that we tried to free the memory twice. +But this **does not copy the data**, this only copies the pointer! So instead we now have two pointers that point to the same data! If we `delete` the memory under both of these pointers we will free the memory twice, which is not allowed. Let's say we remove `ptr_2` first. Now `ptr_1` points to memory that was already released. If we try to `delete` it now we will get a runtime error that will tell us that we are trying to free the memory twice (thus, double free): +``` +*** Error: double free or corruption (fasttop): 0x00000000010a3010 *** +```
-#### Performing shallow assignment by mistake +### Performing shallow assignment by mistake If we initially allocate two objects the situation becomes even worse! ```cpp int main() { @@ -221,7 +243,7 @@ Not only we have the same error as before by freeing the memory twice but we als