Lecture 19

1. Memory Allocation

1.0 - Memory Allocation

1.1 - Dynamic Memory Allocation

• Dynamic allocation of objects takes place on the heap, an area of memory separate from the stack
  • There may be multiple references to a single object.
  • Memory may be allocated: new (Java, C++), malloc(C)
    • Sequentially until the free memory runs out
    • From a free list (using techniques like best fit or free fit)
  • All references to an object may be removed, leaving the object using space but not able to be accessed - it is garbage
• In Java, memory allocation is typed, but in C it is untyped, which can lead to type confusion, where a pointer of one type actually points to an object of a different type.

1.2 - Stack and Heap Example

• Consider the following stack and heap, where:
  • Blue space on the heap is empty space
  • We have the variables X, Y and Z allocated on the heap
  • We have the variables p and q on the stack with p → X and q → Y
  • Additionally a field of X points to Y, and a field of Y points to Z

1.3 - Memory Deallocation

• There are two main forms of memory deallocation used in programming languages:
  1. Explicit deallocation, in which there is a function to explicitly deallocate an object: free in C, and delete in C++
  2. Garbage collection in which the space used by unreachable objects is claimed by a garbage collector
• Deallocation of some objects, but not others can lead to memory fragmentation.

1.3.1 - Explicit Memory Deallocation

With explicit memory de-allocation, the

• Freed memory is returned to the “freed list”
• If the memory freed is adjacent to other blocks of memory in the free list, the adjacent free blocks are merged into a single larger freed block to reduce fragmentation
• The freed memory is then available for re-allocation

We may encounter some issues with explicit memory deallocation

• Dangling References An object can be freed via one reference to it, but still be accessible via other reference, also known as use after free
• Memory Leaks All references to an object are removed but the object was not freed
• Memory Fragmentation The memory consists of alternating allocated and free areas, with no large area of free memory
• Locality of Reference Fragmentation leads to the allocated objects being more spread out in the memory address space

Dangling References

• Consider the following code, with the dangling reference $p$

  p := q;
  ...
  free(q);
  q := null;
  
  

• The pointer variable $p$ now references deallocated memory

• Consequences during development or deployment include:

  • The deallocated memory (that is still referenced by $p$) is returned to the free pool and may be reallocated for some new object $r$ that may be a different type to $p$ - any reference through $p$ will then get a corrupted object, and if the object is updated via $p$, the new object $r$ will be corrupted
  • The program may pass the unit/integration/system tests, but then fail in operation because there is more chance of the dangling reference memory being re-allocated
  • Can lead to bugs that are diabolical to find. Static analysis tools can help.

Memory Leak

p := malloc(size);
...
p := null;

• The allocated memory is inaccessible, and cannot be used or de-allocated
• Consequences during development or deployment include:
  • If the memory leak occurs repeatedly, a long running program will eventually run out of memory
  • The program may pass the unit/integration/system tests, but then fail in operation because it runs for longer than when under test
  • Memory leaks can be difficult to debug. Static analysis tools can help.

Memory Fragmentation

• Example scenario of memory fragmentation
  • If a system alternatively allocates objects of type A and type B and then deallocates all the objects of type $B$ this leads to memory fragmentation.
• Consequences:
  • In fragmented memory there can be a large amount of free space but no single block of free space is large enough to allocate a large object
  • The program may pass the unit/integration/system tests but then fail in operation because it takes some time for the memory to become fragmented
  • Memory compaction is not possible in languages that make use of fixed addresses for objects, e.g. C and C++

Locality of Reference

Fragmentation leads to objects being scattered across the address space of the process, with free space in between

• Objects that are accessed together (e.g. objects forming a list) may be scattered across many pages of memory
• On a virtual memory (VM) system, this requires more pages to be in real memory (higher memory footprint)
• Ideally one would like objects referenced together to be allocated to the same or small set of VM pages
• Related objects allocated together can also lead to fewer cache misses (i.e. better performance)

1.4 - Resolving Issues using a Garbage Collector

• Any object that is still accessible is not deallocated, and hence dangling references can not occur

• The garbage collection process finds all inaccessible objects and returns them to the free pool (therefore, can’t end up with memory that is no longer being used, but can’t be re-allocated)

• If the used memory is compacted when garbage collection takes place then the available free memory can be made contiguous

• Compacting garbage collectors can improve locality of reference.

• When the system needs to recover the space used by inaccessible (garbage) objects, it carries out a garbage collection process

  • An object is accessible if it can be reached either
    • Directly via a reference in a global or local variable (from the stack) or
    • Indirectly via a reference in an object that is accessible
  • Garbage collection determines which objects are not accessible, and recovers the space used by them.

• There are numerous collection schemes. We look at three:

  • Mark-and-sweep
  • Stop-and-copy
  • Generational Schemes

1.4.1 - Mark and Sweep

• Mark and sweep garbage collection consists of a phase that:
  • Marks all the accessible objects
  • Sweeps up all objects left unmarked and adds them to the free list.
• In the mark phase, we need to determine which objects are accessible / which ones aren’t
  1. We start by going through the stack, and looking to what objects on the heap they point to
  2. From each of these objects on the heap, we need to find what other objects on the heap we can access from it.

1. We can go from p → X so we can mark “X” as accessible.
2. Additionally, we can go from X → Y, so we can mark “Y” as accessible too.
3. We can go from q → X which is already marked as accessible.
4. We can also go from r → Y which is already marked as accessible.

• We can then sweep up the items on the heap that haven’t been marked as accessible
• We assume that in the stack at the start of a section of free space or variable space, we have a header indicating how big that section is.
• We jump over the free space before and after X and Y - they are either free space or being used.
• As we jump over these spaces, we unmark them ready for the next cycle.
• Z is allocated, but it hasn’t been marked. We merge it with the surrounding free phase.

1.4.2 - Stop and Copy (or two-space) Garbage Collection

• Stop and copy (or two-space) garbage collection

  • Divides the available memory into two large spaces
  • Memory is allocated from one space sequentially until it runs out
  • Garbage collection consists of relocating all accessible objects from the first space to the second space
  • Because the objects are allocated sequentially in the second space, they are compacted
  • When the copy is completed, the roles of the spaces are swapped for the next garbage collection operation

• This overcomes memory fragmentation problems, but copying can be expensive in time

• We start by looking at the stack and identifying what objects are accessible.

  • Once we find an object that is accessible, we copy the variable on the old heap onto the new heap
  • We update the pointer (of the variable, on the stack) to the freshly copied variable.
  • We update the position of the variable on the old heap to be a pointer to the stored location on the new heap

1. We being by going through the stack - What can we access from the pointer p on the stack?

   

   - We find the pointer `p` pointing to the variable $X$ stored on the heap. - Since $X$ is accessible, we copy it over to the new heap. - We update the reference of the pointer `p` to the position that it is stored on the new heap. - We also update the space in which $X$ was stored in the old heap, with a pointer to where $X$ is stored on the new heap

   

2. We find what other variables stored on the old heap are accessible by the current variable that we’re inspecting - What can we access from the variable $X$ on the heap?

   • We see that $X$ holds a pointer to $Y$, so we must also copy that over to the new heap.
   • In updating this pointer, we need to:
     • Update the pointer in $X$ to point to the new location of $Y$ on the new heap
     • Update the variable storage for $Y$ in the old heap to be a pointer to the new $Y$

   

3. We continue going through the stack - What can we access from the pointer q on the stack?

   • We observe that we have already marked $X$ on the old heap (where q is pointing to) as being copied over.
   • We don’t follow the process for $X$any further (i.e. checking what else is accessible) as we have already done this (as is indicated by $X$ being marked as done).
   • We just update the pointer on the stack.

   

4. We continue to go through the stack - What can we access from the pointer r on the stack?

   • As before, we observe that $Y$ has already been marked on the old heap (where r is pointing to) as being copied over, so we don’t follow the copying-over process
   • We just update the pointer on the stack.

5. We use the “new heap” as the current heap, and the “old heap” as the secondary heap, that we move to when garbage collection is next performed.

   

1.4.3 - Generational Schemes

Optimised two-space scheme

• Generational Schemes use a scheme similar to the two-space scheme, but make use of multiple spaces
  • The spaces are organised based on the length of the its objects have survived
  • The age of an object is the number of times it has been collected
  • The older an object is, the less likely that we’ll need to garbage collect it
  • Older objects are migrated to an old object space, newer objects go in the new object space
  • The objects in the old object space do not need to be copied when the new object space is garbage collected
  • The old object space may have to be garbage collected at some stage

1.4.4 - Issues with Garbage Collection

• Garbage collection has a greater time overhead than explicit deallocation
• Response time can vary because:
  • Garbage collection can happen at any time
  • It can take a significant amount of time
• Real-time response hard to guarantee
  • Pre-allocate all objects necessary (fixed allocation)
  • Use incremental on-the-fly garbage collector
  • Use concurrent garbage collector
• Garbage collection is complex, especially for concurrent garbage collection, and tricky to get correct.

1.4.5 - On-the-Fly Garbage Collection

• Garbage collection process takes place incrementally, interleaved with execution of the program
• Each time an object is allocated, the garbage collector can do a bit of work.
• This avoids the program stopping for a (longish) time interval to garbage collect in one go, which can be a problem when real-time response is required.

1.4.6 - Concurrent Garbage Collection

• The garbage collector runs concurrently with the program on a separate processor
• An interesting challenge to verify correct.