Skip to content

Chapter 06: Memory

Overview

Efficient memory management is crucial for GPU performance. This chapter covers:

  • Memory heaps and types in depth
  • Allocation strategies
  • Memory aliasing and suballocation
  • When to use different memory types

What you'll learn:

  • Understanding GPU memory architecture
  • Optimizing memory usage patterns
  • Avoiding allocation overhead

Memory Architecture

Heaps

Physical memory pools:

VkPhysicalDeviceMemoryProperties mem_props;
vkGetPhysicalDeviceMemoryProperties(physical_device, &mem_props);

for (uint32_t i = 0; i < mem_props.memoryHeapCount; i++) {
    VkMemoryHeap heap = mem_props.memoryHeaps[i];
    printf("Heap %u: %.2f GB", i, heap.size / 1e9);
    if (heap.flags & VK_MEMORY_HEAP_DEVICE_LOCAL_BIT) {
        printf(" (Device Local)");  // VRAM
    }
    printf("\n");
}

Typical discrete GPU: 

Heap 0: 8.00 GB (Device Local)  ← VRAM
Heap 1: 32.00 GB                 ← System RAM

Integrated GPU (or Apple Silicon): 

Heap 0: 16.00 GB (Device Local)  ← Unified Memory

Memory Types

Each type belongs to a heap and has properties:

for (uint32_t i = 0; i < mem_props.memoryTypeCount; i++) {
    VkMemoryType type = mem_props.memoryTypes[i];
    printf("Type %u (Heap %u): ", i, type.heapIndex);

    if (type.propertyFlags & VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT)
        printf("DEVICE_LOCAL ");
    if (type.propertyFlags & VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT)
        printf("HOST_VISIBLE ");
    if (type.propertyFlags & VK_MEMORY_PROPERTY_HOST_COHERENT_BIT)
        printf("HOST_COHERENT ");
    if (type.propertyFlags & VK_MEMORY_PROPERTY_HOST_CACHED_BIT)
        printf("HOST_CACHED ");
}

Choosing Memory Types

For Compute Buffers (GPU Only)

// Fastest GPU access, CPU can't touch
VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT

For Upload (CPU → GPU)

// CPU writes, GPU reads
VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT | 
VK_MEMORY_PROPERTY_HOST_COHERENT_BIT

For Readback (GPU → CPU)

// GPU writes, CPU reads with cache
VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT | 
VK_MEMORY_PROPERTY_HOST_COHERENT_BIT |
VK_MEMORY_PROPERTY_HOST_CACHED_BIT

Memory Allocation Limits

Vulkan has a limit on the number of allocations (typically ~4096):

VkPhysicalDeviceProperties props;
vkGetPhysicalDeviceProperties(physical_device, &props);
// props.limits.maxMemoryAllocationCount

Don't Allocate Per-Buffer

Creating 10,000 buffers with 10,000 allocations will fail!

Suballocation

Allocate large blocks, suballocate within:

┌────────────────────────────────────────────────────────┐
│                   Large Allocation (64 MB)             │
├──────────────┬──────────────┬──────────────┬───────────┤
│   Buffer A   │   Buffer B   │   Buffer C   │   Free    │
│   (16 MB)    │   (8 MB)     │   (32 MB)    │   (8 MB)  │
└──────────────┴──────────────┴──────────────┴───────────┘

Basic Suballocator

typedef struct {
    VkDeviceMemory memory;
    VkDeviceSize size;
    VkDeviceSize offset;  // Current allocation point
} MemoryBlock;

VkDeviceSize suballocate(MemoryBlock* block, VkDeviceSize size, 
                         VkDeviceSize alignment) {
    // Align offset
    VkDeviceSize aligned_offset = (block->offset + alignment - 1) 
                                  & ~(alignment - 1);

    if (aligned_offset + size > block->size) {
        return UINT64_MAX;  // Out of space
    }

    VkDeviceSize result = aligned_offset;
    block->offset = aligned_offset + size;
    return result;
}

Binding with Offset

VkDeviceSize offset = suballocate(&block, buffer_size, alignment);
vkBindBufferMemory(device, buffer, block.memory, offset);

Memory Aliasing

Multiple buffers can share the same memory (if not used simultaneously):

// Allocate once
VkDeviceSize total_size = max(size_a, max(size_b, size_c));
VkDeviceMemory memory;
vkAllocateMemory(device, &alloc_info, NULL, &memory);

// Bind multiple buffers to same memory
vkBindBufferMemory(device, buffer_a, memory, 0);
vkBindBufferMemory(device, buffer_b, memory, 0);
vkBindBufferMemory(device, buffer_c, memory, 0);

Aliasing Rules

Only one aliased buffer can be "active" at a time. Use barriers to transition between them.

Dedicated Allocations

Some resources benefit from dedicated memory:

VkMemoryDedicatedRequirements dedicated_reqs = {
    .sType = VK_STRUCTURE_TYPE_MEMORY_DEDICATED_REQUIREMENTS
};

VkMemoryRequirements2 mem_reqs2 = {
    .sType = VK_STRUCTURE_TYPE_MEMORY_REQUIREMENTS_2,
    .pNext = &dedicated_reqs
};

VkBufferMemoryRequirementsInfo2 buf_reqs_info = {
    .sType = VK_STRUCTURE_TYPE_BUFFER_MEMORY_REQUIREMENTS_INFO_2,
    .buffer = buffer
};

vkGetBufferMemoryRequirements2(device, &buf_reqs_info, &mem_reqs2);

if (dedicated_reqs.requiresDedicatedAllocation ||
    dedicated_reqs.prefersDedicatedAllocation) {
    // Use dedicated allocation
    VkMemoryDedicatedAllocateInfo dedicated_info = {
        .sType = VK_STRUCTURE_TYPE_MEMORY_DEDICATED_ALLOCATE_INFO,
        .buffer = buffer
    };
    alloc_info.pNext = &dedicated_info;
}

Memory Budget

Query available memory (with extension):

// Requires VK_EXT_memory_budget
VkPhysicalDeviceMemoryBudgetPropertiesEXT budget = {
    .sType = VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_MEMORY_BUDGET_PROPERTIES_EXT
};

VkPhysicalDeviceMemoryProperties2 props2 = {
    .sType = VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_MEMORY_PROPERTIES_2,
    .pNext = &budget
};

vkGetPhysicalDeviceMemoryProperties2(physical_device, &props2);

for (uint32_t i = 0; i < props2.memoryProperties.memoryHeapCount; i++) {
    printf("Heap %u: %llu / %llu MB used\n", i,
           budget.heapUsage[i] / (1024*1024),
           budget.heapBudget[i] / (1024*1024));
}

Running the Example

./build/bin/ch06_memory

Output from an Apple M3 Pro, abridged:

=== Memory Heaps (1) ===
  Heap 0:
    Size: 18.00 GB
    Flags: DEVICE_LOCAL 

=== Memory Types (3) ===
  Type 0 (Heap 0):
    Properties: DEVICE_LOCAL 
    Best for: GPU-only data (fastest compute)
  Type 1 (Heap 0):
    Properties: DEVICE_LOCAL HOST_VISIBLE HOST_COHERENT HOST_CACHED 
    Best for: Staging buffers, uniform buffers
  Type 2 (Heap 0):
    Properties: DEVICE_LOCAL LAZILY_ALLOCATED 
    Best for: GPU-only data (fastest compute)

=== Memory Limits ===
  Max Memory Allocations: 1073741824
  Buffer-Image Granularity: 16 bytes
  Non-Coherent Atom Size: 16 bytes

=== Part 2: Alignment Requirements ===
Storage Buffer (1024 bytes):
  Actual size needed: 1024 bytes
  Alignment required: 256 bytes
  Memory type bits: 0x3

=== Part 3: Sub-allocation ===
Allocating 16.00 MB block for multiple buffers...
  Buffer 0: size=1024, offset=0, aligned to 256
  Buffer 1: size=4096, offset=1024, aligned to 256
  Buffer 2: size=2048, offset=5120, aligned to 256
  Buffer 3: size=8192, offset=7168, aligned to 256
Total used: 15360 / 16777216 bytes

=== Part 4: Compute on Sub-allocated Buffer ===
Buffer 1 after compute (doubled):
  [0] = 2000.0 (was 1000.0)
  [1] = 2002.0 (was 1001.0)
  [2] = 2004.0 (was 1002.0)

=== Part 5: Memory Access Performance ===
Host-visible coherent: 0.09 ms per 4MB upload (41.29 GB/s)

Chapter 06 completed!

Sub-allocation offsets

Buffer 1 lands at offset 1024 even though the alignment is 256 — the offset only has to be a multiple of the alignment, and 1024 already is.

Best Practices

Do

  • ✅ Suballocate from large blocks
  • ✅ Use device-local for compute-heavy buffers
  • ✅ Keep staging buffers mapped
  • ✅ Pool similar allocations together

Don't

  • ❌ Allocate per-buffer
  • ❌ Exceed maxMemoryAllocationCount
  • ❌ Map/unmap repeatedly (keep mapped)
  • ❌ Use host-visible for compute-intensive data

Memory Allocator Libraries

For production, use a library:

  • VMA (Vulkan Memory Allocator): AMD's widely-used allocator
  • D3D12MA: Can be adapted for Vulkan concepts
// VMA example
VmaAllocatorCreateInfo allocator_info = {...};
VmaAllocator allocator;
vmaCreateAllocator(&allocator_info, &allocator);

VmaAllocationCreateInfo alloc_ci = {
    .usage = VMA_MEMORY_USAGE_GPU_ONLY
};
vmaCreateBuffer(allocator, &buffer_info, &alloc_ci, 
                &buffer, &allocation, NULL);

Exercises

  1. Allocation Limit: Create buffers until you hit the allocation limit, then implement suballocation.

  2. Memory Pressure: Allocate more than available VRAM and observe fallback to system memory.

  3. Benchmark: Compare compute performance on host-visible vs device-local memory.

Common Errors

VK_ERROR_OUT_OF_DEVICE_MEMORY

  • Check memory budget before allocating
  • Implement suballocation
  • Consider smaller working sets

Wrong Memory Type

Mapped memory returns garbage: - Ensure HOST_VISIBLE flag - Check HOST_COHERENT or flush manually

Alignment Violations

Buffer binding fails: - Respect memoryRequirements.alignment - Some buffers need stricter alignment (uniform buffers especially)

What's Next?

We've been using descriptors to pass data to shaders. In Chapter 07, we'll learn about push constants — a faster way to pass small amounts of data.