C++ 自定义Allocator与多态unique_ptr搭配的踩坑记,附一个完整版的custom allocator.

21 Mar 2024 at 20:09:03

这个礼拜花了两天时间在踩工作中的一个坑,现记录在下,也许将来还会有人遇到类似的情况,可以参考,同时本文也可以加深一下对unique_ptr的认识。

具体来说,坑发生在std::unique_ptr的deleter。在多态的情况下进行unique_ptr的dynamic_cast的时候,通过custom allocator,尤其是自定义了deleter的custom allocator分配内存的unique_ptr会发生deleter无法自动cast的情况。

事故的起因是我工作里的一个活,需要写一个custom allocator来限制一些数据结构的内存分配,使其整个内存使用不超过100MB。这些数据结构有标准的std::vector和std::map。结果在我快要交活的时候,一个同事为了快,把其中一个std::vector里换成了std::unique_ptr,并且,T是一个baseClass,实际存的时候他存了一堆T的derivedClass对象的unique_ptr。于是苦了我。

本来我有一个很简单的custom allocator:

#include <iostream>
#include <memory>
#include <limits>
#include <vector>
#include <functional>

// Custom allocator that tracks memory usage and enforces a limit.
template <typename T>
class LimitedAllocator 
{
public:
  using value_type = T;
  using pointer = T*;
  using const_pointer = const T*;
  using void_pointer = void*;
  using const_void_pointer = const void*;
  using size_type = size_t;
  using difference_type = ptrdiff_t;
 
  static const size_t limit = 100 * 1024 * 1024; // memory limit of 100MB
  static size_t used_memory;

  LimitedAllocator() noexcept = default;

  template <typename U>
  LimitedAllocator(const LimitedAllocator<U>&) noexcept {}

  pointer allocate(size_type n, const_void_pointer hint = 0) 
  {
    size_type bytes = n * sizeof(T);
    if (bytes + used_memory > limit)
    {
      throw std::bad_alloc();
    }
    used_memory += bytes;
    return static_cast<pointer>(::operator new(n * sizeof(T)));
  }

  void deallocate(pointer p, size_type n) noexcept
  {
    used_memory -= n * sizeof(T);
    ::operator delete(p);
  }

  size_type max_size() const noexcept
  {
    return std::numeric_limits<size_type>::max() / sizeof(T);
  }

  template <typename U, typename... Args>
  void construct(U* p, Args&&... args) {
    new(p) U(std::forward<Args>(args)...);
  }

  template <typename U>
  void destroy(U* p) {
    p->~U();
  }

  bool operator==(const LimitedAllocator&) const noexcept {
    return true;
  }

  bool operator!=(const LimitedAllocator&) const noexcept {
    return false;
  }
};


template <typename T>
typename LimitedAllocator<T>::size_type LimitedAllocator<T>::used_memory = 0;

template<typename T>
using LimitedUniquePtr = std::unique_ptr<T, PolymorphicDeleter>;

template<typename T, typename... Args>
std::unique_ptr<T, PolymorphicDeleter> make_unique_limited(LimitedAllocator<T>& allocator, Args&&... args) {
    T* raw_ptr = allocator.allocate(1); // Allocate space for one T
    new (raw_ptr) T(std::forward<Args>(args)...); // Use placement new with forwarded arguments

    return std::unique_ptr<T, PolymorphicDeleter>(raw_ptr, PolymorphicDeleter{});
}

template<typename T>
using LimitedUniquePtr = std::unique_ptr<T, std::function<void(T*)>>;

template<typename T, typename... Args>
LimitedUniquePtr<T> make_unique_limited(LimitedAllocator<T>& allocator, Args&&... args) {
    T* raw_ptr = allocator.allocate(1); // Allocate space for one T
    try {
        new (raw_ptr) T(std::forward<Args>(args)...); // Construct T in allocated space
    } catch (...) {
        allocator.deallocate(raw_ptr, 1); // If construction fails, deallocate and rethrow
        throw;
    }

    // Use std::shared_ptr to share the allocator with the deleter
    auto alloc_shared = std::make_shared<LimitedAllocator<T>>(allocator);

    // Capture the shared_ptr in the deleter lambda
    auto deleter = [alloc_shared](T* ptr) {
        ptr->~T(); // Explicitly call the destructor for the object
        alloc_shared->deallocate(ptr, 1); // Deallocate space for one T
    };

    return LimitedUniquePtr<T>(raw_ptr, deleter);
}

但是,我抽象一下,现在相当于,我有这样的两个class:

class MyClassA {
public:
    virtual ~MyClassA() = default; // Ensure we have a virtual destructor for base class
    virtual void doSomething() const {
        std::cout << "MyClassA doing something." << std::endl;
    }
};

class MyClassB : public MyClassA {
public:
    void doSomething() const override {
        std::cout << "MyClassB doing something different." << std::endl;
    }
};

然后在用的时候,我有一个:

std::vector<LimitedUniquePtr<MyClassA>> m_vector;

而这个m_vector实际在emplace_back的时候呢,推进去的全是
LimitedUniquePtr<MyClassB>!!

我一开始非常天真,觉得我:

auto MyClassAUniquePtr(MyClassBUniquePtr.release());

把ClassBUniquePtr的ownership给释放,然后拷贝构造一个MyClassAUniquePtr呗,反正是derivedClass -> baseClass,按C++的面向对象能力,这点推理没问题的~

但是!!但是!!运行下来确实打出来了
MyClassB doing something different.

然后跟着一个std::bad_function_call 的error,我遂gdb进入,看看是个什么bad_function。然后发现是make_unique_limited里面的deleter,捕获了allocator,但是实际在程序运行结束的时候,allocator已经出了作用域,它没了!

于是我弄了个PolymorphicDeleter,显式override std::unique_ptr的deleter,用不依赖于allocator的函数指针传进去,解决战斗。

于是完整的代码如下:

// 之前的定义就不管了

// Polymorphic deleter
struct PolymorphicDeleter {
    template<typename T>
    void operator()(T* ptr) const {
        delete ptr; // Correctly calls the destructor for T, handling polymorphism
    }
};

template<typename T>
using LimitedUniquePtr = std::unique_ptr<T, PolymorphicDeleter>;

template<typename T, typename... Args>
std::unique_ptr<T, PolymorphicDeleter> make_unique_limited(LimitedAllocator<T>& allocator, Args&&... args) {
    T* raw_ptr = allocator.allocate(1); // Allocate space for one T
    new (raw_ptr) T(std::forward<Args>(args)...); // Use placement new with forwarded arguments

    return std::unique_ptr<T, PolymorphicDeleter>(raw_ptr, PolymorphicDeleter{});
}

解决问题,这下再也没出现过std::bad_function_call。
总结一下,如果你用的是静态的单例的allocator,那没太大问题,allocator一直在内存里,不用考虑deleter里面捕获失败的问题,多态也没太大问题。但是如果你allocator有生命周期的话,一定要考虑到deleter函数指针和allocator生命周期的问题。