将带有 persistent_id 的数据腌制到二进制对象（转储和加载）

Question

我问的第一个问题是 . A next problem I'm facing is that I cannot call dumps or loads 对象到二进制对象。

下面是 ContextAwarePickler 和 ContextAwareUnpickler 的实现。我如何使用这些将对象转换为二进制表示形式或从其二进制表示形式转换回来？据我所知，这只适用于文件。

import pickle

class ContextAwarePickler(pickle.Pickler):
    def persistent_id(self, obj):
        # if this is a context, return the key
        if isinstance(obj, Context):
            return ("Context", context.key)

        # pickle as usual
        return None


class ContextAwareUnpickler(pickle.Unpickler):
    def recover_context(self, key_id):
        ...

    def persistent_load(self, pid):
        type_tag, key_id = pid
        if type_tag == "Context":
            return self.recover_context(key_id)
        else:
            raise pickle.UnpicklingError("unsupported persistent object")

Answer 1

我想我找到了答案：

class ContextawarePickling :

    @staticmethod
    def load_aware (input_file,context=None) :
        return ContextawareUnpickler(input_file).load()

    @staticmethod
    def dump_aware (object_to_save,output_file):
        ContextawarePickler(output_file).dump(object_to_save)

    @staticmethod
    def loads_aware (stream,context=None) :
        file = io.BytesIO(stream)
        return ContextawarePickling.load_aware(file)

    @staticmethod
    def dumps_aware (object_to_save):
        f = io.BytesIO()
        ContextawarePickling.dump_aware(object_to_save,f)
        return f.getvalue()

基本上您首先创建两个实用程序方法：load_aware 和 dump_aware。下一个可以实现 loads_aware 和 dumps_aware，其中一个包装 io.BytesIO 作为处理程序，数据可以是 loaded/stored.

Answer 2

您的解决方案与 dill（我是作者）中的解决方案类似，但不够稳健。

https://github.com/uqfoundation/dill/blob/cccbea9b715e16b742288e1e5a21a687a4d4081b/dill/temp.py#L169（下面复制了代码片段）

def loadIO(buffer, **kwds):
    """load an object that was stored with dill.temp.dumpIO

    buffer: buffer object

    >>> dumpfile = dill.temp.dumpIO([1, 2, 3, 4, 5])
    >>> dill.temp.loadIO(dumpfile)
    [1, 2, 3, 4, 5]
    """
    import dill as pickle
    if PY3:
        from io import BytesIO as StringIO
    else:
        from StringIO import StringIO
    value = getattr(buffer, 'getvalue', buffer) # value or buffer.getvalue
    if value != buffer: value = value() # buffer.getvalue()
    return pickle.load(StringIO(value))

def dumpIO(object, **kwds):
    """dill.dump of object to a buffer.
Loads with "dill.temp.loadIO".  Returns the buffer object.

    >>> dumpfile = dill.temp.dumpIO([1, 2, 3, 4, 5])
    >>> dill.temp.loadIO(dumpfile)
    [1, 2, 3, 4, 5]
    """
    import dill as pickle
    if PY3:
        from io import BytesIO as StringIO
    else:
        from StringIO import StringIO
    file = StringIO()
    pickle.dump(object, file)
    file.flush()
    return file

请注意，您可能需要小心 flush dump 上的缓冲区，就像 dill 一样。

将带有 persistent_id 的数据腌制到二进制对象（转储和加载）

Pickle data with a persistent_id to a binary object (dumps and loads)

python

pickle