U e@sdZddlZejdeddddlZddlmZz4ddlmZm Z m Z m Z m Z m Z mZddlmZWn<ek rdd lmZmZm Z m Z m Z m Z m Z mZYnXdd lmZmZefd d Zd dZeefddZdddZeefddZdddZdS)z*serialization utilities for apply messagesNzHipykernel.serialize is deprecated. It has moved to ipyparallel.serialize) stacklevel)chain) CannedObjectcan can_sequenceistypesequence_typesuncanuncan_sequence)PICKLE_PROTOCOL)r rrrrr r r ) MAX_BYTES MAX_ITEMScCsfg}t|trb|jrbt|jD]B\}}t||krHd|j|<||qt|tr||j|<q|S)z/extract buffers larger than a certain thresholdN) isinstancerbuffers enumeratelenappend memoryviewtobytes)objZ thresholdribufr2lib/python3.8/site-packages/ipykernel/serialize.py_extract_buffers1s    rcCs@t|tr<|jrt||kr>t|}|D]}|t||q&nft|trt||kri}t|D](}t||}|t|||||<q`nt|}|t||| dt |t |S)a Serialize an object into a list of sendable buffers. Parameters ---------- obj : object The object to be serialized buffer_threshold : int The threshold (in bytes) for pulling out data buffers to avoid pickling them. item_threshold : int The maximum number of items over which canning will iterate. Containers (lists, dicts) larger than this will be pickled without introspection. Returns ------- [bufs] : list of buffers representing the serialized object. r) rr rrextendrdictsortedrinsertpickledumpsr )rbuffer_thresholditem_thresholdrZcobjckrrrserialize_objectIs   r(cCst|}|d}t|}t|trRt|tkrR|D]}t||q6t ||}n^t|t rt|tkri}t |D]$}||}t||t ||||<qtnt||t ||}||fS)a"reconstruct an object serialized by serialize_object from data buffers. Parameters ---------- buffers : list of buffers/bytes g : globals to be used when uncanning Returns ------- (newobj, bufs) : unpacked object, and the list of remaining unused buffers. r) listrr"loadsrr rrrr rr r )rgbufsZpobjZcannedr&Znewobjr'rrrdeserialize_objectos         r-c sttfdd|D}t}ttfdd|D}tt|t||d}tt |t g} | t|t | || || S)apack up a function, args, and kwargs to be sent over the wire Each element of args/kwargs will be canned for special treatment, but inspection will not go any deeper than that. Any object whose data is larger than `threshold` will not have their data copied (only numpy arrays and bytes/buffers support zero-copy) Message will be a list of bytes/buffers of the format: [ cf, pinfo, , ] With length at least two + len(args) + len(kwargs) c3s|]}t|VqdSNr().0arg)r$r%rr sz%pack_apply_message..c3s|]}t|VqdSr.r/)r0keyr$r%kwargsrrr2s)nargs narg_bufskw_keys) r)r from_iterabler keysrrr"r#rr rr) fargsr5r$r%arg_bufsr8 kwarg_bufsinfomsgrr4rpack_apply_messages   rATcCst|}t|dkstd|d}tt||}|d}t|}|d|d||dd}}g} t|dD]} t||\} }| | q|t | } |rtdi} |dD]}t||\}}|| |<q|rtd || | fS) zdunpack f,args,kwargs from buffers packed by pack_apply_message() Returns: original f,args,kwargsrznot enough buffers!rNr7r6z#Shouldn't be any arg bufs left overr8z%Shouldn't be any kwarg bufs left over) r)rAssertionErrorrr r"r*ranger-rtuple)r,r+copyZpfr;Zpinfor?r=r>Z args_list_r1r<r5r3Zkwargrrrunpack_apply_messages&   "     rG)N)NT)__doc__warningswarnDeprecationWarningr" itertoolsrZipyparallel.serialize.canningrrrrr r r Zipyparallel.serialize.serializer ImportErrorZipykernel.pickleutilZjupyter_client.sessionr rrrr(r-rArGrrrrs( $ .  & %