a ,gh@sddlZddlZddlZddlZddlZddlZddlZddlZddl m Z m Z m Z m Z mZmZmZmZejejdddZdaddZddZejd d d ZdS) N)ObjectIDObjectNotAvailable PlasmaBuffer PlasmaClientconnectPlasmaObjectExistsPlasmaObjectNotFoundPlasmaStoreFull tensorflow plasma_op.socCsddl}|tadS)Nr)r load_op_libraryTF_PLASMA_OP_PATH tf_plasma_op)tfr^/mounts/lovelace/software/anaconda3/envs/metaDMG/lib/python3.9/site-packages/pyarrow/plasma.pyload_plasma_tensorflow_op.src Cs&zddl}td|jWnty,Yn0tdtjtjt}tj |dd}tj |dd}|j }t j dkrdd g|}d d d d |d|ddtg }|ddtD7}|gd7}||7}||j 7}|dg7}|jr|dg7}tdt|t||tadS)NrzTensorFlow version: z!Compiling Plasma TensorFlow Op...r z plasma_op.ccr darwinz -undefinedZdynamic_lookupzg++z -std=c++11z-gz-sharedz-oz-DNDEBUGz-IcSsg|] }d|qS)z-Lr).0dirrrr Ez.build_plasma_tensorflow_op..)z-lplasmaz-larrow_pythonz-larrowz-fPICz-O2z -DGOOGLE_CUDAzRunning command )r print __version__ ImportErrorospathdirnamerealpath__file__join sysconfigZget_compile_flagssysplatformpaZ get_includeZget_library_dirsZget_link_flagstestZis_built_with_cudastr subprocess check_callr r r)rZdir_pathZcc_pathZso_pathZ tf_cflagscmdrrrbuild_plasma_tensorflow_op4s2          r*Fc cs|r|rtdtjdd}zNtj|d}tjtjdd}tj|s`tjt j dd}|d|d t |g} |r| d |g7} |r| d g7} |d ur| d |g7} d } d } |rgd| } t j | | | d} tdnH|rddg| } t j | | | d} tdnt j | | | d} td| } | d ur@td| f|| fVW| d urb| t|n"| d ur| t|0d S)ab Start a plasma store process. Parameters ---------- plasma_store_memory : int Capacity of the plasma store in bytes. use_valgrind : bool True if the plasma store should be started inside of valgrind. If this is True, use_profiler must be False. use_profiler : bool True if the plasma store should be started inside a profiler. If this is True, use_valgrind must be False. plasma_directory : str Directory where plasma memory mapped files will be stored. use_hugepages : bool True if the plasma store should use huge pages. external_store : str External store to use for evicted objects. Returns ------- result : (str, subprocess.Popen) A tuple of the name of the plasma store socket and the process ID of the plasma store process. z2Cannot use valgrind and profiler at the same time.z test_plasma-)prefixz plasma.sockrzplasma-store-serverbinz-sz-mz-dz-hNz-e)valgrindz--track-origins=yesz--leak-check=fullz--show-leak-kinds=allz!--leak-check-heuristics=stdstringz--error-exitcode=1)stdoutstderrg?r-z--tool=callgrindg?z-plasma_store exited unexpectedly with code %d) Exceptiontempfilemkdtemprrr r$__path__existsr"r+r&r'Popentimesleeppoll RuntimeErrorkillshutilrmtree)Zplasma_store_memoryZ use_valgrindZ use_profilerZplasma_directoryZ use_hugepagesZexternal_storeZtmpdirZplasma_store_nameZplasma_store_executablecommandZ stdout_fileZ stderr_fileprocrcrrrstart_plasma_storeQsl             r@)FFNFN) contextlibrZpyarrowr$r;r'r"r1r6Zpyarrow._plasmarrrrrrrr rr r3r rrr*contextmanagerr@rrrrs"(