+gd2/ddlZddlmZmZddlmZddlmZmZm Z m Z m Z m Z ddl ZddlZddlmZddlmZmZddlmZdd lmZmZerd d lmZd \ZZZeGd dZ dS)N) dataclassfield)BytesIO) TYPE_CHECKINGAnyClassVarDictOptionalUnion)config)xopen xsplitext) array_cast)no_op_if_value_is_nullstring_to_dict) FeatureType)FFFc eZdZUdZdZeeed<dZe ed<dZ e ed<dZ ee ed<dZ ee ed <ejejejd Zeeed <edd d Ze ed<dZdee eefdefdZ ddedeee ee e dffdefdZdedee dfffdZdeejej fdej fdZ!dej dej fdZ"dS)Audioa1Audio [`Feature`] to extract audio data from an audio file. Input: The Audio feature accepts as input: - A `str`: Absolute path to the audio file (i.e. random access is allowed). - A `dict` with the keys: - `path`: String with relative path of the audio file to the archive file. - `bytes`: Bytes content of the audio file. This is useful for archived files with sequential access. - A `dict` with the keys: - `path`: String with relative path of the audio file to the archive file. - `array`: Array containing the audio sample - `sampling_rate`: Integer corresponding to the sampling rate of the audio sample. This is useful for archived files with sequential access. Args: sampling_rate (`int`, *optional*): Target sampling rate. If `None`, the native sampling rate is used. mono (`bool`, defaults to `True`): Whether to convert the audio signal to mono by averaging samples across channels. decode (`bool`, defaults to `True`): Whether to decode the audio data. If `False`, returns the underlying dictionary in the format `{"path": audio_path, "bytes": audio_bytes}`. Example: ```py >>> from datasets import load_dataset, Audio >>> ds = load_dataset("PolyAI/minds14", name="en-US", split="train") >>> ds = ds.cast_column("audio", Audio(sampling_rate=16000)) >>> ds[0]["audio"] {'array': array([ 2.3443763e-05, 2.1729663e-04, 2.2145823e-04, ..., 3.8356509e-05, -7.3497440e-06, -2.1754686e-05], dtype=float32), 'path': '/root/.cache/huggingface/datasets/downloads/extracted/f14948e0e84be638dd7943ac36518a4cf3324e8b7aa331c5ab11541518e9368c/en-US~JOINT_ACCOUNT/602ba55abb1e6d0fbce92065.wav', 'sampling_rate': 16000} ``` N sampling_rateTmonodecodeiddictdtypebytespathpa_typeF)defaultinitrepr_typec|jSN)r )selfs 7lib/python3.11/site-packages/datasets/features/audio.py__call__zAudio.__call__Ls |valuereturnc" ddl}n"#t$r}td|d}~wwxYwt|trd|dSt|tr|ddSd|vrJt }|||d|dd|ddS|d htj |d rB|d  d r|dtd |d rGtj|d tj tjdz }n=tj|d ddtjdz }t t }||||dd|ddSd|d dS|d |d +|d |d dSt)d|d)zEncode example into a format for Arrow. Args: value (`str` or `dict`): Data passed as input to Audio feature. Returns: `dict` rNz;To support encoding audio data, please install 'soundfile'.rarrayrwav)formatrpcmzBTo use PCM files, please specify a 'sampling_rate' in Audio objectr)rihr)rmodezUAn audio sample should have one of 'path' or 'bytes' but they are missing or None in .) soundfile ImportError isinstancestrrrwritegetvaluegetosrisfileendswithKeyErrornp frombufferint16astypefloat32memmap ValueError)r'r+sferrbuffer bytes_values r(encode_examplezAudio.encode_exampleOs f " " " " " f f f[\\be e f eS ! !! !511 1 u % % "D11 1   YYF HHVU7^U?-CEH R R R#__..== = YYv   *rw~~eFm/L/L *V}%%e,, B99_--5"#ghhh99W%%k"$-gbh"O"O"O"V"VWYWa"b"bej"jKK"$)E&M3"O"O"O"V"VWYWa"b"bej"jK ))eO.DUSSS!'!2!2DAAA!%uyy/@/@AAA YYw   +uyy/@/@/L"YYw//69J9JKK Kphmppp s &!&token_per_repo_idc>|jstd|d|dt|dfn |ddf\}}||td|d ddl}ddl}n"#t $r}t d|d}~wwxYw|/t|d d dnd}tj s|d krtd tj s|d krtd ||pi}| dd} t| tjd} || } n#ttf$rd} YnwxYwt!|d| 5} || \} }dddn #1swxYwYn||\} }| j} |jr|| } |jr/|j|kr$|| ||j} |j}|| |dS)aDecode example audio file into audio data. Args: value (`dict`): A dictionary with keys: - `path`: String with relative audio file path. - `bytes`: Bytes of the audio file. token_per_repo_id (`dict`, *optional*): To access and decode audio files from private repositories on the Hub, you can pass a dictionary repo_id (`str`) -> token (`bool` or `str`) Returns: `dict` zMDecoding is disabled for this feature. Please use Audio(decode=True) instead.rNrzJAn audio sample should have one of 'path' or 'bytes' but both are None in r5rzJTo support decoding audio files, please install 'librosa' and 'soundfile'.ropuszDecoding 'opus' files requires system library 'libsndfile'>=1.0.31, You can try to update `soundfile` python library: `pip install "soundfile>=0.12.1"`. mp3zDecoding 'mp3' files requires system library 'libsndfile'>=1.1.0, You can try to update `soundfile` python library: `pip install "soundfile>=0.12.1"`. z::repo_idrb)use_auth_token)orig_sr target_sr)rr.r)r RuntimeErrorrrGlibrosar6r7rlowerr IS_OPUS_SUPPORTEDIS_MP3_SUPPORTEDsplitrHUB_DATASETS_URLr@rreadTrto_monorresample)r'r+rMrfilerXrHrI audio_format source_urlrRrTfr.rs r(decode_examplezAudio.decode_examples&{ pnoo oAFwA[eFmWU7^%<%<==bghnboquav d 9Iyq)!""-33555t ' LF,B,Bh ( \U-B-Bh  < 1 7R D))"-J &(V5LMMiX!27!;) & & &!% &tT.AAA 2Q')wwqzz$} 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2$&774== E= 9 +OOE**E   /$"4 "E"E$$UMTM_$``E .Mu}MMMs<!A** B 4BB #(E E"!E"7FF #F rchddlm}|jrtd|d|ddS)z[If in the decodable state, raise an error, otherwise flatten the feature into a dictionary.r)Valuez'Cannot flatten a decoded Audio feature.binarystringr)featuresrhrrG)r'rhs r(flattenz Audio.flattensS###### ; HFGG GU8__E(OO   r*storagectj|jrrtjdgt |ztj}tj||gddg| }n;tj |jrrtjdgt |ztj }tj||gddg| }ntj |jrL|j dr2tjd|D}n5tj |jr|jddkr|d}n8tjdgt |ztj}|jddkr|d}n8tjdgt |ztj }tj||gddg| }t#||jS) aCast an Arrow array to the Audio arrow storage type. The Arrow types that can be converted to the Audio pyarrow storage type are: - `pa.string()` - it must contain the "path" data - `pa.binary()` - it must contain the audio bytes - `pa.struct({"bytes": pa.binary()})` - `pa.struct({"path": pa.string()})` - `pa.struct({"bytes": pa.binary(), "path": pa.string()})` - order doesn't matter Args: storage (`Union[pa.StringArray, pa.StructArray]`): PyArrow array to cast. Returns: `pa.StructArray`: Array in the Audio arrow storage type, that is `pa.struct({"bytes": pa.binary(), "path": pa.string()})` Ntyperrmaskr.cXg|]'}|!t|nd(Sr&)rrL).0xs r( z&Audio.cast_storage..s5rrrYZQ] 6 6q 9 9 9PTrrrr*r)patypes is_stringrpr.lenri StructArray from_arraysis_null is_binaryrj is_structget_all_field_indices to_pylistget_field_indexrrr )r'rm bytes_array path_arrays r( cast_storagezAudio.cast_storages`$ 8  gl + + w(D6CLL#8ry{{KKKKn00+w1G'SYIZahapaparar0ssGG X   - - w4&3w<<"7bikkJJJJn00':1FRXHY`g`o`o`q`q0rrGG X   - - w',2T2TU\2]2] whrr^e^o^o^q^qrrrssGG X   - - w|++G4499%mmG44  hvG '<29;;OOO |++F33q88$]]622 Xtfs7||&;")++NNN n00+z1JWV\L]dkdsdsdudu0vvG'4<000r*ctdtjfd|Dtj}tjd|dDtj}tj||gddg| }t||j S)a8Embed audio files into the Arrow array. Args: storage (`pa.StructArray`): PyArrow array to embed. Returns: `pa.StructArray`: Array in the Audio arrow storage type, that is `pa.struct({"bytes": pa.binary(), "path": pa.string()})`. c~t|d5}|}dddn #1swxYwY|S)NrS)rr^)rrebytes_s r( path_to_bytesz*Audio.embed_storage..path_to_bytessvtT"" "a " " " " " " " " " " " " " " "Ms 266cZg|]'}|!|d|dn |dnd(S)Nrr)rtrurs r(rvz'Audio.embed_storage..sU   UVTaQwZ-?qy)))QwZZgk   r*rocTg|]%}|tj|nd&Sr&)r=rbasename)rtrs r(rvz'Audio.embed_storage..s2 p p pdt'7RW  d # # #T p p pr*rrrq) rrwr.rrirrjr{r|r}rr )r'rmrrrs @r( embed_storagezAudio.embed_storages      h     **,,       X p pgmm\bNcNcNmNmNoNo p p p   .,,k:-FRXHY`k`s`s`u`u,vv'4<000r*r&)#__name__ __module__ __qualname____doc__rr int__annotations__rboolrrr9rrrwstructrirjr rrr$r)r rrrLr rfrl StringArrayr{rrrr*r(rrs))V$(M8C='''D$FDB !E8C=!!!&RYibikk'R'RSSGXc]SSSwU???E3???/E#ud*:$;/////d]aBNBNBN.6tCsDRVAWrsH ((((((((FFFFFFFFFFFFFFFFBBBBBBBBCCCCCCCC&%%%%%%6I2!2 |1|1|1|1|1|1|1 |1|1|1r*