a :gh#ã@sdZddlmZmZmZmZmZmZmZm Z m Z m Z m Z m Z mZmZmZmZmZmZmZmZmZmZmZmZmZmZmZmZmZmZm Z m!Z!m"Z"m#Z#m$Z$m%Z%m&Z&m'Z'm(Z)m*Z*m+Z+m,Z,m-Z-m.Z.m/Z/m0Z0m1Z1dde)fiZ2ddl3Z3e3j4rèe)Z(nddl5m6Z7e7e8e2ƒZ9[7[3dS)u£Utilities for pseudo-random number generation. The :mod:`jax.random` package provides a number of routines for deterministic generation of sequences of pseudorandom numbers. Basic usage ----------- >>> seed = 1701 >>> num_steps = 100 >>> key = jax.random.key(seed) >>> for i in range(num_steps): ... key, subkey = jax.random.split(key) ... params = compiled_update(subkey, params, next(batches)) # doctest: +SKIP PRNG keys --------- Unlike the *stateful* pseudorandom number generators (PRNGs) that users of NumPy and SciPy may be accustomed to, JAX random functions all require an explicit PRNG state to be passed as a first argument. The random state is described by a special array element type that we call a **key**, usually generated by the :py:func:`jax.random.key` function:: >>> from jax import random >>> key = random.key(0) >>> key Array((), dtype=key) overlaying: [0 0] This key can then be used in any of JAX's random number generation routines:: >>> random.uniform(key) Array(0.41845703, dtype=float32) Note that using a key does not modify it, so reusing the same key will lead to the same result:: >>> random.uniform(key) Array(0.41845703, dtype=float32) If you need a new random number, you can use :meth:`jax.random.split` to generate new subkeys:: >>> key, subkey = random.split(key) >>> random.uniform(subkey) Array(0.10536897, dtype=float32) .. note:: Typed key arrays, with element types such as ``key`` above, were introduced in JAX v0.4.16. Before then, keys were conventionally represented in ``uint32`` arrays, whose final dimension represented the key's bit-level representation. Both forms of key array can still be created and used with the :mod:`jax.random` module. New-style typed key arrays are made with :py:func:`jax.random.key`. Legacy ``uint32`` key arrays are made with :py:func:`jax.random.PRNGKey`. To convert between the two, use :py:func:`jax.random.key_data` and :py:func:`jax.random.wrap_key_data`. The legacy key format may be needed when interfacing with systems outside of JAX (e.g. exporting arrays to a serializable format), or when passing keys to JAX-based libraries that assume the legacy format. Otherwise, typed keys are recommended. Caveats of legacy keys relative to typed ones include: * They have an extra trailing dimension. * They have a numeric dtype (``uint32``), allowing for operations that are typically not meant to be carried out over keys, such as integer arithmetic. * They do not carry information about the RNG implementation. When legacy keys are passed to :mod:`jax.random` functions, a global configuration setting determines the RNG implementation (see "Advanced RNG configuration" below). To learn more about this upgrade, and the design of key types, see `JEP 9263 `_. Advanced -------- Design and background ===================== **TLDR**: JAX PRNG = `Threefry counter PRNG `_ + a functional array-oriented `splitting model `_ See `docs/jep/263-prng.md `_ for more details. To summarize, among other requirements, the JAX PRNG aims to: 1. ensure reproducibility, 2. parallelize well, both in terms of vectorization (generating array values) and multi-replica, multi-core computation. In particular it should not use sequencing constraints between random function calls. Advanced RNG configuration ========================== JAX provides several PRNG implementations. A specific one can be selected with the optional `impl` keyword argument to `jax.random.key`. When no `impl` option is passed to the `key` constructor, the implementation is determined by the global `jax_default_prng_impl` configuration flag. - **default**, `"threefry2x32"`: `A counter-based PRNG built around the Threefry hash function `_. - *experimental* A PRNG that thinly wraps the XLA Random Bit Generator (RBG) algorithm. See `TF doc `_. - `"rbg"` uses ThreeFry for splitting, and XLA RBG for data generation. - `"unsafe_rbg"` exists only for demonstration purposes, using RBG both for splitting (using an untested made up algorithm) and generating. The random streams generated by these experimental implementations haven't been subject to any empirical randomness testing (e.g. Big Crush). The random bits generated may change between JAX versions. The possible reasons not use the default RNG are: 1. it may be slow to compile (specifically for Google Cloud TPUs) 2. it's slower to execute on TPUs 3. it doesn't support efficient automatic sharding / partitioning Here is a short summary: .. table:: :widths: auto ================================= ======== ========= === ========== ===== ============ Property Threefry Threefry* rbg unsafe_rbg rbg** unsafe_rbg** ================================= ======== ========= === ========== ===== ============ Fastest on TPU ✅ ✅ ✅ ✅ efficiently shardable (w/ pjit) ✅ ✅ ✅ identical across shardings ✅ ✅ ✅ ✅ identical across CPU/GPU/TPU ✅ ✅ identical across JAX/XLA versions ✅ ✅ ================================= ======== ========= === ========== ===== ============ (*): with ``jax_threefry_partitionable=1`` set (**): with ``XLA_FLAGS=--xla_tpu_spmd_rng_bit_generator_unsafe=1`` set The difference between "rbg" and "unsafe_rbg" is that while "rbg" uses a less robust/studied hash function for random value generation (but not for `jax.random.split` or `jax.random.fold_in`), "unsafe_rbg" additionally uses less robust hash functions for `jax.random.split` and `jax.random.fold_in`. Therefore less safe in the sense that the quality of random streams it generates from different keys is less well understood. For more about `jax_threefry_partitionable`, see https://jax.readthedocs.io/en/latest/notebooks/Distributed_arrays_and_automatic_parallelization.html#generating-random-numbers é)/ÚPRNGKeyÚballÚ bernoulliÚbinomialÚbetaÚbitsÚ categoricalÚcauchyÚ chisquareÚchoiceÚcloneÚ dirichletÚdouble_sided_maxwellÚ exponentialÚfÚfold_inÚgammaÚgeneralized_normalÚ geometricÚgumbelÚkeyÚkey_dataÚkey_implÚlaplaceÚlogisticÚloggammaÚ lognormalÚmaxwellÚmultivariate_normalÚnormalÚ orthogonalÚparetoÚ permutationÚpoissonÚ rademacherÚrandintÚrandom_gamma_pÚrayleighÚshuffleÚsplitÚtÚ triangularÚtruncated_normalÚuniformÚwaldÚ weibull_minÚ wrap_key_datar(zSjax.random.shuffle is deprecated. Use jax.random.permutation with independent=True.N)Údeprecation_getattr):Ú__doc__Zjax._src.randomrrrrrrrr r r r r rrrrrrrrrrrrrrrrrrr r!r"r#r$r%r&r'r(Z_deprecated_shuffler)r*r+r,r-r.r/r0Z _deprecationsÚtypingÚ TYPE_CHECKINGZjax._src.deprecationsr1Z_deprecation_getattrÚ__name__Ú __getattr__©r7r7úZ/mounts/lovelace/software/anaconda3/envs/metaDMG/lib/python3.9/site-packages/jax/random.pyÚs#Ä4þþ