From cb510644b2fd65e4ce216a7799ce7401f71548b8 Mon Sep 17 00:00:00 2001 From: Nathan Cassereau <84033440+ncassereau-idris@users.noreply.github.com> Date: Fri, 19 Nov 2021 11:06:48 +0100 Subject: [MRG] Solve bug of contribution link in docs + automated insert of readme in doc (#301) * first try * bug solve * attempt to use m2r2 * More elegant dependency * readme.rst now generated automatically from readme.md * weird attempt * Revert "weird attempt" This reverts commit b45f0e6495148877ad3e93ad608b3b0cfe9bbb16. * fixing readme links * Revert "fixing readme links" This reverts commit 0c6ca7612ea297674a3afc50f8e6538c58c27701. * full link for readme * correct CONTRIBUTING guidelines mistakes --- .github/CONTRIBUTING.md | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) (limited to '.github') diff --git a/.github/CONTRIBUTING.md b/.github/CONTRIBUTING.md index 54e7e42..9bc8e87 100644 --- a/.github/CONTRIBUTING.md +++ b/.github/CONTRIBUTING.md @@ -182,11 +182,10 @@ reStructuredText documents live in the source code repository under the doc/ directory. You can edit the documentation using any text editor and then generate -the HTML output by typing ``make html`` from the doc/ directory. +the HTML output by typing ``make html`` from the ``docs/`` directory. Alternatively, ``make`` can be used to quickly generate the -documentation without the example gallery. The resulting HTML files will -be placed in ``_build/html/`` and are viewable in a web browser. See the -``README`` file in the ``doc/`` directory for more information. +documentation without the example gallery with `make html-noplot`. The resulting HTML files will +be placed in `docs/build/html/` and are viewable in a web browser. For building the documentation, you will need [sphinx](http://sphinx.pocoo.org/), -- cgit v1.2.3 From f8d871e8c6f15009f559ece6a12eb8d8891c60fb Mon Sep 17 00:00:00 2001 From: Nathan Cassereau <84033440+ncassereau-idris@users.noreply.github.com> Date: Thu, 9 Dec 2021 17:55:12 +0100 Subject: [MRG] Tensorflow backend & Benchmarker & Myst_parser (#316) * First batch of tf methods (to be continued) * Second batch of method (yet to debug) * tensorflow for cpu * add tf requirement * pep8 + bug * small changes * attempt to solve pymanopt bug with tf2 * attempt #2 * attempt #3 * attempt 4 * docstring * correct pep8 violation introduced in merge conflicts resolution * attempt 5 * attempt 6 * just a random try * Revert "just a random try" This reverts commit 8223e768bfe33635549fb66cca2267514a60ebbf. * GPU tests for tensorflow * pep8 * attempt to solve issue with m2r2 * Remove transpose backend method * first draft of benchmarker (need to correct time measurement) * prettier bench table * Bitsize and prettier device methods * prettified table bench * Bug corrected (results were mixed up in the final table) * Better perf counter (for GPU support) * pep8 * EMD bench * solve bug if no GPU available * pep8 * warning about tensorflow numpy api being required in the backend.py docstring * Bug solve in backend docstring * not covering code which requires a GPU * Tensorflow gradients manipulation tested * Number of warmup runs is now customizable * typo * Remove some warnings while building docs * Change prettier_device to device_type in backend * Correct JAX mistakes preventing to see the CPU if a GPU is present * Attempt to solve JAX bug in case no GPU is found * Reworked benchmarks order and results storage & clear GPU after usage by benchmark * Add bench to backend docstring * better benchs * remove useless stuff * Better device_type * Now using MYST_PARSER and solving links issue in the README.md / online docs --- .github/requirements_test_windows.txt | 2 +- README.md | 8 +- benchmarks/__init__.py | 5 + benchmarks/benchmark.py | 105 ++++++ benchmarks/emd.py | 40 +++ benchmarks/sinkhorn_knopp.py | 42 +++ docs/requirements.txt | 2 +- docs/requirements_rtd.txt | 2 +- docs/source/.github/CODE_OF_CONDUCT.rst | 6 + docs/source/.github/CONTRIBUTING.rst | 6 + docs/source/code_of_conduct.rst | 1 - docs/source/conf.py | 2 +- docs/source/contributing.rst | 1 - docs/source/index.rst | 9 +- ot/backend.py | 580 +++++++++++++++++++++++++++++++- ot/bregman.py | 72 ++-- ot/da.py | 44 +-- ot/datasets.py | 2 +- ot/dr.py | 2 + ot/gromov.py | 2 +- ot/lp/solver_1d.py | 4 +- ot/plot.py | 4 +- requirements.txt | 3 +- test/conftest.py | 12 +- test/test_1d_solver.py | 68 +++- test/test_backend.py | 52 ++- test/test_bregman.py | 45 ++- test/test_gromov.py | 44 ++- test/test_ot.py | 36 +- test/test_sliced.py | 57 ++++ 30 files changed, 1161 insertions(+), 97 deletions(-) create mode 100644 benchmarks/__init__.py create mode 100644 benchmarks/benchmark.py create mode 100644 benchmarks/emd.py create mode 100644 benchmarks/sinkhorn_knopp.py create mode 100644 docs/source/.github/CODE_OF_CONDUCT.rst create mode 100644 docs/source/.github/CONTRIBUTING.rst delete mode 100644 docs/source/code_of_conduct.rst delete mode 100644 docs/source/contributing.rst (limited to '.github') diff --git a/.github/requirements_test_windows.txt b/.github/requirements_test_windows.txt index 331dd57..b94392f 100644 --- a/.github/requirements_test_windows.txt +++ b/.github/requirements_test_windows.txt @@ -4,7 +4,7 @@ cython matplotlib autograd pymanopt==0.2.4; python_version <'3' -pymanopt; python_version >= '3' +pymanopt==0.2.6rc1; python_version >= '3' cvxopt scikit-learn pytest \ No newline at end of file diff --git a/README.md b/README.md index 18064a3..17fbe81 100644 --- a/README.md +++ b/README.md @@ -35,7 +35,7 @@ POT provides the following generic OT solvers (links to examples): * [Partial Wasserstein and Gromov-Wasserstein](https://pythonot.github.io/auto_examples/unbalanced-partial/plot_partial_wass_and_gromov.html) (exact [29] and entropic [3] formulations). * [Sliced Wasserstein](https://pythonot.github.io/auto_examples/sliced-wasserstein/plot_variance.html) [31, 32] and Max-sliced Wasserstein [35] that can be used for gradient flows [36]. -* [Several backends](https://pythonot.github.io/quickstart.html#solving-ot-with-multiple-backends) for easy use of POT with [Pytorch](https://pytorch.org/)/[jax](https://github.com/google/jax)/[Numpy](https://numpy.org/) arrays. +* [Several backends](https://pythonot.github.io/quickstart.html#solving-ot-with-multiple-backends) for easy use of POT with [Pytorch](https://pytorch.org/)/[jax](https://github.com/google/jax)/[Numpy](https://numpy.org/)/[Cupy](https://cupy.dev/)/[Tensorflow](https://www.tensorflow.org/) arrays. POT provides the following Machine Learning related solvers: @@ -202,12 +202,12 @@ This toolbox benefit a lot from open source research and we would like to thank * [Gabriel Peyré](http://gpeyre.github.io/) (Wasserstein Barycenters in Matlab) * [Mathieu Blondel](https://mblondel.org/) (original implementation smooth OT) -* [Nicolas Bonneel](http://liris.cnrs.fr/~nbonneel/) ( C++ code for EMD) +* [Nicolas Bonneel](http://liris.cnrs.fr/~nbonneel/) (C++ code for EMD) * [Marco Cuturi](http://marcocuturi.net/) (Sinkhorn Knopp in Matlab/Cuda) ## Contributions and code of conduct -Every contribution is welcome and should respect the [contribution guidelines](https://pythonot.github.io/contributing.html). Each member of the project is expected to follow the [code of conduct](https://pythonot.github.io/code_of_conduct.html). +Every contribution is welcome and should respect the [contribution guidelines](.github/CONTRIBUTING.md). Each member of the project is expected to follow the [code of conduct](.github/CODE_OF_CONDUCT.md). ## Support @@ -217,7 +217,7 @@ You can ask questions and join the development discussion: * On the POT [gitter channel](https://gitter.im/PythonOT/community) * On the POT [mailing list](https://mail.python.org/mm3/mailman3/lists/pot.python.org/) -You can also post bug reports and feature requests in Github issues. Make sure to read our [guidelines](https://pythonot.github.io/contributing.html) first. +You can also post bug reports and feature requests in Github issues. Make sure to read our [guidelines](.github/CONTRIBUTING.md) first. ## References diff --git a/benchmarks/__init__.py b/benchmarks/__init__.py new file mode 100644 index 0000000..37f5e56 --- /dev/null +++ b/benchmarks/__init__.py @@ -0,0 +1,5 @@ +from . import benchmark +from . import sinkhorn_knopp +from . import emd + +__all__= ["benchmark", "sinkhorn_knopp", "emd"] diff --git a/benchmarks/benchmark.py b/benchmarks/benchmark.py new file mode 100644 index 0000000..7973c6b --- /dev/null +++ b/benchmarks/benchmark.py @@ -0,0 +1,105 @@ +# /usr/bin/env python3 +# -*- coding: utf-8 -*- + +from ot.backend import get_backend_list, jax, tf +import gc + + +def setup_backends(): + if jax: + from jax.config import config + config.update("jax_enable_x64", True) + + if tf: + from tensorflow.python.ops.numpy_ops import np_config + np_config.enable_numpy_behavior() + + +def exec_bench(setup, tested_function, param_list, n_runs, warmup_runs): + backend_list = get_backend_list() + for i, nx in enumerate(backend_list): + if nx.__name__ == "tf" and i < len(backend_list) - 1: + # Tensorflow should be the last one to be benchmarked because + # as far as I'm aware, there is no way to force it to release + # GPU memory. Hence, if any other backend is benchmarked after + # Tensorflow and requires the usage of a GPU, it will not have the + # full memory available and you may have a GPU Out Of Memory error + # even though your GPU can technically hold your tensors in memory. + backend_list.pop(i) + backend_list.append(nx) + break + + inputs = [setup(param) for param in param_list] + results = dict() + for nx in backend_list: + for i in range(len(param_list)): + print(nx, param_list[i]) + args = inputs[i] + results_nx = nx._bench( + tested_function, + *args, + n_runs=n_runs, + warmup_runs=warmup_runs + ) + gc.collect() + results_nx_with_param_in_key = dict() + for key in results_nx: + new_key = (param_list[i], *key) + results_nx_with_param_in_key[new_key] = results_nx[key] + results.update(results_nx_with_param_in_key) + return results + + +def convert_to_html_table(results, param_name, main_title=None, comments=None): + string = "\n" + keys = list(results.keys()) + params, names, devices, bitsizes = zip(*keys) + + devices_names = sorted(list(set(zip(devices, names)))) + params = sorted(list(set(params))) + bitsizes = sorted(list(set(bitsizes))) + length = len(devices_names) + 1 + cpus_cols = list(devices).count("CPU") / len(bitsizes) / len(params) + gpus_cols = list(devices).count("GPU") / len(bitsizes) / len(params) + assert cpus_cols + gpus_cols == len(devices_names) + + if main_title is not None: + string += f'\n' + + for i, bitsize in enumerate(bitsizes): + + if i != 0: + string += f'\n' + + # make bitsize header + text = f"{bitsize} bits" + if comments is not None: + text += " - " + if isinstance(comments, (tuple, list)) and len(comments) == len(bitsizes): + text += str(comments[i]) + else: + text += str(comments) + string += f'' + string += f'\n' + + # make device header + string += f'' + string += f'' + string += f'\n' + + # make param_name / backend header + string += f'' + for device, name in devices_names: + string += f'' + string += "\n" + + # make results rows + for param in params: + string += f'' + for device, name in devices_names: + key = (param, name, device, bitsize) + string += f'' + string += "\n" + + string += "
{str(main_title)}
 
Bitsize{text}
DeviceCPUGPU
{param_name}{name}
{param}{results[key]:.4f}
" + return string diff --git a/benchmarks/emd.py b/benchmarks/emd.py new file mode 100644 index 0000000..9f64863 --- /dev/null +++ b/benchmarks/emd.py @@ -0,0 +1,40 @@ +# /usr/bin/env python3 +# -*- coding: utf-8 -*- + +import numpy as np +import ot +from .benchmark import ( + setup_backends, + exec_bench, + convert_to_html_table +) + + +def setup(n_samples): + rng = np.random.RandomState(789465132) + x = rng.randn(n_samples, 2) + y = rng.randn(n_samples, 2) + + a = ot.utils.unif(n_samples) + M = ot.dist(x, y) + return a, M + + +if __name__ == "__main__": + n_runs = 100 + warmup_runs = 10 + param_list = [50, 100, 500, 1000, 2000, 5000] + + setup_backends() + results = exec_bench( + setup=setup, + tested_function=lambda a, M: ot.emd(a, a, M), + param_list=param_list, + n_runs=n_runs, + warmup_runs=warmup_runs + ) + print(convert_to_html_table( + results, + param_name="Sample size", + main_title=f"EMD - Averaged on {n_runs} runs" + )) diff --git a/benchmarks/sinkhorn_knopp.py b/benchmarks/sinkhorn_knopp.py new file mode 100644 index 0000000..3a1ef3f --- /dev/null +++ b/benchmarks/sinkhorn_knopp.py @@ -0,0 +1,42 @@ +# /usr/bin/env python3 +# -*- coding: utf-8 -*- + +import numpy as np +import ot +from .benchmark import ( + setup_backends, + exec_bench, + convert_to_html_table +) + + +def setup(n_samples): + rng = np.random.RandomState(123456789) + a = rng.rand(n_samples // 4, 100) + b = rng.rand(n_samples, 100) + + wa = ot.unif(n_samples // 4) + wb = ot.unif(n_samples) + + M = ot.dist(a.copy(), b.copy()) + return wa, wb, M + + +if __name__ == "__main__": + n_runs = 100 + warmup_runs = 10 + param_list = [50, 100, 500, 1000, 2000, 5000] + + setup_backends() + results = exec_bench( + setup=setup, + tested_function=lambda *args: ot.bregman.sinkhorn(*args, reg=1, stopThr=1e-7), + param_list=param_list, + n_runs=n_runs, + warmup_runs=warmup_runs + ) + print(convert_to_html_table( + results, + param_name="Sample size", + main_title=f"Sinkhorn Knopp - Averaged on {n_runs} runs" + )) diff --git a/docs/requirements.txt b/docs/requirements.txt index 95147d2..2e060b9 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -4,4 +4,4 @@ numpydoc memory_profiler pillow networkx -m2r2 \ No newline at end of file +myst-parser \ No newline at end of file diff --git a/docs/requirements_rtd.txt b/docs/requirements_rtd.txt index 5963ea2..11957fb 100644 --- a/docs/requirements_rtd.txt +++ b/docs/requirements_rtd.txt @@ -3,7 +3,7 @@ numpydoc memory_profiler pillow networkx -m2r2 +myst-parser numpy scipy>=1.0 cython diff --git a/docs/source/.github/CODE_OF_CONDUCT.rst b/docs/source/.github/CODE_OF_CONDUCT.rst new file mode 100644 index 0000000..d4c5cec --- /dev/null +++ b/docs/source/.github/CODE_OF_CONDUCT.rst @@ -0,0 +1,6 @@ +Code of Conduct +=============== + +.. include:: ../../../.github/CODE_OF_CONDUCT.md + :parser: myst_parser.sphinx_ + :start-line: 2 diff --git a/docs/source/.github/CONTRIBUTING.rst b/docs/source/.github/CONTRIBUTING.rst new file mode 100644 index 0000000..aef24e9 --- /dev/null +++ b/docs/source/.github/CONTRIBUTING.rst @@ -0,0 +1,6 @@ +Contributing to POT +=================== + +.. include:: ../../../.github/CONTRIBUTING.md + :parser: myst_parser.sphinx_ + :start-line: 3 diff --git a/docs/source/code_of_conduct.rst b/docs/source/code_of_conduct.rst deleted file mode 100644 index b37ba7b..0000000 --- a/docs/source/code_of_conduct.rst +++ /dev/null @@ -1 +0,0 @@ -.. mdinclude:: ../../.github/CODE_OF_CONDUCT.md \ No newline at end of file diff --git a/docs/source/conf.py b/docs/source/conf.py index 1320afa..849e97c 100644 --- a/docs/source/conf.py +++ b/docs/source/conf.py @@ -69,7 +69,7 @@ extensions = [ 'sphinx.ext.viewcode', 'sphinx.ext.napoleon', 'sphinx_gallery.gen_gallery', - 'm2r2' + 'myst_parser' ] autosummary_generate = True diff --git a/docs/source/contributing.rst b/docs/source/contributing.rst deleted file mode 100644 index dc81e75..0000000 --- a/docs/source/contributing.rst +++ /dev/null @@ -1 +0,0 @@ -.. mdinclude:: ../../.github/CONTRIBUTING.md \ No newline at end of file diff --git a/docs/source/index.rst b/docs/source/index.rst index 7aaa524..8de31ae 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -17,12 +17,11 @@ Contents all auto_examples/index releases - contributing - Code of Conduct - -.. mdinclude:: ../../README.md - :start-line: 2 + .github/CONTRIBUTING + .github/CODE_OF_CONDUCT +.. include:: ../../README.md + :parser: myst_parser.sphinx_ Indices and tables diff --git a/ot/backend.py b/ot/backend.py index 1630ac4..58b652b 100644 --- a/ot/backend.py +++ b/ot/backend.py @@ -3,7 +3,7 @@ Multi-lib backend for POT The goal is to write backend-agnostic code. Whether you're using Numpy, PyTorch, -Jax, or Cupy, POT code should work nonetheless. +Jax, Cupy, or Tensorflow, POT code should work nonetheless. To achieve that, POT provides backend classes which implements functions in their respective backend imitating Numpy API. As a convention, we use nx instead of np to refer to the backend. @@ -17,6 +17,68 @@ Examples ... nx = get_backend(a, b) # infer the backend from the arguments ... c = nx.dot(a, b) # now use the backend to do any calculation ... return c + +.. warning:: + Tensorflow only works with the Numpy API. To activate it, please run the following: + + .. code-block:: + + from tensorflow.python.ops.numpy_ops import np_config + np_config.enable_numpy_behavior() + +Performance +-------- + +- CPU: Intel(R) Xeon(R) Gold 6248 CPU @ 2.50GHz +- GPU: Tesla V100-SXM2-32GB +- Date of the benchmark: December 8th, 2021 +- Commit of benchmark: PR #316, https://github.com/PythonOT/POT/pull/316 + +.. raw:: html + + + +
+ + + + + + + + + + + + + + + + + + + + + +
Sinkhorn Knopp - Averaged on 100 runs
Bitsize32 bits
DeviceCPUGPU
Sample sizeNumpyPytorchTensorflowCupyJaxPytorchTensorflow
500.00080.00220.01510.00950.01930.00510.0293
1000.00050.00130.00970.00570.01150.00290.0173
5000.00090.00160.01100.00580.01150.00290.0166
10000.00210.00210.01450.00560.01180.00290.0168
20000.00690.00430.02780.00590.01180.00300.0165
50000.07070.03140.13950.00740.01250.00350.0198
 
Bitsize64 bits
DeviceCPUGPU
Sample sizeNumpyPytorchTensorflowCupyJaxPytorchTensorflow
500.00080.00200.01540.00930.01910.00510.0328
1000.00050.00130.00940.00560.01140.00290.0169
5000.00130.00170.01200.00590.01160.00290.0168
10000.00340.00270.01770.00580.01180.00290.0167
20000.01460.00750.04360.00590.01200.00290.0165
50000.14670.05680.24680.00770.01460.00450.0204
+
""" # Author: Remi Flamary @@ -27,6 +89,8 @@ Examples import numpy as np import scipy.special as scipy from scipy.sparse import issparse, coo_matrix, csr_matrix +import warnings +import time try: import torch @@ -39,6 +103,7 @@ try: import jax import jax.numpy as jnp import jax.scipy.special as jscipy + from jax.lib import xla_bridge jax_type = jax.numpy.ndarray except ImportError: jax = False @@ -52,6 +117,15 @@ except ImportError: cp = False cp_type = float +try: + import tensorflow as tf + import tensorflow.experimental.numpy as tnp + tf_type = tf.Tensor +except ImportError: + tf = False + tf_type = float + + str_type_error = "All array should be from the same type/backend. Current types are : {}" @@ -65,9 +139,12 @@ def get_backend_list(): if jax: lst.append(JaxBackend()) - if cp: + if cp: # pragma: no cover lst.append(CupyBackend()) + if tf: + lst.append(TensorflowBackend()) + return lst @@ -89,8 +166,10 @@ def get_backend(*args): return TorchBackend() elif isinstance(args[0], jax_type): return JaxBackend() - elif isinstance(args[0], cp_type): + elif isinstance(args[0], cp_type): # pragma: no cover return CupyBackend() + elif isinstance(args[0], tf_type): + return TensorflowBackend() else: raise ValueError("Unknown type of non implemented backend.") @@ -108,7 +187,7 @@ class Backend(): """ Backend abstract class. Implementations: :py:class:`JaxBackend`, :py:class:`NumpyBackend`, :py:class:`TorchBackend`, - :py:class:`CupyBackend` + :py:class:`CupyBackend`, :py:class:`TensorflowBackend` - The `__name__` class attribute refers to the name of the backend. - The `__type__` class attribute refers to the data structure used by the backend. @@ -679,6 +758,34 @@ class Backend(): """ raise NotImplementedError() + def squeeze(self, a, axis=None): + r""" + Remove axes of length one from a. + + This function follows the api from :any:`numpy.squeeze`. + + See: https://numpy.org/doc/stable/reference/generated/numpy.squeeze.html + """ + raise NotImplementedError() + + def bitsize(self, type_as): + r""" + Gives the number of bits used by the data type of the given tensor. + """ + raise NotImplementedError() + + def device_type(self, type_as): + r""" + Returns CPU or GPU depending on the device where the given tensor is located. + """ + raise NotImplementedError() + + def _bench(self, callable, *args, n_runs=1, warmup_runs=1): + r""" + Executes a benchmark of the given callable with the given arguments. + """ + raise NotImplementedError() + class NumpyBackend(Backend): """ @@ -916,6 +1023,29 @@ class NumpyBackend(Backend): # numpy has implicit type conversion so we automatically validate the test pass + def squeeze(self, a, axis=None): + return np.squeeze(a, axis=axis) + + def bitsize(self, type_as): + return type_as.itemsize * 8 + + def device_type(self, type_as): + return "CPU" + + def _bench(self, callable, *args, n_runs=1, warmup_runs=1): + results = dict() + for type_as in self.__type_list__: + inputs = [self.from_numpy(arg, type_as=type_as) for arg in args] + for _ in range(warmup_runs): + callable(*inputs) + t0 = time.perf_counter() + for _ in range(n_runs): + callable(*inputs) + t1 = time.perf_counter() + key = ("Numpy", self.device_type(type_as), self.bitsize(type_as)) + results[key] = (t1 - t0) / n_runs + return results + class JaxBackend(Backend): """ @@ -934,9 +1064,16 @@ class JaxBackend(Backend): def __init__(self): self.rng_ = jax.random.PRNGKey(42) - for d in jax.devices(): - self.__type_list__ = [jax.device_put(jnp.array(1, dtype=jnp.float32), d), - jax.device_put(jnp.array(1, dtype=jnp.float64), d)] + self.__type_list__ = [] + # available_devices = jax.devices("cpu") + available_devices = [] + if xla_bridge.get_backend().platform == "gpu": + available_devices += jax.devices("gpu") + for d in available_devices: + self.__type_list__ += [ + jax.device_put(jnp.array(1, dtype=jnp.float32), d), + jax.device_put(jnp.array(1, dtype=jnp.float64), d) + ] def to_numpy(self, a): return np.array(a) @@ -1176,6 +1313,32 @@ class JaxBackend(Backend): assert a_dtype == b_dtype, "Dtype discrepancy" assert a_device == b_device, f"Device discrepancy. First input is on {str(a_device)}, whereas second input is on {str(b_device)}" + def squeeze(self, a, axis=None): + return jnp.squeeze(a, axis=axis) + + def bitsize(self, type_as): + return type_as.dtype.itemsize * 8 + + def device_type(self, type_as): + return self.dtype_device(type_as)[1].platform.upper() + + def _bench(self, callable, *args, n_runs=1, warmup_runs=1): + results = dict() + + for type_as in self.__type_list__: + inputs = [self.from_numpy(arg, type_as=type_as) for arg in args] + for _ in range(warmup_runs): + a = callable(*inputs) + a.block_until_ready() + t0 = time.perf_counter() + for _ in range(n_runs): + a = callable(*inputs) + a.block_until_ready() + t1 = time.perf_counter() + key = ("Jax", self.device_type(type_as), self.bitsize(type_as)) + results[key] = (t1 - t0) / n_runs + return results + class TorchBackend(Backend): """ @@ -1515,6 +1678,46 @@ class TorchBackend(Backend): assert a_dtype == b_dtype, "Dtype discrepancy" assert a_device == b_device, f"Device discrepancy. First input is on {str(a_device)}, whereas second input is on {str(b_device)}" + def squeeze(self, a, axis=None): + if axis is None: + return torch.squeeze(a) + else: + return torch.squeeze(a, dim=axis) + + def bitsize(self, type_as): + return torch.finfo(type_as.dtype).bits + + def device_type(self, type_as): + return type_as.device.type.replace("cuda", "gpu").upper() + + def _bench(self, callable, *args, n_runs=1, warmup_runs=1): + results = dict() + for type_as in self.__type_list__: + inputs = [self.from_numpy(arg, type_as=type_as) for arg in args] + for _ in range(warmup_runs): + callable(*inputs) + if self.device_type(type_as) == "GPU": # pragma: no cover + torch.cuda.synchronize() + start = torch.cuda.Event(enable_timing=True) + end = torch.cuda.Event(enable_timing=True) + start.record() + else: + start = time.perf_counter() + for _ in range(n_runs): + callable(*inputs) + if self.device_type(type_as) == "GPU": # pragma: no cover + end.record() + torch.cuda.synchronize() + duration = start.elapsed_time(end) / 1000. + else: + end = time.perf_counter() + duration = end - start + key = ("Pytorch", self.device_type(type_as), self.bitsize(type_as)) + results[key] = duration / n_runs + if torch.cuda.is_available(): + torch.cuda.empty_cache() + return results + class CupyBackend(Backend): # pragma: no cover """ @@ -1798,3 +2001,366 @@ class CupyBackend(Backend): # pragma: no cover # cupy has implicit type conversion so # we automatically validate the test for type assert a_device == b_device, f"Device discrepancy. First input is on {str(a_device)}, whereas second input is on {str(b_device)}" + + def squeeze(self, a, axis=None): + return cp.squeeze(a, axis=axis) + + def bitsize(self, type_as): + return type_as.itemsize * 8 + + def device_type(self, type_as): + return "GPU" + + def _bench(self, callable, *args, n_runs=1, warmup_runs=1): + mempool = cp.get_default_memory_pool() + pinned_mempool = cp.get_default_pinned_memory_pool() + + results = dict() + for type_as in self.__type_list__: + inputs = [self.from_numpy(arg, type_as=type_as) for arg in args] + start_gpu = cp.cuda.Event() + end_gpu = cp.cuda.Event() + for _ in range(warmup_runs): + callable(*inputs) + start_gpu.synchronize() + start_gpu.record() + for _ in range(n_runs): + callable(*inputs) + end_gpu.record() + end_gpu.synchronize() + key = ("Cupy", self.device_type(type_as), self.bitsize(type_as)) + t_gpu = cp.cuda.get_elapsed_time(start_gpu, end_gpu) / 1000. + results[key] = t_gpu / n_runs + mempool.free_all_blocks() + pinned_mempool.free_all_blocks() + return results + + +class TensorflowBackend(Backend): + + __name__ = "tf" + __type__ = tf_type + __type_list__ = None + + rng_ = None + + def __init__(self): + self.seed(None) + + self.__type_list__ = [ + tf.convert_to_tensor([1], dtype=tf.float32), + tf.convert_to_tensor([1], dtype=tf.float64) + ] + + tmp = self.randn(15, 10) + try: + tmp.reshape((150, 1)) + except AttributeError: + warnings.warn( + "To use TensorflowBackend, you need to activate the tensorflow " + "numpy API. You can activate it by running: \n" + "from tensorflow.python.ops.numpy_ops import np_config\n" + "np_config.enable_numpy_behavior()" + ) + + def to_numpy(self, a): + return a.numpy() + + def from_numpy(self, a, type_as=None): + if not isinstance(a, self.__type__): + if type_as is None: + return tf.convert_to_tensor(a) + else: + return tf.convert_to_tensor(a, dtype=type_as.dtype) + else: + if type_as is None: + return a + else: + return tf.cast(a, dtype=type_as.dtype) + + def set_gradients(self, val, inputs, grads): + @tf.custom_gradient + def tmp(input): + def grad(upstream): + return grads + return val, grad + return tmp(inputs) + + def zeros(self, shape, type_as=None): + if type_as is None: + return tnp.zeros(shape) + else: + return tnp.zeros(shape, dtype=type_as.dtype) + + def ones(self, shape, type_as=None): + if type_as is None: + return tnp.ones(shape) + else: + return tnp.ones(shape, dtype=type_as.dtype) + + def arange(self, stop, start=0, step=1, type_as=None): + return tnp.arange(start, stop, step) + + def full(self, shape, fill_value, type_as=None): + if type_as is None: + return tnp.full(shape, fill_value) + else: + return tnp.full(shape, fill_value, dtype=type_as.dtype) + + def eye(self, N, M=None, type_as=None): + if type_as is None: + return tnp.eye(N, M) + else: + return tnp.eye(N, M, dtype=type_as.dtype) + + def sum(self, a, axis=None, keepdims=False): + return tnp.sum(a, axis, keepdims=keepdims) + + def cumsum(self, a, axis=None): + return tnp.cumsum(a, axis) + + def max(self, a, axis=None, keepdims=False): + return tnp.max(a, axis, keepdims=keepdims) + + def min(self, a, axis=None, keepdims=False): + return tnp.min(a, axis, keepdims=keepdims) + + def maximum(self, a, b): + return tnp.maximum(a, b) + + def minimum(self, a, b): + return tnp.minimum(a, b) + + def dot(self, a, b): + if len(b.shape) == 1: + if len(a.shape) == 1: + # inner product + return tf.reduce_sum(tf.multiply(a, b)) + else: + # matrix vector + return tf.linalg.matvec(a, b) + else: + if len(a.shape) == 1: + return tf.linalg.matvec(b.T, a.T).T + else: + return tf.matmul(a, b) + + def abs(self, a): + return tnp.abs(a) + + def exp(self, a): + return tnp.exp(a) + + def log(self, a): + return tnp.log(a) + + def sqrt(self, a): + return tnp.sqrt(a) + + def power(self, a, exponents): + return tnp.power(a, exponents) + + def norm(self, a): + return tf.math.reduce_euclidean_norm(a) + + def any(self, a): + return tnp.any(a) + + def isnan(self, a): + return tnp.isnan(a) + + def isinf(self, a): + return tnp.isinf(a) + + def einsum(self, subscripts, *operands): + return tnp.einsum(subscripts, *operands) + + def sort(self, a, axis=-1): + return tnp.sort(a, axis) + + def argsort(self, a, axis=-1): + return tnp.argsort(a, axis) + + def searchsorted(self, a, v, side='left'): + return tf.searchsorted(a, v, side=side) + + def flip(self, a, axis=None): + return tnp.flip(a, axis) + + def outer(self, a, b): + return tnp.outer(a, b) + + def clip(self, a, a_min, a_max): + return tnp.clip(a, a_min, a_max) + + def repeat(self, a, repeats, axis=None): + return tnp.repeat(a, repeats, axis) + + def take_along_axis(self, arr, indices, axis): + return tnp.take_along_axis(arr, indices, axis) + + def concatenate(self, arrays, axis=0): + return tnp.concatenate(arrays, axis) + + def zero_pad(self, a, pad_width): + return tnp.pad(a, pad_width, mode="constant") + + def argmax(self, a, axis=None): + return tnp.argmax(a, axis=axis) + + def mean(self, a, axis=None): + return tnp.mean(a, axis=axis) + + def std(self, a, axis=None): + return tnp.std(a, axis=axis) + + def linspace(self, start, stop, num): + return tnp.linspace(start, stop, num) + + def meshgrid(self, a, b): + return tnp.meshgrid(a, b) + + def diag(self, a, k=0): + return tnp.diag(a, k) + + def unique(self, a): + return tf.sort(tf.unique(tf.reshape(a, [-1]))[0]) + + def logsumexp(self, a, axis=None): + return tf.math.reduce_logsumexp(a, axis=axis) + + def stack(self, arrays, axis=0): + return tnp.stack(arrays, axis) + + def reshape(self, a, shape): + return tnp.reshape(a, shape) + + def seed(self, seed=None): + if isinstance(seed, int): + self.rng_ = tf.random.Generator.from_seed(seed) + elif isinstance(seed, tf.random.Generator): + self.rng_ = seed + elif seed is None: + self.rng_ = tf.random.Generator.from_non_deterministic_state() + else: + raise ValueError("Non compatible seed : {}".format(seed)) + + def rand(self, *size, type_as=None): + if type_as is None: + return self.rng_.uniform(size, minval=0., maxval=1.) + else: + return self.rng_.uniform( + size, minval=0., maxval=1., dtype=type_as.dtype + ) + + def randn(self, *size, type_as=None): + if type_as is None: + return self.rng_.normal(size) + else: + return self.rng_.normal(size, dtype=type_as.dtype) + + def _convert_to_index_for_coo(self, tensor): + if isinstance(tensor, self.__type__): + return int(self.max(tensor)) + 1 + else: + return int(np.max(tensor)) + 1 + + def coo_matrix(self, data, rows, cols, shape=None, type_as=None): + if shape is None: + shape = ( + self._convert_to_index_for_coo(rows), + self._convert_to_index_for_coo(cols) + ) + if type_as is not None: + data = self.from_numpy(data, type_as=type_as) + + sparse_tensor = tf.sparse.SparseTensor( + indices=tnp.stack([rows, cols]).T, + values=data, + dense_shape=shape + ) + # if type_as is not None: + # sparse_tensor = self.from_numpy(sparse_tensor, type_as=type_as) + # SparseTensor are not subscriptable so we use dense tensors + return self.todense(sparse_tensor) + + def issparse(self, a): + return isinstance(a, tf.sparse.SparseTensor) + + def tocsr(self, a): + return a + + def eliminate_zeros(self, a, threshold=0.): + if self.issparse(a): + values = a.values + if threshold > 0: + mask = self.abs(values) <= threshold + else: + mask = values == 0 + return tf.sparse.retain(a, ~mask) + else: + if threshold > 0: + a = tnp.where(self.abs(a) > threshold, a, 0.) + return a + + def todense(self, a): + if self.issparse(a): + return tf.sparse.to_dense(tf.sparse.reorder(a)) + else: + return a + + def where(self, condition, x, y): + return tnp.where(condition, x, y) + + def copy(self, a): + return tf.identity(a) + + def allclose(self, a, b, rtol=1e-05, atol=1e-08, equal_nan=False): + return tnp.allclose( + a, b, rtol=rtol, atol=atol, equal_nan=equal_nan + ) + + def dtype_device(self, a): + return a.dtype, a.device.split("device:")[1] + + def assert_same_dtype_device(self, a, b): + a_dtype, a_device = self.dtype_device(a) + b_dtype, b_device = self.dtype_device(b) + + assert a_dtype == b_dtype, "Dtype discrepancy" + assert a_device == b_device, f"Device discrepancy. First input is on {str(a_device)}, whereas second input is on {str(b_device)}" + + def squeeze(self, a, axis=None): + return tnp.squeeze(a, axis=axis) + + def bitsize(self, type_as): + return type_as.dtype.size * 8 + + def device_type(self, type_as): + return self.dtype_device(type_as)[1].split(":")[0] + + def _bench(self, callable, *args, n_runs=1, warmup_runs=1): + results = dict() + device_contexts = [tf.device("/CPU:0")] + if len(tf.config.list_physical_devices('GPU')) > 0: # pragma: no cover + device_contexts.append(tf.device("/GPU:0")) + + for device_context in device_contexts: + with device_context: + for type_as in self.__type_list__: + inputs = [self.from_numpy(arg, type_as=type_as) for arg in args] + for _ in range(warmup_runs): + callable(*inputs) + t0 = time.perf_counter() + for _ in range(n_runs): + res = callable(*inputs) + _ = res.numpy() + t1 = time.perf_counter() + key = ( + "Tensorflow", + self.device_type(inputs[0]), + self.bitsize(type_as) + ) + results[key] = (t1 - t0) / n_runs + + return results diff --git a/ot/bregman.py b/ot/bregman.py index cce52e2..fc20175 100644 --- a/ot/bregman.py +++ b/ot/bregman.py @@ -830,9 +830,9 @@ def greenkhorn(a, b, M, reg, numItermax=10000, stopThr=1e-9, verbose=False, a, b, M = list_to_array(a, b, M) nx = get_backend(M, a, b) - if nx.__name__ == "jax": - raise TypeError("JAX arrays have been received. Greenkhorn is not " - "compatible with JAX") + if nx.__name__ in ("jax", "tf"): + raise TypeError("JAX or TF arrays have been received. Greenkhorn is not " + "compatible with neither JAX nor TF") if len(a) == 0: a = nx.ones((M.shape[0],), type_as=M) / M.shape[0] @@ -865,20 +865,20 @@ def greenkhorn(a, b, M, reg, numItermax=10000, stopThr=1e-9, verbose=False, if m_viol_1 > m_viol_2: old_u = u[i_1] - new_u = a[i_1] / (K[i_1, :].dot(v)) + new_u = a[i_1] / nx.dot(K[i_1, :], v) G[i_1, :] = new_u * K[i_1, :] * v - viol[i_1] = new_u * K[i_1, :].dot(v) - a[i_1] + viol[i_1] = nx.dot(new_u * K[i_1, :], v) - a[i_1] viol_2 += (K[i_1, :].T * (new_u - old_u) * v) u[i_1] = new_u else: old_v = v[i_2] - new_v = b[i_2] / (K[:, i_2].T.dot(u)) + new_v = b[i_2] / nx.dot(K[:, i_2].T, u) G[:, i_2] = u * K[:, i_2] * new_v # aviol = (G@one_m - a) # aviol_2 = (G.T@one_n - b) viol += (-old_v + new_v) * K[:, i_2] * u - viol_2[i_2] = new_v * K[:, i_2].dot(u) - b[i_2] + viol_2[i_2] = new_v * nx.dot(K[:, i_2], u) - b[i_2] v[i_2] = new_v if stopThr_val <= stopThr: @@ -1550,9 +1550,11 @@ def _barycenter_sinkhorn_log(A, M, reg, weights=None, numItermax=1000, nx = get_backend(A, M) - if nx.__name__ == "jax": - raise NotImplementedError("Log-domain functions are not yet implemented" - " for Jax. Use numpy or torch arrays instead.") + if nx.__name__ in ("jax", "tf"): + raise NotImplementedError( + "Log-domain functions are not yet implemented" + " for Jax and tf. Use numpy or torch arrays instead." + ) if weights is None: weights = nx.ones(n_hists, type_as=A) / n_hists @@ -1886,9 +1888,11 @@ def _barycenter_debiased_log(A, M, reg, weights=None, numItermax=1000, dim, n_hists = A.shape nx = get_backend(A, M) - if nx.__name__ == "jax": - raise NotImplementedError("Log-domain functions are not yet implemented" - " for Jax. Use numpy or torch arrays instead.") + if nx.__name__ in ("jax", "tf"): + raise NotImplementedError( + "Log-domain functions are not yet implemented" + " for Jax and TF. Use numpy or torch arrays instead." + ) if weights is None: weights = nx.ones(n_hists, type_as=A) / n_hists @@ -2043,7 +2047,7 @@ def _convolutional_barycenter2d(A, reg, weights=None, numItermax=10000, log = {'err': []} bar = nx.ones(A.shape[1:], type_as=A) - bar /= bar.sum() + bar /= nx.sum(bar) U = nx.ones(A.shape, type_as=A) V = nx.ones(A.shape, type_as=A) err = 1 @@ -2069,9 +2073,11 @@ def _convolutional_barycenter2d(A, reg, weights=None, numItermax=10000, KV = convol_imgs(V) U = A / KV KU = convol_imgs(U) - bar = nx.exp((weights[:, None, None] * nx.log(KU + stabThr)).sum(axis=0)) + bar = nx.exp( + nx.sum(weights[:, None, None] * nx.log(KU + stabThr), axis=0) + ) if ii % 10 == 9: - err = (V * KU).std(axis=0).sum() + err = nx.sum(nx.std(V * KU, axis=0)) # log and verbose print if log: log['err'].append(err) @@ -2106,9 +2112,11 @@ def _convolutional_barycenter2d_log(A, reg, weights=None, numItermax=10000, A = list_to_array(A) nx = get_backend(A) - if nx.__name__ == "jax": - raise NotImplementedError("Log-domain functions are not yet implemented" - " for Jax. Use numpy or torch arrays instead.") + if nx.__name__ in ("jax", "tf"): + raise NotImplementedError( + "Log-domain functions are not yet implemented" + " for Jax and TF. Use numpy or torch arrays instead." + ) n_hists, width, height = A.shape @@ -2298,13 +2306,15 @@ def _convolutional_barycenter2d_debiased(A, reg, weights=None, numItermax=10000, KV = convol_imgs(V) U = A / KV KU = convol_imgs(U) - bar = c * nx.exp((weights[:, None, None] * nx.log(KU + stabThr)).sum(axis=0)) + bar = c * nx.exp( + nx.sum(weights[:, None, None] * nx.log(KU + stabThr), axis=0) + ) for _ in range(10): - c = (c * bar / convol_imgs(c[None]).squeeze()) ** 0.5 + c = (c * bar / nx.squeeze(convol_imgs(c[None]))) ** 0.5 if ii % 10 == 9: - err = (V * KU).std(axis=0).sum() + err = nx.sum(nx.std(V * KU, axis=0)) # log and verbose print if log: log['err'].append(err) @@ -2340,9 +2350,11 @@ def _convolutional_barycenter2d_debiased_log(A, reg, weights=None, numItermax=10 A = list_to_array(A) n_hists, width, height = A.shape nx = get_backend(A) - if nx.__name__ == "jax": - raise NotImplementedError("Log-domain functions are not yet implemented" - " for Jax. Use numpy or torch arrays instead.") + if nx.__name__ in ("jax", "tf"): + raise NotImplementedError( + "Log-domain functions are not yet implemented" + " for Jax and TF. Use numpy or torch arrays instead." + ) if weights is None: weights = nx.ones((n_hists,), type_as=A) / n_hists else: @@ -2382,7 +2394,7 @@ def _convolutional_barycenter2d_debiased_log(A, reg, weights=None, numItermax=10 c = 0.5 * (c + log_bar - convol_img(c)) if ii % 10 == 9: - err = nx.exp(G + log_KU).std(axis=0).sum() + err = nx.sum(nx.std(nx.exp(G + log_KU), axis=0)) # log and verbose print if log: log['err'].append(err) @@ -3312,9 +3324,9 @@ def screenkhorn(a, b, M, reg, ns_budget=None, nt_budget=None, uniform=False, a, b, M = list_to_array(a, b, M) nx = get_backend(M, a, b) - if nx.__name__ == "jax": - raise TypeError("JAX arrays have been received but screenkhorn is not " - "compatible with JAX.") + if nx.__name__ in ("jax", "tf"): + raise TypeError("JAX or TF arrays have been received but screenkhorn is not " + "compatible with neither JAX nor TF.") ns, nt = M.shape @@ -3328,7 +3340,7 @@ def screenkhorn(a, b, M, reg, ns_budget=None, nt_budget=None, uniform=False, K = nx.exp(-M / reg) def projection(u, epsilon): - u[u <= epsilon] = epsilon + u = nx.maximum(u, epsilon) return u # ----------------------------------------------------------------------------------------------------------------# diff --git a/ot/da.py b/ot/da.py index 4fd97df..841f31a 100644 --- a/ot/da.py +++ b/ot/da.py @@ -906,7 +906,7 @@ def emd_laplace(a, b, xs, xt, M, sim='knn', sim_param=None, reg='pos', eta=1, al def distribution_estimation_uniform(X): - """estimates a uniform distribution from an array of samples :math:`\mathbf{X}` + r"""estimates a uniform distribution from an array of samples :math:`\mathbf{X}` Parameters ---------- @@ -950,7 +950,7 @@ class BaseTransport(BaseEstimator): """ def fit(self, Xs=None, ys=None, Xt=None, yt=None): - """Build a coupling matrix from source and target sets of samples + r"""Build a coupling matrix from source and target sets of samples :math:`(\mathbf{X_s}, \mathbf{y_s})` and :math:`(\mathbf{X_t}, \mathbf{y_t})` Parameters @@ -1010,7 +1010,7 @@ class BaseTransport(BaseEstimator): return self def fit_transform(self, Xs=None, ys=None, Xt=None, yt=None): - """Build a coupling matrix from source and target sets of samples + r"""Build a coupling matrix from source and target sets of samples :math:`(\mathbf{X_s}, \mathbf{y_s})` and :math:`(\mathbf{X_t}, \mathbf{y_t})` and transports source samples :math:`\mathbf{X_s}` onto target ones :math:`\mathbf{X_t}` @@ -1038,7 +1038,7 @@ class BaseTransport(BaseEstimator): return self.fit(Xs, ys, Xt, yt).transform(Xs, ys, Xt, yt) def transform(self, Xs=None, ys=None, Xt=None, yt=None, batch_size=128): - """Transports source samples :math:`\mathbf{X_s}` onto target ones :math:`\mathbf{X_t}` + r"""Transports source samples :math:`\mathbf{X_s}` onto target ones :math:`\mathbf{X_t}` Parameters ---------- @@ -1105,7 +1105,7 @@ class BaseTransport(BaseEstimator): return transp_Xs def transform_labels(self, ys=None): - """Propagate source labels :math:`\mathbf{y_s}` to obtain estimated target labels as in + r"""Propagate source labels :math:`\mathbf{y_s}` to obtain estimated target labels as in :ref:`[27] `. Parameters @@ -1152,7 +1152,7 @@ class BaseTransport(BaseEstimator): def inverse_transform(self, Xs=None, ys=None, Xt=None, yt=None, batch_size=128): - """Transports target samples :math:`\mathbf{X_t}` onto source samples :math:`\mathbf{X_s}` + r"""Transports target samples :math:`\mathbf{X_t}` onto source samples :math:`\mathbf{X_s}` Parameters ---------- @@ -1218,7 +1218,7 @@ class BaseTransport(BaseEstimator): return transp_Xt def inverse_transform_labels(self, yt=None): - """Propagate target labels :math:`\mathbf{y_t}` to obtain estimated source labels + r"""Propagate target labels :math:`\mathbf{y_t}` to obtain estimated source labels :math:`\mathbf{y_s}` Parameters @@ -1307,7 +1307,7 @@ class LinearTransport(BaseTransport): self.distribution_estimation = distribution_estimation def fit(self, Xs=None, ys=None, Xt=None, yt=None): - """Build a coupling matrix from source and target sets of samples + r"""Build a coupling matrix from source and target sets of samples :math:`(\mathbf{X_s}, \mathbf{y_s})` and :math:`(\mathbf{X_t}, \mathbf{y_t})` Parameters @@ -1354,7 +1354,7 @@ class LinearTransport(BaseTransport): return self def transform(self, Xs=None, ys=None, Xt=None, yt=None, batch_size=128): - """Transports source samples :math:`\mathbf{X_s}` onto target ones :math:`\mathbf{X_t}` + r"""Transports source samples :math:`\mathbf{X_s}` onto target ones :math:`\mathbf{X_t}` Parameters ---------- @@ -1387,7 +1387,7 @@ class LinearTransport(BaseTransport): def inverse_transform(self, Xs=None, ys=None, Xt=None, yt=None, batch_size=128): - """Transports target samples :math:`\mathbf{X_t}` onto source samples :math:`\mathbf{X_s}` + r"""Transports target samples :math:`\mathbf{X_t}` onto source samples :math:`\mathbf{X_s}` Parameters ---------- @@ -1493,7 +1493,7 @@ class SinkhornTransport(BaseTransport): self.out_of_sample_map = out_of_sample_map def fit(self, Xs=None, ys=None, Xt=None, yt=None): - """Build a coupling matrix from source and target sets of samples + r"""Build a coupling matrix from source and target sets of samples :math:`(\mathbf{X_s}, \mathbf{y_s})` and :math:`(\mathbf{X_t}, \mathbf{y_t})` Parameters @@ -1592,7 +1592,7 @@ class EMDTransport(BaseTransport): self.max_iter = max_iter def fit(self, Xs, ys=None, Xt=None, yt=None): - """Build a coupling matrix from source and target sets of samples + r"""Build a coupling matrix from source and target sets of samples :math:`(\mathbf{X_s}, \mathbf{y_s})` and :math:`(\mathbf{X_t}, \mathbf{y_t})` Parameters @@ -1711,7 +1711,7 @@ class SinkhornLpl1Transport(BaseTransport): self.limit_max = limit_max def fit(self, Xs, ys=None, Xt=None, yt=None): - """Build a coupling matrix from source and target sets of samples + r"""Build a coupling matrix from source and target sets of samples :math:`(\mathbf{X_s}, \mathbf{y_s})` and :math:`(\mathbf{X_t}, \mathbf{y_t})` Parameters @@ -1839,7 +1839,7 @@ class EMDLaplaceTransport(BaseTransport): self.out_of_sample_map = out_of_sample_map def fit(self, Xs, ys=None, Xt=None, yt=None): - """Build a coupling matrix from source and target sets of samples + r"""Build a coupling matrix from source and target sets of samples :math:`(\mathbf{X_s}, \mathbf{y_s})` and :math:`(\mathbf{X_t}, \mathbf{y_t})` Parameters @@ -1962,7 +1962,7 @@ class SinkhornL1l2Transport(BaseTransport): self.limit_max = limit_max def fit(self, Xs, ys=None, Xt=None, yt=None): - """Build a coupling matrix from source and target sets of samples + r"""Build a coupling matrix from source and target sets of samples :math:`(\mathbf{X_s}, \mathbf{y_s})` and :math:`(\mathbf{X_t}, \mathbf{y_t})` Parameters @@ -2088,7 +2088,7 @@ class MappingTransport(BaseEstimator): self.verbose2 = verbose2 def fit(self, Xs=None, ys=None, Xt=None, yt=None): - """Builds an optimal coupling and estimates the associated mapping + r"""Builds an optimal coupling and estimates the associated mapping from source and target sets of samples :math:`(\mathbf{X_s}, \mathbf{y_s})` and :math:`(\mathbf{X_t}, \mathbf{y_t})` @@ -2146,7 +2146,7 @@ class MappingTransport(BaseEstimator): return self def transform(self, Xs): - """Transports source samples :math:`\mathbf{X_s}` onto target ones :math:`\mathbf{X_t}` + r"""Transports source samples :math:`\mathbf{X_s}` onto target ones :math:`\mathbf{X_t}` Parameters ---------- @@ -2261,7 +2261,7 @@ class UnbalancedSinkhornTransport(BaseTransport): self.limit_max = limit_max def fit(self, Xs, ys=None, Xt=None, yt=None): - """Build a coupling matrix from source and target sets of samples + r"""Build a coupling matrix from source and target sets of samples :math:`(\mathbf{X_s}, \mathbf{y_s})` and :math:`(\mathbf{X_t}, \mathbf{y_t})` Parameters @@ -2373,7 +2373,7 @@ class JCPOTTransport(BaseTransport): self.out_of_sample_map = out_of_sample_map def fit(self, Xs, ys=None, Xt=None, yt=None): - """Building coupling matrices from a list of source and target sets of samples + r"""Building coupling matrices from a list of source and target sets of samples :math:`(\mathbf{X_s}, \mathbf{y_s})` and :math:`(\mathbf{X_t}, \mathbf{y_t})` Parameters @@ -2419,7 +2419,7 @@ class JCPOTTransport(BaseTransport): return self def transform(self, Xs=None, ys=None, Xt=None, yt=None, batch_size=128): - """Transports source samples :math:`\mathbf{X_s}` onto target ones :math:`\mathbf{X_t}` + r"""Transports source samples :math:`\mathbf{X_s}` onto target ones :math:`\mathbf{X_t}` Parameters ---------- @@ -2491,7 +2491,7 @@ class JCPOTTransport(BaseTransport): return transp_Xs def transform_labels(self, ys=None): - """Propagate source labels :math:`\mathbf{y_s}` to obtain target labels as in + r"""Propagate source labels :math:`\mathbf{y_s}` to obtain target labels as in :ref:`[27] ` Parameters @@ -2542,7 +2542,7 @@ class JCPOTTransport(BaseTransport): return yt.T def inverse_transform_labels(self, yt=None): - """Propagate target labels :math:`\mathbf{y_t}` to obtain estimated source labels + r"""Propagate target labels :math:`\mathbf{y_t}` to obtain estimated source labels :math:`\mathbf{y_s}` Parameters diff --git a/ot/datasets.py b/ot/datasets.py index ad6390c..a839074 100644 --- a/ot/datasets.py +++ b/ot/datasets.py @@ -41,7 +41,7 @@ def get_1D_gauss(n, m, sigma): def make_2D_samples_gauss(n, m, sigma, random_state=None): - """Return `n` samples drawn from 2D gaussian :math:`\mathcal{N}(m, \sigma)` + r"""Return `n` samples drawn from 2D gaussian :math:`\mathcal{N}(m, \sigma)` Parameters ---------- diff --git a/ot/dr.py b/ot/dr.py index c2f51f8..1671ca0 100644 --- a/ot/dr.py +++ b/ot/dr.py @@ -16,6 +16,7 @@ Dimension reduction with OT from scipy import linalg import autograd.numpy as np +from pymanopt.function import Autograd from pymanopt.manifolds import Stiefel from pymanopt import Problem from pymanopt.solvers import SteepestDescent, TrustRegions @@ -181,6 +182,7 @@ def wda(X, y, p=2, reg=1, k=10, solver=None, maxiter=100, verbose=0, P0=None, no else: regmean = np.ones((len(xc), len(xc))) + @Autograd def cost(P): # wda loss loss_b = 0 diff --git a/ot/gromov.py b/ot/gromov.py index dc95c74..6544260 100644 --- a/ot/gromov.py +++ b/ot/gromov.py @@ -947,7 +947,7 @@ def pointwise_gromov_wasserstein(C1, C2, p, q, loss_fun, index[0] = generator.choice(len_p, size=1, p=nx.to_numpy(p)) T_index0 = nx.reshape(nx.todense(T[index[0], :]), (-1,)) index[1] = generator.choice( - len_q, size=1, p=nx.to_numpy(T_index0 / T_index0.sum()) + len_q, size=1, p=nx.to_numpy(T_index0 / nx.sum(T_index0)) ) if alpha == 1: diff --git a/ot/lp/solver_1d.py b/ot/lp/solver_1d.py index 8b4d0c3..43763a9 100644 --- a/ot/lp/solver_1d.py +++ b/ot/lp/solver_1d.py @@ -100,11 +100,11 @@ def wasserstein_1d(u_values, v_values, u_weights=None, v_weights=None, p=1, requ m = v_values.shape[0] if u_weights is None: - u_weights = nx.full(u_values.shape, 1. / n) + u_weights = nx.full(u_values.shape, 1. / n, type_as=u_values) elif u_weights.ndim != u_values.ndim: u_weights = nx.repeat(u_weights[..., None], u_values.shape[-1], -1) if v_weights is None: - v_weights = nx.full(v_values.shape, 1. / m) + v_weights = nx.full(v_values.shape, 1. / m, type_as=v_values) elif v_weights.ndim != v_values.ndim: v_weights = nx.repeat(v_weights[..., None], v_values.shape[-1], -1) diff --git a/ot/plot.py b/ot/plot.py index 3e3bed7..2208c90 100644 --- a/ot/plot.py +++ b/ot/plot.py @@ -18,7 +18,7 @@ from matplotlib import gridspec def plot1D_mat(a, b, M, title=''): - """ Plot matrix :math:`\mathbf{M}` with the source and target 1D distribution + r""" Plot matrix :math:`\mathbf{M}` with the source and target 1D distribution Creates a subplot with the source distribution :math:`\mathbf{a}` on the left and target distribution :math:`\mathbf{b}` on the top. The matrix :math:`\mathbf{M}` is shown in between. @@ -61,7 +61,7 @@ def plot1D_mat(a, b, M, title=''): def plot2D_samples_mat(xs, xt, G, thr=1e-8, **kwargs): - """ Plot matrix :math:`\mathbf{G}` in 2D with lines using alpha values + r""" Plot matrix :math:`\mathbf{G}` in 2D with lines using alpha values Plot lines between source and target 2D samples with a color proportional to the value of the matrix :math:`\mathbf{G}` between samples. diff --git a/requirements.txt b/requirements.txt index 4353247..d43be7a 100644 --- a/requirements.txt +++ b/requirements.txt @@ -4,10 +4,11 @@ cython matplotlib autograd pymanopt==0.2.4; python_version <'3' -pymanopt; python_version >= '3' +pymanopt==0.2.6rc1; python_version >= '3' cvxopt scikit-learn torch jax jaxlib +tensorflow pytest \ No newline at end of file diff --git a/test/conftest.py b/test/conftest.py index 987d98e..c0db8ab 100644 --- a/test/conftest.py +++ b/test/conftest.py @@ -5,7 +5,7 @@ # License: MIT License import pytest -from ot.backend import jax +from ot.backend import jax, tf from ot.backend import get_backend_list import functools @@ -13,6 +13,10 @@ if jax: from jax.config import config config.update("jax_enable_x64", True) +if tf: + from tensorflow.python.ops.numpy_ops import np_config + np_config.enable_numpy_behavior() + backend_list = get_backend_list() @@ -24,16 +28,16 @@ def nx(request): def skip_arg(arg, value, reason=None, getter=lambda x: x): - if isinstance(arg, tuple) or isinstance(arg, list): + if isinstance(arg, (tuple, list)): n = len(arg) else: arg = (arg, ) n = 1 - if n != 1 and (isinstance(value, tuple) or isinstance(value, list)): + if n != 1 and isinstance(value, (tuple, list)): pass else: value = (value, ) - if isinstance(getter, tuple) or isinstance(value, list): + if isinstance(getter, (tuple, list)): pass else: getter = [getter] * n diff --git a/test/test_1d_solver.py b/test/test_1d_solver.py index cb85cb9..6a42cfe 100644 --- a/test/test_1d_solver.py +++ b/test/test_1d_solver.py @@ -11,7 +11,7 @@ import pytest import ot from ot.lp import wasserstein_1d -from ot.backend import get_backend_list +from ot.backend import get_backend_list, tf from scipy.stats import wasserstein_distance backend_list = get_backend_list() @@ -86,7 +86,6 @@ def test_wasserstein_1d(nx): def test_wasserstein_1d_type_devices(nx): - rng = np.random.RandomState(0) n = 10 @@ -108,6 +107,37 @@ def test_wasserstein_1d_type_devices(nx): nx.assert_same_dtype_device(xb, res) +@pytest.mark.skipif(not tf, reason="tf not installed") +def test_wasserstein_1d_device_tf(): + if not tf: + return + nx = ot.backend.TensorflowBackend() + rng = np.random.RandomState(0) + n = 10 + x = np.linspace(0, 5, n) + rho_u = np.abs(rng.randn(n)) + rho_u /= rho_u.sum() + rho_v = np.abs(rng.randn(n)) + rho_v /= rho_v.sum() + + # Check that everything stays on the CPU + with tf.device("/CPU:0"): + xb = nx.from_numpy(x) + rho_ub = nx.from_numpy(rho_u) + rho_vb = nx.from_numpy(rho_v) + res = wasserstein_1d(xb, xb, rho_ub, rho_vb, p=1) + nx.assert_same_dtype_device(xb, res) + + if len(tf.config.list_physical_devices('GPU')) > 0: + # Check that everything happens on the GPU + xb = nx.from_numpy(x) + rho_ub = nx.from_numpy(rho_u) + rho_vb = nx.from_numpy(rho_v) + res = wasserstein_1d(xb, xb, rho_ub, rho_vb, p=1) + nx.assert_same_dtype_device(xb, res) + assert nx.dtype_device(res)[1].startswith("GPU") + + def test_emd_1d_emd2_1d(): # test emd1d gives similar results as emd n = 20 @@ -148,7 +178,6 @@ def test_emd_1d_emd2_1d(): def test_emd1d_type_devices(nx): - rng = np.random.RandomState(0) n = 10 @@ -170,3 +199,36 @@ def test_emd1d_type_devices(nx): nx.assert_same_dtype_device(xb, emd) nx.assert_same_dtype_device(xb, emd2) + + +@pytest.mark.skipif(not tf, reason="tf not installed") +def test_emd1d_device_tf(): + nx = ot.backend.TensorflowBackend() + rng = np.random.RandomState(0) + n = 10 + x = np.linspace(0, 5, n) + rho_u = np.abs(rng.randn(n)) + rho_u /= rho_u.sum() + rho_v = np.abs(rng.randn(n)) + rho_v /= rho_v.sum() + + # Check that everything stays on the CPU + with tf.device("/CPU:0"): + xb = nx.from_numpy(x) + rho_ub = nx.from_numpy(rho_u) + rho_vb = nx.from_numpy(rho_v) + emd = ot.emd_1d(xb, xb, rho_ub, rho_vb) + emd2 = ot.emd2_1d(xb, xb, rho_ub, rho_vb) + nx.assert_same_dtype_device(xb, emd) + nx.assert_same_dtype_device(xb, emd2) + + if len(tf.config.list_physical_devices('GPU')) > 0: + # Check that everything happens on the GPU + xb = nx.from_numpy(x) + rho_ub = nx.from_numpy(rho_u) + rho_vb = nx.from_numpy(rho_v) + emd = ot.emd_1d(xb, xb, rho_ub, rho_vb) + emd2 = ot.emd2_1d(xb, xb, rho_ub, rho_vb) + nx.assert_same_dtype_device(xb, emd) + nx.assert_same_dtype_device(xb, emd2) + assert nx.dtype_device(emd)[1].startswith("GPU") diff --git a/test/test_backend.py b/test/test_backend.py index 2e7eecc..027c4cd 100644 --- a/test/test_backend.py +++ b/test/test_backend.py @@ -7,7 +7,7 @@ import ot import ot.backend -from ot.backend import torch, jax, cp +from ot.backend import torch, jax, cp, tf import pytest @@ -101,6 +101,20 @@ def test_get_backend(): with pytest.raises(ValueError): get_backend(A, B2) + if tf: + A2 = tf.convert_to_tensor(A) + B2 = tf.convert_to_tensor(B) + + nx = get_backend(A2) + assert nx.__name__ == 'tf' + + nx = get_backend(A2, B2) + assert nx.__name__ == 'tf' + + # test not unique types in input + with pytest.raises(ValueError): + get_backend(A, B2) + def test_convert_between_backends(nx): @@ -242,6 +256,14 @@ def test_empty_backend(): nx.copy(M) with pytest.raises(NotImplementedError): nx.allclose(M, M) + with pytest.raises(NotImplementedError): + nx.squeeze(M) + with pytest.raises(NotImplementedError): + nx.bitsize(M) + with pytest.raises(NotImplementedError): + nx.device_type(M) + with pytest.raises(NotImplementedError): + nx._bench(lambda x: x, M, n_runs=1) def test_func_backends(nx): @@ -491,7 +513,7 @@ def test_func_backends(nx): lst_name.append('coo_matrix') assert not nx.issparse(Mb), 'Assert fail on: issparse (expected False)' - assert nx.issparse(sp_Mb) or nx.__name__ == "jax", 'Assert fail on: issparse (expected True)' + assert nx.issparse(sp_Mb) or nx.__name__ in ("jax", "tf"), 'Assert fail on: issparse (expected True)' A = nx.tocsr(sp_Mb) lst_b.append(nx.to_numpy(nx.todense(A))) @@ -516,6 +538,18 @@ def test_func_backends(nx): assert nx.allclose(Mb, Mb), 'Assert fail on: allclose (expected True)' assert not nx.allclose(2 * Mb, Mb), 'Assert fail on: allclose (expected False)' + A = nx.squeeze(nx.zeros((3, 1, 4, 1))) + assert tuple(A.shape) == (3, 4), 'Assert fail on: squeeze' + + A = nx.bitsize(Mb) + lst_b.append(float(A)) + lst_name.append("bitsize") + + A = nx.device_type(Mb) + assert A in ("CPU", "GPU") + + nx._bench(lambda x: x, M, n_runs=1) + lst_tot.append(lst_b) lst_np = lst_tot[0] @@ -590,3 +624,17 @@ def test_gradients_backends(): np.testing.assert_almost_equal(fun(v, c, e), c * np.sum(v ** 4) + e, decimal=4) np.testing.assert_allclose(grad_val[0], v, atol=1e-4) np.testing.assert_allclose(grad_val[2], 2 * e, atol=1e-4) + + if tf: + nx = ot.backend.TensorflowBackend() + w = tf.Variable(tf.random.normal((3, 2)), name='w') + b = tf.Variable(tf.random.normal((2,), dtype=tf.float32), name='b') + x = tf.random.normal((1, 3), dtype=tf.float32) + + with tf.GradientTape() as tape: + y = x @ w + b + loss = tf.reduce_mean(y ** 2) + manipulated_loss = nx.set_gradients(loss, (w, b), (w, b)) + [dl_dw, dl_db] = tape.gradient(manipulated_loss, [w, b]) + assert nx.allclose(dl_dw, w) + assert nx.allclose(dl_db, b) diff --git a/test/test_bregman.py b/test/test_bregman.py index f42ac6f..6e90aa4 100644 --- a/test/test_bregman.py +++ b/test/test_bregman.py @@ -12,7 +12,7 @@ import numpy as np import pytest import ot -from ot.backend import torch +from ot.backend import torch, tf @pytest.mark.parametrize("verbose, warn", product([True, False], [True, False])) @@ -248,6 +248,7 @@ def test_sinkhorn_empty(): ot.sinkhorn([], [], M, 1, method='greenkhorn', stopThr=1e-10, log=True) +@pytest.skip_backend('tf') @pytest.skip_backend("jax") def test_sinkhorn_variants(nx): # test sinkhorn @@ -282,6 +283,8 @@ def test_sinkhorn_variants(nx): "sinkhorn_epsilon_scaling", "greenkhorn", "sinkhorn_log"]) +@pytest.skip_arg(("nx", "method"), ("tf", "sinkhorn_epsilon_scaling"), reason="tf does not support sinkhorn_epsilon_scaling", getter=str) +@pytest.skip_arg(("nx", "method"), ("tf", "greenkhorn"), reason="tf does not support greenkhorn", getter=str) @pytest.skip_arg(("nx", "method"), ("jax", "sinkhorn_epsilon_scaling"), reason="jax does not support sinkhorn_epsilon_scaling", getter=str) @pytest.skip_arg(("nx", "method"), ("jax", "greenkhorn"), reason="jax does not support greenkhorn", getter=str) def test_sinkhorn_variants_dtype_device(nx, method): @@ -323,6 +326,36 @@ def test_sinkhorn2_variants_dtype_device(nx, method): nx.assert_same_dtype_device(Mb, lossb) +@pytest.mark.skipif(not tf, reason="tf not installed") +@pytest.mark.parametrize("method", ["sinkhorn", "sinkhorn_stabilized", "sinkhorn_log"]) +def test_sinkhorn2_variants_device_tf(method): + nx = ot.backend.TensorflowBackend() + n = 100 + x = np.random.randn(n, 2) + u = ot.utils.unif(n) + M = ot.dist(x, x) + + # Check that everything stays on the CPU + with tf.device("/CPU:0"): + ub = nx.from_numpy(u) + Mb = nx.from_numpy(M) + Gb = ot.sinkhorn(ub, ub, Mb, 1, method=method, stopThr=1e-10) + lossb = ot.sinkhorn2(ub, ub, Mb, 1, method=method, stopThr=1e-10) + nx.assert_same_dtype_device(Mb, Gb) + nx.assert_same_dtype_device(Mb, lossb) + + if len(tf.config.list_physical_devices('GPU')) > 0: + # Check that everything happens on the GPU + ub = nx.from_numpy(u) + Mb = nx.from_numpy(M) + Gb = ot.sinkhorn(ub, ub, Mb, 1, method=method, stopThr=1e-10) + lossb = ot.sinkhorn2(ub, ub, Mb, 1, method=method, stopThr=1e-10) + nx.assert_same_dtype_device(Mb, Gb) + nx.assert_same_dtype_device(Mb, lossb) + assert nx.dtype_device(Gb)[1].startswith("GPU") + + +@pytest.skip_backend('tf') @pytest.skip_backend("jax") def test_sinkhorn_variants_multi_b(nx): # test sinkhorn @@ -352,6 +385,7 @@ def test_sinkhorn_variants_multi_b(nx): np.testing.assert_allclose(G0, Gs, atol=1e-05) +@pytest.skip_backend('tf') @pytest.skip_backend("jax") def test_sinkhorn2_variants_multi_b(nx): # test sinkhorn @@ -454,7 +488,7 @@ def test_barycenter(nx, method, verbose, warn): weights_nx = nx.from_numpy(weights) reg = 1e-2 - if nx.__name__ == "jax" and method == "sinkhorn_log": + if nx.__name__ in ("jax", "tf") and method == "sinkhorn_log": with pytest.raises(NotImplementedError): ot.bregman.barycenter(A_nx, M_nx, reg, weights, method=method) else: @@ -495,7 +529,7 @@ def test_barycenter_debiased(nx, method, verbose, warn): # wasserstein reg = 1e-2 - if nx.__name__ == "jax" and method == "sinkhorn_log": + if nx.__name__ in ("jax", "tf") and method == "sinkhorn_log": with pytest.raises(NotImplementedError): ot.bregman.barycenter_debiased(A_nx, M_nx, reg, weights, method=method) else: @@ -597,7 +631,7 @@ def test_wasserstein_bary_2d(nx, method): # wasserstein reg = 1e-2 - if nx.__name__ == "jax" and method == "sinkhorn_log": + if nx.__name__ in ("jax", "tf") and method == "sinkhorn_log": with pytest.raises(NotImplementedError): ot.bregman.convolutional_barycenter2d(A_nx, reg, method=method) else: @@ -629,7 +663,7 @@ def test_wasserstein_bary_2d_debiased(nx, method): # wasserstein reg = 1e-2 - if nx.__name__ == "jax" and method == "sinkhorn_log": + if nx.__name__ in ("jax", "tf") and method == "sinkhorn_log": with pytest.raises(NotImplementedError): ot.bregman.convolutional_barycenter2d_debiased(A_nx, reg, method=method) else: @@ -888,6 +922,7 @@ def test_implemented_methods(): ot.bregman.sinkhorn2(a, b, M, epsilon, method=method) +@pytest.skip_backend('tf') @pytest.skip_backend("cupy") @pytest.skip_backend("jax") @pytest.mark.filterwarnings("ignore:Bottleneck") diff --git a/test/test_gromov.py b/test/test_gromov.py index 38a7fd7..4b995d5 100644 --- a/test/test_gromov.py +++ b/test/test_gromov.py @@ -9,7 +9,7 @@ import numpy as np import ot from ot.backend import NumpyBackend -from ot.backend import torch +from ot.backend import torch, tf import pytest @@ -113,6 +113,45 @@ def test_gromov_dtype_device(nx): nx.assert_same_dtype_device(C1b, gw_valb) +@pytest.mark.skipif(not tf, reason="tf not installed") +def test_gromov_device_tf(): + nx = ot.backend.TensorflowBackend() + n_samples = 50 # nb samples + mu_s = np.array([0, 0]) + cov_s = np.array([[1, 0], [0, 1]]) + xs = ot.datasets.make_2D_samples_gauss(n_samples, mu_s, cov_s, random_state=4) + xt = xs[::-1].copy() + p = ot.unif(n_samples) + q = ot.unif(n_samples) + C1 = ot.dist(xs, xs) + C2 = ot.dist(xt, xt) + C1 /= C1.max() + C2 /= C2.max() + + # Check that everything stays on the CPU + with tf.device("/CPU:0"): + C1b = nx.from_numpy(C1) + C2b = nx.from_numpy(C2) + pb = nx.from_numpy(p) + qb = nx.from_numpy(q) + Gb = ot.gromov.gromov_wasserstein(C1b, C2b, pb, qb, 'square_loss', verbose=True) + gw_valb = ot.gromov.gromov_wasserstein2(C1b, C2b, pb, qb, 'kl_loss', log=False) + nx.assert_same_dtype_device(C1b, Gb) + nx.assert_same_dtype_device(C1b, gw_valb) + + if len(tf.config.list_physical_devices('GPU')) > 0: + # Check that everything happens on the GPU + C1b = nx.from_numpy(C1) + C2b = nx.from_numpy(C2) + pb = nx.from_numpy(p) + qb = nx.from_numpy(q) + Gb = ot.gromov.gromov_wasserstein(C1b, C2b, pb, qb, 'square_loss', verbose=True) + gw_valb = ot.gromov.gromov_wasserstein2(C1b, C2b, pb, qb, 'kl_loss', log=False) + nx.assert_same_dtype_device(C1b, Gb) + nx.assert_same_dtype_device(C1b, gw_valb) + assert nx.dtype_device(Gb)[1].startswith("GPU") + + def test_gromov2_gradients(): n_samples = 50 # nb samples @@ -150,6 +189,7 @@ def test_gromov2_gradients(): @pytest.skip_backend("jax", reason="test very slow with jax backend") +@pytest.skip_backend("tf", reason="test very slow with tf backend") def test_entropic_gromov(nx): n_samples = 50 # nb samples @@ -208,6 +248,7 @@ def test_entropic_gromov(nx): @pytest.skip_backend("jax", reason="test very slow with jax backend") +@pytest.skip_backend("tf", reason="test very slow with tf backend") def test_entropic_gromov_dtype_device(nx): # setup n_samples = 50 # nb samples @@ -306,6 +347,7 @@ def test_pointwise_gromov(nx): np.testing.assert_allclose(float(logb['gw_dist_std']), 0.0015952535464736394, atol=1e-8) +@pytest.skip_backend("tf", reason="test very slow with tf backend") @pytest.skip_backend("jax", reason="test very slow with jax backend") def test_sampled_gromov(nx): n_samples = 50 # nb samples diff --git a/test/test_ot.py b/test/test_ot.py index c4d7713..53edf4f 100644 --- a/test/test_ot.py +++ b/test/test_ot.py @@ -11,7 +11,7 @@ import pytest import ot from ot.datasets import make_1D_gauss as gauss -from ot.backend import torch +from ot.backend import torch, tf def test_emd_dimension_and_mass_mismatch(): @@ -101,6 +101,40 @@ def test_emd_emd2_types_devices(nx): nx.assert_same_dtype_device(Mb, w) +@pytest.mark.skipif(not tf, reason="tf not installed") +def test_emd_emd2_devices_tf(): + if not tf: + return + nx = ot.backend.TensorflowBackend() + + n_samples = 100 + n_features = 2 + rng = np.random.RandomState(0) + x = rng.randn(n_samples, n_features) + y = rng.randn(n_samples, n_features) + a = ot.utils.unif(n_samples) + M = ot.dist(x, y) + + # Check that everything stays on the CPU + with tf.device("/CPU:0"): + ab = nx.from_numpy(a) + Mb = nx.from_numpy(M) + Gb = ot.emd(ab, ab, Mb) + w = ot.emd2(ab, ab, Mb) + nx.assert_same_dtype_device(Mb, Gb) + nx.assert_same_dtype_device(Mb, w) + + if len(tf.config.list_physical_devices('GPU')) > 0: + # Check that everything happens on the GPU + ab = nx.from_numpy(a) + Mb = nx.from_numpy(M) + Gb = ot.emd(ab, ab, Mb) + w = ot.emd2(ab, ab, Mb) + nx.assert_same_dtype_device(Mb, Gb) + nx.assert_same_dtype_device(Mb, w) + assert nx.dtype_device(Gb)[1].startswith("GPU") + + def test_emd2_gradients(): n_samples = 100 n_features = 2 diff --git a/test/test_sliced.py b/test/test_sliced.py index 245202c..91e0961 100644 --- a/test/test_sliced.py +++ b/test/test_sliced.py @@ -10,6 +10,7 @@ import pytest import ot from ot.sliced import get_random_projections +from ot.backend import tf def test_get_random_projections(): @@ -161,6 +162,34 @@ def test_sliced_backend_type_devices(nx): nx.assert_same_dtype_device(xb, valb) +@pytest.mark.skipif(not tf, reason="tf not installed") +def test_sliced_backend_device_tf(): + nx = ot.backend.TensorflowBackend() + n = 100 + rng = np.random.RandomState(0) + x = rng.randn(n, 2) + y = rng.randn(2 * n, 2) + P = rng.randn(2, 20) + P = P / np.sqrt((P**2).sum(0, keepdims=True)) + + # Check that everything stays on the CPU + with tf.device("/CPU:0"): + xb = nx.from_numpy(x) + yb = nx.from_numpy(y) + Pb = nx.from_numpy(P) + valb = ot.sliced_wasserstein_distance(xb, yb, projections=Pb) + nx.assert_same_dtype_device(xb, valb) + + if len(tf.config.list_physical_devices('GPU')) > 0: + # Check that everything happens on the GPU + xb = nx.from_numpy(x) + yb = nx.from_numpy(y) + Pb = nx.from_numpy(P) + valb = ot.sliced_wasserstein_distance(xb, yb, projections=Pb) + nx.assert_same_dtype_device(xb, valb) + assert nx.dtype_device(valb)[1].startswith("GPU") + + def test_max_sliced_backend(nx): n = 100 @@ -211,3 +240,31 @@ def test_max_sliced_backend_type_devices(nx): valb = ot.max_sliced_wasserstein_distance(xb, yb, projections=Pb) nx.assert_same_dtype_device(xb, valb) + + +@pytest.mark.skipif(not tf, reason="tf not installed") +def test_max_sliced_backend_device_tf(): + nx = ot.backend.TensorflowBackend() + n = 100 + rng = np.random.RandomState(0) + x = rng.randn(n, 2) + y = rng.randn(2 * n, 2) + P = rng.randn(2, 20) + P = P / np.sqrt((P**2).sum(0, keepdims=True)) + + # Check that everything stays on the CPU + with tf.device("/CPU:0"): + xb = nx.from_numpy(x) + yb = nx.from_numpy(y) + Pb = nx.from_numpy(P) + valb = ot.max_sliced_wasserstein_distance(xb, yb, projections=Pb) + nx.assert_same_dtype_device(xb, valb) + + if len(tf.config.list_physical_devices('GPU')) > 0: + # Check that everything happens on the GPU + xb = nx.from_numpy(x) + yb = nx.from_numpy(y) + Pb = nx.from_numpy(P) + valb = ot.max_sliced_wasserstein_distance(xb, yb, projections=Pb) + nx.assert_same_dtype_device(xb, valb) + assert nx.dtype_device(valb)[1].startswith("GPU") -- cgit v1.2.3 From 4f63ea223e27667d4bb275601301d12871fb475f Mon Sep 17 00:00:00 2001 From: Rémi Flamary Date: Wed, 15 Dec 2021 17:30:49 +0100 Subject: [WIP] Update version in to 0.8.1dev and release file (#323) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * update delease file and add it to doc ith mystè_parser * typos in release file * update PR template --- .github/PULL_REQUEST_TEMPLATE.md | 13 +- RELEASES.md | 28 +++ docs/source/releases.rst | 469 +-------------------------------------- ot/__init__.py | 2 +- 4 files changed, 38 insertions(+), 474 deletions(-) (limited to '.github') diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md index 7cfe4e6..f2c6606 100644 --- a/.github/PULL_REQUEST_TEMPLATE.md +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -1,10 +1,6 @@ ## Types of changes -- [ ] Docs change / refactoring / dependency upgrade -- [ ] Bug fix (non-breaking change which fixes an issue) -- [ ] New feature (non-breaking change which adds functionality) -- [ ] Breaking change (fix or feature that would cause existing functionality to change) ## Motivation and context / Related issue @@ -13,16 +9,19 @@ + ## How has this been tested (if it applies) -## Checklist + +## PR checklist -- [ ] The documentation is up-to-date with the changes I made. - [ ] I have read the [**CONTRIBUTING**](CONTRIBUTING.md) document. -- [ ] All tests passed, and additional code has been covered with new tests. +- [ ] The documentation is up-to-date with the changes I made (check build artifacts). +- [ ] All tests passed, and additional code has been **covered with new tests**. +- [ ] I have added the PR and Issue fix to the [**RELEASES.md**](RELEASES.md) file. diff --git a/RELEASES.md b/RELEASES.md index 6eb1502..6d4f565 100644 --- a/RELEASES.md +++ b/RELEASES.md @@ -1,6 +1,34 @@ # Releases +## 0.8.1dev +*December 2021* + +This release fixes several bugs and introduce two new backends: Cupy +and Tensorflow. Note that teh tensorflow backend will work only when tensorflow +has enabled the Numpy behavior (that's for transpose that is not by default in +tensorflow). We also introduce a simple benchmark on CPU GPU for te sinkhorn +solver that will be provided in teh documentation. + +As always we want to that the contributors who helped mak POT better (and bug free). + +#### New features + +- New benchmark for sinkhorn solver on CPU/GPU and between backends (PR #316) +- New tensorflow backend (PR #316) +- New Cupy backend (PR #315) +- Documentation always up-to-date with README, RELEASES, CONTRIBUTING and + CODE_OF_CONDUCT files (PR #316, PR #322). + +#### Closed issues + +- Fix bug in `ot.dist` function when non euclidean distance (Issue #305, PR #306) +- Fix gradient scaling for functions using `nx.set_gradients` (Issue #309, PR + #310) +- Fix bug in generalized Conditional gradient solver and SinkhornL1L2 (Issue + #311, PR #313) +- Fix log error in `gromov_barycenters` (Issue #317, PR #3018) + ## 0.8.0 *November 2021* diff --git a/docs/source/releases.rst b/docs/source/releases.rst index aa06105..8250a4d 100644 --- a/docs/source/releases.rst +++ b/docs/source/releases.rst @@ -1,469 +1,6 @@ Releases ======== -0.8.0 ------ - -*November 2021* - -This new stable release introduces several important features. - -First we now have an OpenMP compatible exact ot solver in ``ot.emd``. -The OpenMP version is used when the parameter ``numThreads`` is greater -than one and can lead to nice speedups on multi-core machines. - -| Second we have introduced a backend mechanism that allows to use - standard POT function seamlessly on Numpy, Pytorch and Jax arrays. - Other backends are coming but right now POT can be used seamlessly for - training neural networks in Pytorch. Notably we propose the first - differentiable computation of the exact OT loss with ``ot.emd2`` (can - be differentiated w.r.t. both cost matrix and sample weights), but - also for the classical Sinkhorn loss with ``ot.sinkhorn2``, the - Wasserstein distance in 1D with ``ot.wasserstein_1d``, sliced - Wasserstein with ``ot.sliced_wasserstein_distance`` and - Gromov-Wasserstein with ``ot.gromov_wasserstein2``. Examples of how - this new feature can be used are now available in the documentation - where the Pytorch backend is used to estimate a `minimal Wasserstein - estimator `__, - a `Generative Network - (GAN) `__, - for a `sliced Wasserstein gradient - flow `__ - and `optimizing the Gromov-Wassersein - distance `__. - Note that the Jax backend is still in early development and quite slow - at the moment, we strongly recommend for Jax users to use the `OTT - toolbox `__ when possible. -| As a result of this new feature, the old ``ot.gpu`` submodule is now - deprecated since GPU implementations can be done using GPU arrays on - the torch backends. - -Other novel features include implementation for `Sampled Gromov -Wasserstein and Pointwise Gromov -Wasserstein `__, -Sinkhorn in log space with ``method='sinkhorn_log'``, `Projection Robust -Wasserstein `__, -ans `deviased Sinkorn -barycenters `__. - -This release will also simplify the installation process. We have now a -``pyproject.toml`` that defines the build dependency and POT should now -build even when cython is not installed yet. Also we now provide -pe-compiled wheels for linux ``aarch64`` that is used on Raspberry PI -and android phones and for MacOS on ARM processors. - -Finally POT was accepted for publication in the Journal of Machine -Learning Research (JMLR) open source software track and we ask the POT -users to cite `this -paper `__ from now on. The -documentation has been improved in particular by adding a "Why OT?" -section to the quick start guide and several new examples illustrating -the new features. The documentation now has two version : the stable -version https://pythonot.github.io/ corresponding to the last release -and the master version https://pythonot.github.io/master that -corresponds to the current master branch on GitHub. - -As usual, we want to thank all the POT contributors (now 37 people have -contributed to the toolbox). But for this release we thank in particular -Nathan Cassereau and Kamel Guerda from the AI support team at -`IDRIS `__ for their support to the development of -the backend and OpenMP implementations. - -New features -^^^^^^^^^^^^ - -- OpenMP support for exact OT solvers (PR #260) -- Backend for running POT in numpy/torch + exact solver (PR #249) -- Backend implementation of most functions in ``ot.bregman`` (PR #280) -- Backend implementation of most functions in ``ot.optim`` (PR #282) -- Backend implementation of most functions in ``ot.gromov`` (PR #294, - PR #302) -- Test for arrays of different type and device (CPU/GPU) (PR #304, - #303) -- Implementation of Sinkhorn in log space with - ``method='sinkhorn_log'`` (PR #290) -- Implementation of regularization path for L2 Unbalanced OT (PR #274) -- Implementation of Projection Robust Wasserstein (PR #267) -- Implementation of Debiased Sinkhorn Barycenters (PR #291) -- Implementation of Sampled Gromov Wasserstein and Pointwise Gromov - Wasserstein (PR #275) -- Add ``pyproject.toml`` and build POT without installing cython first - (PR #293) -- Lazy implementation in log space for sinkhorn on samples (PR #259) -- Documentation cleanup (PR #298) -- Two up-to-date documentations `for stable - release `__ and for `master - branch `__. -- Building wheels on ARM for Raspberry PI and smartphones (PR #238) -- Update build wheels to new version and new pythons (PR #236, #253) -- Implementation of sliced Wasserstein distance (Issue #202, PR #203) -- Add minimal build to CI and perform pep8 test separately (PR #210) -- Speedup of tests and return run time (PR #262) -- Add "Why OT" discussion to the documentation (PR #220) -- New introductory example to discrete OT in the documentation (PR - #191) -- Add templates for Issues/PR on Github (PR#181) - -Closed issues -^^^^^^^^^^^^^ - -- Debug Memory leak in GAN example (#254) -- DEbug GPU bug (Issue #284, #287, PR #288) -- set\_gradients method for JAX backend (PR #278) -- Quicker GAN example for CircleCI build (PR #258) -- Better formatting in Readme (PR #234) -- Debug CI tests (PR #240, #241, #242) -- Bug in Partial OT solver dummy points (PR #215) -- Bug when Armijo linesearch (Issue #184, #198, #281, PR #189, #199, - #286) -- Bug Barycenter Sinkhorn (Issue 134, PR #195) -- Infeasible solution in exact OT (Issues #126,#93, PR #217) -- Doc for SUpport Barycenters (Issue #200, PR #201) -- Fix labels transport in BaseTransport (Issue #207, PR #208) -- Bug in ``emd_1d``, non respected bounds (Issue #169, PR #170) -- Removed Python 2.7 support and update codecov file (PR #178) -- Add normalization for WDA and test it (PR #172, #296) -- Cleanup code for new version of ``flake8`` (PR #176) -- Fixed requirements in ``setup.py`` (PR #174) -- Removed specific MacOS flags (PR #175) - -0.7.0 ------ - -*May 2020* - -This is the new stable release for POT. We made a lot of changes in the -documentation and added several new features such as Partial OT, -Unbalanced and Multi Sources OT Domain Adaptation and several bug fixes. -One important change is that we have created the GitHub organization -`PythonOT `__ that now owns the main POT -repository https://github.com/PythonOT/POT and the repository for the -new documentation is now hosted at https://PythonOT.github.io/. - -This is the first release where the Python 2.7 tests have been removed. -Most of the toolbox should still work but we do not offer support for -Python 2.7 and will close related Issues. - -A lot of changes have been done to the documentation that is now hosted -on https://PythonOT.github.io/ instead of readthedocs. It was a hard -choice but readthedocs did not allow us to run sphinx-gallery to update -our beautiful examples and it was a huge amount of work to maintain. The -documentation is now automatically compiled and updated on merge. We -also removed the notebooks from the repository for space reason and also -because they are all available in the `example -gallery `__. Note -that now the output of the documentation build for each commit in the PR -is available to check that the doc builds correctly before merging which -was not possible with readthedocs. - -The CI framework has also been changed with a move from Travis to Github -Action which allows to get faster tests on Windows, MacOS and Linux. We -also now report our coverage on -`Codecov.io `__ and we have a -reasonable 92% coverage. We also now generate wheels for a number of OS -and Python versions at each merge in the master branch. They are -available as outputs of this -`action `__. -This will allow simpler multi-platform releases from now on. - -In terms of new features we now have `OTDA Classes for unbalanced -OT `__, -a new Domain adaptation class form `multi domain problems -(JCPOT) `__, -and several solvers to solve the `Partial Optimal -Transport `__ -problems. - -This release is also the moment to thank all the POT contributors (old -and new) for helping making POT such a nice toolbox. A lot of changes -(also in the API) are coming for the next versions. - -Features -^^^^^^^^ - -- New documentation on https://PythonOT.github.io/ (PR #160, PR #143, - PR #144) -- Documentation build on CircleCI with sphinx-gallery (PR #145,PR #146, - #155) -- Run sphinx gallery in CI (PR #146) -- Remove notebooks from repo because available in doc (PR #156) -- Build wheels in CI (#157) -- Move from travis to GitHub Action for Windows, MacOS and Linux (PR - #148, PR #150) -- Partial Optimal Transport (PR#141 and PR #142) -- Laplace regularized OTDA (PR #140) -- Multi source DA with target shift (PR #137) -- Screenkhorn algorithm (PR #121) - -Closed issues -^^^^^^^^^^^^^ - -- Add JMLR paper to teh readme ad Mathieu Blondel to the Acknoledgments - (PR #231, #232) -- Bug in Unbalanced OT example (Issue #127) -- Clean Cython output when calling setup.py clean (Issue #122) -- Various Macosx compilation problems (Issue #113, Issue #118, PR#130) -- EMD dimension mismatch (Issue #114, Fixed in PR #116) -- 2D barycenter bug for non square images (Issue #124, fixed in PR - #132) -- Bad value in EMD 1D (Issue #138, fixed in PR #139) -- Log bugs for Gromov-Wassertein solver (Issue #107, fixed in PR #108) -- Weight issues in barycenter function (PR #106) - -0.6.0 ------ - -*July 2019* - -This is the first official stable release of POT and this means a jump -to 0.6! The library has been used in the wild for a while now and we -have reached a state where a lot of fundamental OT solvers are available -and tested. It has been quite stable in the last months but kept the -beta flag in its Pypi classifiers until now. - -Note that this release will be the last one supporting officially Python -2.7 (See https://python3statement.org/ for more reasons). For next -release we will keep the travis tests for Python 2 but will make them -non necessary for merge in 2020. - -The features are never complete in a toolbox designed for solving -mathematical problems and research but with the new contributions we now -implement algorithms and solvers from 24 scientific papers (listed in -the README.md file). New features include a direct implementation of the -`empirical Sinkhorn -divergence `__, -a new efficient (Cython implementation) solver for `EMD in -1D `__ and -corresponding `Wasserstein -1D `__. -We now also have implementations for `Unbalanced -OT `__ -and a solver for `Unbalanced OT -barycenters `__. -A new variant of Gromov-Wasserstein divergence called `Fused -Gromov-Wasserstein `__ -has been also contributed with exemples of use on `structured -data `__ -and computing `barycenters of labeld -graphs `__. - -A lot of work has been done on the documentation with several new -examples corresponding to the new features and a lot of corrections for -the docstrings. But the most visible change is a new `quick start -guide `__ for POT -that gives several pointers about which function or classes allow to -solve which specific OT problem. When possible a link is provided to -relevant examples. - -We will also provide with this release some pre-compiled Python wheels -for Linux 64bit on github and pip. This will simplify the install -process that before required a C compiler and numpy/cython already -installed. - -Finally we would like to acknowledge and thank the numerous contributors -of POT that has helped in the past build the foundation and are still -contributing to bring new features and solvers to the library. - -Features -^^^^^^^^ - -- Add compiled manylinux 64bits wheels to pip releases (PR #91) -- Add quick start guide (PR #88) -- Make doctest work on travis (PR #90) -- Update documentation (PR #79, PR #84) -- Solver for EMD in 1D (PR #89) -- Solvers for regularized unbalanced OT (PR #87, PR#99) -- Solver for Fused Gromov-Wasserstein (PR #86) -- Add empirical Sinkhorn and empirical Sinkhorn divergences (PR #80) - -Closed issues -^^^^^^^^^^^^^ - -- Issue #59 fail when using "pip install POT" (new details in doc+ - hopefully wheels) -- Issue #85 Cannot run gpu modules -- Issue #75 Greenkhorn do not return log (solved in PR #76) -- Issue #82 Gromov-Wasserstein fails when the cost matrices are - slightly different -- Issue #72 Macosx build problem - -0.5.0 ------ - -*Sep 2018* - -POT is 2 years old! This release brings numerous new features to the -toolbox as listed below but also several bug correction. - -| Among the new features, we can highlight a `non-regularized - Gromov-Wasserstein - solver `__, - a new `greedy variant of - sinkhorn `__, -| `non-regularized `__, - `convolutional - (2D) `__ - and `free - support `__ - Wasserstein barycenters and - `smooth `__ - and - `stochastic `__ - implementation of entropic OT. - -POT 0.5 also comes with a rewriting of ot.gpu using the cupy framework -instead of the unmaintained cudamat. Note that while we tried to keed -changes to the minimum, the OTDA classes were deprecated. If you are -happy with the cudamat implementation, we recommend you stay with stable -release 0.4 for now. - -The code quality has also improved with 92% code coverage in tests that -is now printed to the log in the Travis builds. The documentation has -also been greatly improved with new modules and examples/notebooks. - -This new release is so full of new stuff and corrections thanks to the -old and new POT contributors (you can see the list in the -`readme `__). - -Features -^^^^^^^^ - -- Add non regularized Gromov-Wasserstein solver (PR #41) -- Linear OT mapping between empirical distributions and 90% test - coverage (PR #42) -- Add log parameter in class EMDTransport and SinkhornLpL1Transport (PR - #44) -- Add Markdown format for Pipy (PR #45) -- Test for Python 3.5 and 3.6 on Travis (PR #46) -- Non regularized Wasserstein barycenter with scipy linear solver - and/or cvxopt (PR #47) -- Rename dataset functions to be more sklearn compliant (PR #49) -- Smooth and sparse Optimal transport implementation with entropic and - quadratic regularization (PR #50) -- Stochastic OT in the dual and semi-dual (PR #52 and PR #62) -- Free support barycenters (PR #56) -- Speed-up Sinkhorn function (PR #57 and PR #58) -- Add convolutional Wassersein barycenters for 2D images (PR #64) -- Add Greedy Sinkhorn variant (Greenkhorn) (PR #66) -- Big ot.gpu update with cupy implementation (instead of un-maintained - cudamat) (PR #67) - -Deprecation -^^^^^^^^^^^ - -Deprecated OTDA Classes were removed from ot.da and ot.gpu for version -0.5 (PR #48 and PR #67). The deprecation message has been for a year -here since 0.4 and it is time to pull the plug. - -Closed issues -^^^^^^^^^^^^^ - -- Issue #35 : remove import plot from ot/\ **init**.py (See PR #41) -- Issue #43 : Unusable parameter log for EMDTransport (See PR #44) -- Issue #55 : UnicodeDecodeError: 'ascii' while installing with pip - -0.4 ---- - -*15 Sep 2017* - -This release contains a lot of contribution from new contributors. - -Features -^^^^^^^^ - -- Automatic notebooks and doc update (PR #27) -- Add gromov Wasserstein solver and Gromov Barycenters (PR #23) -- emd and emd2 can now return dual variables and have max\_iter (PR #29 - and PR #25) -- New domain adaptation classes compatible with scikit-learn (PR #22) -- Proper tests with pytest on travis (PR #19) -- PEP 8 tests (PR #13) - -Closed issues -^^^^^^^^^^^^^ - -- emd convergence problem du to fixed max iterations (#24) -- Semi supervised DA error (#26) - -0.3.1 ------ - -*11 Jul 2017* - -- Correct bug in emd on windows - -0.3 ---- - -*7 Jul 2017* - -- emd\* and sinkhorn\* are now performed in parallel for multiple - target distributions -- emd and sinkhorn are for OT matrix computation -- emd2 and sinkhorn2 are for OT loss computation -- new notebooks for emd computation and Wasserstein Discriminant - Analysis -- relocate notebooks -- update documentation -- clean\_zeros(a,b,M) for removimg zeros in sparse distributions -- GPU implementations for sinkhorn and group lasso regularization - -V0.2 ----- - -*7 Apr 2017* - -- New dimensionality reduction method (WDA) -- Efficient method emd2 returns only tarnsport (in paralell if several - histograms given) - -0.1.11 ------- - -*5 Jan 2017* - -- Add sphinx gallery for better documentation -- Small efficiency tweak in sinkhorn -- Add simple tic() toc() functions for timing - -0.1.10 ------- - -*7 Nov 2016* \* numerical stabilization for sinkhorn (log domain and -epsilon scaling) - -0.1.9 ------ - -*4 Nov 2016* - -- Update classes and examples for domain adaptation -- Joint OT matrix and mapping estimation - -0.1.7 ------ - -*31 Oct 2016* - -- Original Domain adaptation classes - -0.1.3 ------ - -- pipy works - -First pre-release ------------------ - -*28 Oct 2016* - -It provides the following solvers: \* OT solver for the linear program/ -Earth Movers Distance. \* Entropic regularization OT solver with -Sinkhorn Knopp Algorithm. \* Bregman projections for Wasserstein -barycenter [3] and unmixing. \* Optimal transport for domain adaptation -with group lasso regularization \* Conditional gradient and Generalized -conditional gradient for regularized OT. - -Some demonstrations (both in Python and Jupyter Notebook format) are -available in the examples folder. +.. include:: ../../RELEASES.md + :parser: myst_parser.sphinx_ + :start-line: 3 diff --git a/ot/__init__.py b/ot/__init__.py index b6dc2b4..55972ea 100644 --- a/ot/__init__.py +++ b/ot/__init__.py @@ -50,7 +50,7 @@ from .gromov import (gromov_wasserstein, gromov_wasserstein2, # utils functions from .utils import dist, unif, tic, toc, toq -__version__ = "0.8.0" +__version__ = "0.8.1dev" __all__ = ['emd', 'emd2', 'emd_1d', 'sinkhorn', 'sinkhorn2', 'utils', 'datasets', 'bregman', 'lp', 'tic', 'toc', 'toq', 'gromov', -- cgit v1.2.3 From 176c6b4a72e06233f6e238e4a80c94b853a0d493 Mon Sep 17 00:00:00 2001 From: Rémi Flamary Date: Sun, 26 Dec 2021 22:27:18 +0100 Subject: [MRG] Release 0.8.1 (#325) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * update delease file and add it to doc ith mystè_parser * typos in release file * update PR template * test debug doc build * test debug doc build * wrog circleci * set proper version number * add numpy 1.20 constraint * remove python 3.6 deprecated in december * add python 3.10 * debug pip on windows * proper yml * remoe 3.10 becauqe of troch * next try * try distutils * back * try something * new stuf * debug yaml * test back to old vriso f numpy * try something * windows is worksing? --- .github/workflows/build_tests.yml | 21 +++++++++++++++++++-- RELEASES.md | 16 +++++++++------- ot/__init__.py | 2 +- requirements.txt | 2 +- setup.cfg | 2 +- setup.py | 1 + 6 files changed, 32 insertions(+), 12 deletions(-) (limited to '.github') diff --git a/.github/workflows/build_tests.yml b/.github/workflows/build_tests.yml index ee5a435..3c99da8 100644 --- a/.github/workflows/build_tests.yml +++ b/.github/workflows/build_tests.yml @@ -22,7 +22,7 @@ jobs: strategy: max-parallel: 4 matrix: - python-version: [ "3.6", "3.7", "3.8", "3.9"] + python-version: ["3.7", "3.8", "3.9"] steps: - uses: actions/checkout@v1 @@ -128,12 +128,29 @@ jobs: uses: actions/setup-python@v1 with: python-version: ${{ matrix.python-version }} + - name: RC.exe + run: | + function Invoke-VSDevEnvironment { + $vswhere = "${env:ProgramFiles(x86)}\Microsoft Visual Studio\Installer\vswhere.exe" + $installationPath = & $vswhere -prerelease -legacy -latest -property installationPath + $Command = Join-Path $installationPath "Common7\Tools\vsdevcmd.bat" + & "${env:COMSPEC}" /s /c "`"$Command`" -no_logo && set" | Foreach-Object { + if ($_ -match '^([^=]+)=(.*)') { + [System.Environment]::SetEnvironmentVariable($matches[1], $matches[2]) + } + } + } + Invoke-VSDevEnvironment + Get-Command rc.exe | Format-Table -AutoSize + - name: Update pip + run : | + python -m pip install --upgrade pip setuptools + python -m pip install cython - name: Install POT run: | python -m pip install -e . - name: Install dependencies run: | - python -m pip install --upgrade pip python -m pip install -r .github/requirements_test_windows.txt python -m pip install torch==1.8.1+cpu -f https://download.pytorch.org/whl/torch_stable.html python -m pip install pytest "pytest-cov<2.6" diff --git a/RELEASES.md b/RELEASES.md index 6d4f565..ff65fc1 100644 --- a/RELEASES.md +++ b/RELEASES.md @@ -1,16 +1,18 @@ # Releases -## 0.8.1dev +## 0.8.1 *December 2021* -This release fixes several bugs and introduce two new backends: Cupy -and Tensorflow. Note that teh tensorflow backend will work only when tensorflow -has enabled the Numpy behavior (that's for transpose that is not by default in -tensorflow). We also introduce a simple benchmark on CPU GPU for te sinkhorn -solver that will be provided in teh documentation. +This release fixes several bugs and introduces two new backends: Cupy +and Tensorflow. Note that the tensorflow backend will work only when tensorflow +has enabled the Numpy behavior (for transpose that is not by default in +tensorflow). We also introduce a simple benchmark on CPU GPU for the sinkhorn +solver that will be provided in the +[backend](https://pythonot.github.io/gen_modules/ot.backend.html) documentation. -As always we want to that the contributors who helped mak POT better (and bug free). + +As always we want to that the contributors who helped make POT better (and bug free). #### New features diff --git a/ot/__init__.py b/ot/__init__.py index 55972ea..e436571 100644 --- a/ot/__init__.py +++ b/ot/__init__.py @@ -50,7 +50,7 @@ from .gromov import (gromov_wasserstein, gromov_wasserstein2, # utils functions from .utils import dist, unif, tic, toc, toq -__version__ = "0.8.1dev" +__version__ = "0.8.1" __all__ = ['emd', 'emd2', 'emd_1d', 'sinkhorn', 'sinkhorn2', 'utils', 'datasets', 'bregman', 'lp', 'tic', 'toc', 'toq', 'gromov', diff --git a/requirements.txt b/requirements.txt index d43be7a..8b75241 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,4 +1,4 @@ -numpy +numpy>=1.16 scipy>=1.3 cython matplotlib diff --git a/setup.cfg b/setup.cfg index 1177faf..9a4c434 100644 --- a/setup.cfg +++ b/setup.cfg @@ -1,5 +1,5 @@ [metadata] -description-file = README.md +description_file = README.md [flake8] exclude = __init__.py diff --git a/setup.py b/setup.py index 86c7c8d..44cc6dd 100644 --- a/setup.py +++ b/setup.py @@ -95,5 +95,6 @@ setup( 'Programming Language :: Python :: 3.7', 'Programming Language :: Python :: 3.8', 'Programming Language :: Python :: 3.9', + 'Programming Language :: Python :: 3.10', ] ) -- cgit v1.2.3 From e39e05b66a0f2089aa8ebcbf44a14b8d311cbbee Mon Sep 17 00:00:00 2001 From: Rémi Flamary Date: Mon, 27 Dec 2021 14:06:07 +0100 Subject: [MRG] Remove Python3.6 from wheels (#327) * remove wheel 3.6 --- .github/workflows/build_wheels.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to '.github') diff --git a/.github/workflows/build_wheels.yml b/.github/workflows/build_wheels.yml index a935a5e..b58e7b3 100644 --- a/.github/workflows/build_wheels.yml +++ b/.github/workflows/build_wheels.yml @@ -80,7 +80,7 @@ jobs: - name: Build wheels env: - CIBW_SKIP: "pp*-win* pp*-macosx* cp2* pp* cp*musl*" # remove pypy on mac and win (wrong version) + CIBW_SKIP: "pp*-win* pp*-macosx* cp2* pp* cp*musl* cp36*" # remove pypy on mac and win (wrong version) CIBW_BEFORE_BUILD: "pip install numpy cython" CIBW_ARCHS_LINUX: auto aarch64 # force aarch64 with QEMU CIBW_ARCHS_MACOS: x86_64 universal2 arm64 -- cgit v1.2.3 From 5ed61689a41350fac40ce995515e6cbcb7203f48 Mon Sep 17 00:00:00 2001 From: Rémi Flamary Date: Mon, 27 Dec 2021 14:46:13 +0100 Subject: [MRG] Remove python 3.6 properly for all wheel building (#328) * remove python3.6 properly --- .github/workflows/build_wheels.yml | 2 +- .github/workflows/build_wheels_weekly.yml | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) (limited to '.github') diff --git a/.github/workflows/build_wheels.yml b/.github/workflows/build_wheels.yml index b58e7b3..c746eb8 100644 --- a/.github/workflows/build_wheels.yml +++ b/.github/workflows/build_wheels.yml @@ -36,7 +36,7 @@ jobs: - name: Build wheels env: - CIBW_SKIP: "pp*-win* pp*-macosx* cp2* pp*" # remove pypy on mac and win (wrong version) + CIBW_SKIP: "pp*-win* pp*-macosx* cp2* pp* cp36*" # remove pypy on mac and win (wrong version) CIBW_BEFORE_BUILD: "pip install numpy cython" run: | python -m cibuildwheel --output-dir wheelhouse diff --git a/.github/workflows/build_wheels_weekly.yml b/.github/workflows/build_wheels_weekly.yml index 2964844..dbf342f 100644 --- a/.github/workflows/build_wheels_weekly.yml +++ b/.github/workflows/build_wheels_weekly.yml @@ -41,7 +41,7 @@ jobs: - name: Build wheels env: - CIBW_SKIP: "pp*-win* pp*-macosx* cp2* pp* cp*musl*" # remove pypy on mac and win (wrong version) + CIBW_SKIP: "pp*-win* pp*-macosx* cp2* pp* cp*musl* cp36*" # remove pypy on mac and win (wrong version) CIBW_BEFORE_BUILD: "pip install numpy cython" CIBW_ARCHS_LINUX: auto aarch64 # force aarch64 with QEMU CIBW_ARCHS_MACOS: x86_64 universal2 arm64 -- cgit v1.2.3