Recommend to use np.testing.assert_allclose instead of assertTrue(np.allclose(...))

[jeng1220]

需求描述 Feature Description

Now many unit tests use assertTrue(np.allclose(x, y), "error message") to compare two tensors, but its error message is helpless. Because assertTrue cannot tell developer the details, such as which value in tensor causes wrong, how different between two tensors is. For instance:

>>> import numpy as np
>>> import unittest
>>> x = [1, 2, 3]
>>> y = [1, 3, 3]
>>> class Foo(unittest.TestCase):
...   def __init__(self):
...     self.assertTrue(np.allclose(x, y), "compare x and y")
...
>>> a = Foo()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 3, in __init__
  File "/usr/lib/python3.8/unittest/case.py", line 765, in assertTrue
    raise self.failureException(msg)
AssertionError: False is not true : compare x and y

The error message only tells developer that False is not true. That's all.

If using np.testing.assert_allclose, the error message will be much clearer. The developer can quickly understand what and where goes wrong from CI log.

>>> import numpy as np
>>> import unittest
>>> x = [1, 2, 3]
>>> y = [1, 3, 3]
>>> class Bar(unittest.TestCase):
...   def __init__(self):
...    np.testing.assert_allclose(x, y, err_msg="compare x and y")
...
>>> b = Bar()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 3, in __init__
  File "/home/rjeng/.local/lib/python3.8/site-packages/numpy/testing/_private/utils.py", line 1532, in assert_allclose
    assert_array_compare(compare, actual, desired, err_msg=str(err_msg),
  File "/home/rjeng/.local/lib/python3.8/site-packages/numpy/testing/_private/utils.py", line 846, in assert_array_compare
    raise AssertionError(msg)
AssertionError:
Not equal to tolerance rtol=0, atol=0
compare x and y
Mismatched elements: 1 / 3 (33.3%)
Max absolute difference: 1
Max relative difference: 0.33333333
 x: array([1, 2, 3])
 y: array([1, 3, 3])

替代实现 Alternatives

Replace assertTrue(np.allclose(...)) with np.testing.assert_allclose in all unit-tests, please.

[jeng1220]

Reopen this issue. I saw np.allclose in unittest again.

test_fused_feedforward_pass failed
 F
======================================================================
FAIL: test_pass (test_fused_feedforward_pass.TestFusedFeedforwadPass)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/paddle/paddle/build/test/legacy_test/test_fused_feedforward_pass.py", line 168, in test_pass
    assert np.allclose(ret_loss, ret_loss_fused)
AssertionError

[SigureMo]

We replaced self.assertTrue(np.allclose(...)) but not assert np.allclose(...). @zrr1999 would you be interested in doing this? I'll create an issue to track this task.

UPDATE: see #55348