Improve Figure.show for displaying previews in Jupyter notebooks and external viewers

[seisman]

Problem

The problem was first raised by @leouieda in issue #269. In summary, there are too many inconsistent ways for displaying PyGMT images:

• Figure.show() inserts a PNG image in a Jupyter notebook
• Figure.show(method="external") opens a preview in an external PDF viewer or the browser.
• Putting the Figure object at the end of a notebook cell displays the figure as PNG
• Figure.savefig(..., show=True) save the figure and open it on an external viewer

The main problem is, Figure.show() only works in Jupyter notebooks, and when running PyGMT in scripts, we have to use Figure.show(method="external") (see a related issue #366). In the PyGMT documentation, we use Figure.show() in the tutorials and gallery examples, and these examples won't work if users copy and save the codes in Python files. It definitely cause a lot of confusion, that's why we have to add some notes at the beginning of each tutorial (#806).

The Design in PR #528

In #269, @leouieda also proposed a new design of the PyGMT display mechanism.

• Calling Figure.show() opens a preview of the figure and halts the Python process until the viewer is closed. It can also return self so that if it's called at the end of a notebook cell it would get displayed on the notebook as well.
• Putting the Figure at the end of a notebook cell displays a PNG preview.
• Having some way of tell Figure.show() to not open the viewer. This is required when using a notebook or when building the docs (we don't want windows opening when we build the gallery or tutorials). This can be done through a function pygmt.enable_notebook which would set a flag to tell show not to do anything and/or through an environment variable PYGMT_DISPLAY which can be set to none or other options that the user might want. Ideally, we should have both. The env variable will be useful when building the docs.

The core idea of the proposal in #269 is to have a uniform Figure.show() function to open images using external viewers or insert images in Jupyter notebooks. #528 is a draft implementation by @leouieda. It was started a year ago but never finished.

The New Design in this PR

This PR implements a new design for the display mechanism. The new design follows the discussions in #269, but is slightly different from #269 and #528 in details.

In the new design of this PR, PyGMT tries to detect the current running environment (is it in a normal terminal or a Jupyter notebook), and show images in a Jupyter notebook if possible. Otherwise, it opens images using external viewers. So, Figure.show() will work in both Jupyter notebooks and Python scripts in a consistent way.

Here are the default behaviors for some common use cases:

Cases	Open externally?	Inline display?	Notes
Run a script by python display.py	YES	NO	note1
Run a script by ipython display.py.	YES	NO
Run codes in a Python terminal interactively	YES	NO
Run codes in an IPython terminal interactively	YES	NO
Run codes in jupyter qtconsole	NO	YES
Run codes in a notebook, i.e., jupyter notebook or jupyter lab	NO	YES
Run codes in jupyter console	YES	NO	note2

• note1: It's non-blocking. If the script calls Figure.show() multiple times, ideally, it should display the 1st figure, halts the Python process until the viewer is closed, and then display the 2nd figure. As I understand it, it's impossible.
• note2: jupyter console generates a PNG file, and opens it using an external viewer.

When building documentation, it's also possible to disable external display using any one of the two ways:

1. export PYGMT_USE_EXTERNAL_DISPLAY=false
2. Use pygmt.set_display(method="notebook")

Summarized features in this PR

1. Consistent way (Figure.show) to preview images in both Jupyter notebooks and external viewers
2. Disable external viewers by setting PYGMT_USE_EXTERNAL_DISPLAY to false (useful for building documentation)
3. New function pygmt.set_display(method=None, dpi=None, width=None) to set the display method ("external" or "notebook"), and image dpi (default to 300) and width (default to 500) in Jupyter notebooks, globally. It affects all the subsequent figures.
4. Figure.show(method=None, dpi=None, width=None) can set the display method ("external" or "notebook"), and image dpi and width in Jupyter notebooks, locally. It only affects the current figure.

Testing this PR

As listed in the "TODO" section, I don't write add unit tests for the new feature yet. So we need your help to test this PR manually. Here I provide a series of steps you can follow to help test this PR.

Testing the choice of notebook display or external viewer

import pygmt

fig = pygmt.Figure()
fig.basemap(region=[0, 10, 0, 10], projection="X10c/10c", frame="WSrt+tFigure1")
fig.show()

1. Save the script into a Python file (e.g., test1.py), and run it using python test1.py, or ipython test1.py. It will open the image in an external viewer, but you may encounter the issue in Fail to open images in external viewer programs if running PyGMT in a Python script #1061
2. Run the codes in a Python/IPython interpreter iteratively. It will open the image in an external viewer.
3. Run the codes in a Jupyter notebook (e.g., ipython notebook, jupyter notebook or jupyter lab). You should see an image inserted in the notebook.

Testing the display in notebook

In a Jupyter notebook, run:

import pygmt

fig = pygmt.Figure()
fig.basemap(region=[0, 10, 0, 10], projection="X10c/10c", frame="WSrt+tFigure1")
fig

Putting the Figure instance fig in the last cell and run, it will also display an image, similar to Figure.show()

Testing changing display method globally or locally

Images will be inserted into the notebook by default. As mentioned above, there are two ways to change the display method:

• The GLOBAL way: pygmt.set_display(method="xxx")
• The LOCAL way: Figure.show(method="xxx")

The below script can be used to test if the two methods work as expected. You could run it in a Jupyter notebook, in a Python file, or in a Python/IPython interpreter. The comments in the script indicate the expected behavior in a Jupyter notebook:

import pygmt

# Figure 1 should be inserted in the notebook
fig1 = pygmt.Figure()
fig1.basemap(region=[0, 10, 0, 10], projection="X10c/10c", frame="WSrt+tFigure1")
fig1.show()

pygmt.set_display(method="external")
# Figure 2 should be opened in an external viewer
fig2 = pygmt.Figure()
fig2.basemap(region=[0, 10, 0, 10], projection="X10c/10c", frame="WSrt+tFigure2")
fig2.show()

# Figure 3 should be opened in an external viewer
fig3 = pygmt.Figure()
fig3.basemap(region=[0, 10, 0, 10], projection="X10c/10c", frame="WSrt+tFigure3")
fig3.show()

pygmt.set_display(method="notebook")
# Figure 4 should be inserted in the notebook
fig4 = pygmt.Figure()
fig4.basemap(region=[0, 10, 0, 10], projection="X10c/10c", frame="WSrt+tFigure4")
fig4.show()

# Figure 5 should be inserted in the notebook
fig5 = pygmt.Figure()
fig5.basemap(region=[0, 10, 0, 10], projection="X10c/10c", frame="WSrt+tFigure5")
fig5.show()

# Figure 6 should be opened in an external viewer
fig6 = pygmt.Figure()
fig6.basemap(region=[0, 10, 0, 10], projection="X10c/10c", frame="WSrt+tFigure6")
fig6.show(method="external")

Testing change dpi and width

SKIP THIS TEST! THE FEATURE IS TEMPORARILY REMOVED FROM THIS PR!

import pygmt

# dpi=300
fig1 = pygmt.Figure()
fig1.basemap(region=[0, 10, 0, 10], projection="X10c/10c", frame="WSrt+tFigure1")
fig1.show()
# still Figure 1, but dpi=10
fig1.show(dpi=10)

pygmt.set_display(dpi=50, width=200)
# dpi=50
fig2 = pygmt.Figure()
fig2.basemap(region=[0, 10, 0, 10], projection="X10c/10c", frame="WSrt+tFigure2")
fig2.show()

# dpi=50
fig3 = pygmt.Figure()
fig3.basemap(region=[0, 10, 0, 10], projection="X10c/10c", frame="WSrt+tFigure3")
fig3.show()

# dpi=100
fig4 = pygmt.Figure()
fig4.basemap(region=[0, 10, 0, 10], projection="X10c/10c", frame="WSrt+tFigure4")
fig4.show(dpi=100, width=200)

pygmt.set_display(dpi=300)
# dpi=300
fig5 = pygmt.Figure()
fig5.basemap(region=[0, 10, 0, 10], projection="X10c/10c", frame="WSrt+tFigure5")
fig5.show()

fig6 = pygmt.Figure()
# dpi=100
fig6.basemap(region=[0, 10, 0, 10], projection="X10c/10c", frame="WSrt+tFigure6")
fig6.show(dpi=100)

TODO

• Add unit tests to make sure the new display mechanism works as expected
• Discuss about the default dpi (currently 300) and width (currently 500) in Jupyter notebooks
• More?

Fixes #269. Fixes #366.

Reminders

• Run make format and make check to make sure the code follows the style guide.
• Add tests for new features or tests that would have caught the bug that you're fixing.
• Add new public functions/methods/classes to doc/api/index.rst.
• Write detailed docstrings for all functions/methods.
• If adding new functionality, add an example to docstrings or tutorials.

[leouieda]

@seisman thanks for pushing this forward! It would be good to have a manual method of setting the display since there is no guarantee that the notebook detection always works (and this is by design of jupyter). How about merging the two functions you have into pygmt.set_display("notebook") and pygmt.set_display("external")?

[weiji14]

cc @MarkWieczorek who made some comments in #366 (and this PR should ideally close it). Also might want to check out how tqdm handles the jupyter auto-detection at https://github.com/tqdm/tqdm#ipythonjupyter-integration, not sure if I like their import style but it might give you some ideas.

I'd also like to second @leouieda's comments about merging the two functions, since what we need is essentially a boolean True/False toggle (similar to the xarray accessor at #500).

[maxrjones]

Do you think that there should be warnings if dpi or width are used with method=external or method=none?

[seisman]

Do you think that there should be warnings if dpi or width are used with method=external or method=none?

Sounds good to me. I will open a separate PR about dpi and width settings after this PR is merged.

[maxrjones]

This seems like a great improvement to me! It worked well in a few different environments on MacOS and Windows.

Regarding whether figure.show() should return an image, I think that could be a separate feature request and that this PR is a good baseline for any subsequent improvements before v0.4.0.

The TODO lists unit tests, is that still the plan or do you think this is good to go @seisman?

[seisman]

The TODO lists unit tests, is that still the plan or do you think this is good to go @seisman?

Actually, I have no idea how to test the new Figure.show() behavior automatically, because when running make tests, IPython is not imported so Figure.show() will do nothing.

[maxrjones]

I think if you can add a test for invalid input to set_display that would be good. Otherwise this looks good to me. I also do not know a way to add more unit tests, but expect any breaking changes would be noticed very soon. Please wait for @weiji14's approval before merging.

[weiji14]

Ok, let's get this in then! Will see if anyone else wants to comment, otherwise merge it tomorrow.