Review re-usable silx data viewer: only tree or tree + data viewer

[woutdenolf]

TL;DR

• silx has no embeddable HDF5 tree panel — only a bare tree, or a whole application.
• silx has no single hook for "how is a file opened".
• ewoksorange copies ~450 lines from silx; ewoksfluo inherits a QMainWindow app + monkeypatches insertFile.
• Both fixable without touching silx: move file access into an Hdf5TreeModel subclass.

silx — reusable

silx.gui.hdf5.Hdf5TreeView

• The tree widget: drag/drop, context-menu callbacks, selectedH5Nodes().
• createDefaultModel() is documented as the hook to swap the model.
• No toolbar, no actions.

silx.gui.hdf5.Hdf5TreeModel

• Holds the open files.
• The only place a file is opened: insertFile() (sync), insertFileAsync() (threaded).
• Emits sigH5pyObjectLoaded / Removed / Synchronized.

silx.gui.data.DataViewerFrame

• The right-hand panel.
• Picks the best view (plot1d / image / table / NXdata) for whatever you setData().

silx — application (silx.app.view.Viewer), not reusable

• A QMainWindow: menu bar, ApplicationContext (shared colormap + Qt settings), custom-NXdata panel, About dialog.
• Also holds the tree toolbar: refresh / close / expand / collapse / sort.
• Also holds the expanded-state and synchronize bookkeeping.
• All of it name-mangled private: __refreshAction, __splitter, __treeview, …

ewoksorange

DataViewer(QWidget) on main

• Tree + DataViewerFrame.
• Re-implements Viewer's toolbar, expand/refresh/synchronize logic.
• Reason: must fit an Orange mainArea, and must set mode/locking instead of silx.io.open.
• Tree and data panel welded together.

Hdf5TreeViewer + DataViewer on split-data-viewer

• Splits the above into a standalone tree panel and a splitter composing it with DataViewerFrame.
• Solves "tree alone vs tree+data". Does not solve data access.

ewoksfluo

ViewerFile / ViewerGroup / ViewerDataset

• commonh5 proxies that re-open the file per read, so files are never held open.
• The actual value ewoksfluo adds — pure data access, nothing GUI.

DataViewer(SilxViewer)

• Subclasses the application just to get a viewer.
• Costs: setWindowFlags(Qt.Widget) to un-QMainWindow it.
• Costs: _Viewer__treeview, _Viewer__refreshAction, _Viewer__splitter private access.
• Costs: per-instance monkeypatch of model.insertFile.

The two missing seams

1. An embeddable tree panel with a toolbar
   • Exists only fused into Viewer.
   • So ewoksorange copies it, and ewoksfluo inherits an application to get it.
2. One hook for opening a file
   • Hdf5TreeModel has two entry points; only the sync one is practically overridable.
   • So ewoksfluo monkeypatches it.
   • So ewoksorange re-implements synchronize by hand instead of calling model.synchronizeH5pyObject().

Both are currently solved at the wrong layer: inside a subclass of an application window.

Proposal (no silx changes)

Use the seam silx already gives us — Hdf5TreeView.createDefaultModel() + Hdf5TreeModel.insertFile():

ewoksorange:
  Hdf5TreeModel(silx.Hdf5TreeModel)   # one overridable openFile(filename)
  Hdf5TreeViewer(QWidget)             # toolbar + tree                   -> tree alone
  DataViewer(QWidget)                 # splitter(tree, DataViewerFrame)  -> tree + data
                                      # both take model=... for data access

ewoksfluo:
  class ViewerTreeModel(Hdf5TreeModel):
      def openFile(self, filename):
          return ViewerFile(filename)

  DataViewer(parent, model=ViewerTreeModel())

Result

• ewoksfluo drops the QMainWindow, every _Viewer__* access, and the monkeypatch.
• ewoksfluo keeps only the proxy classes that are genuinely its own.
• The tree-panel widget stays the one thing duplicated from silx — the piece worth upstreaming later.

[woutdenolf]

Analysis based on first-principles rather than existing code (human, not AI).

We open HDF5 files in two places in the same python process:

1. the ewoks Task class
   • transient: stays alive until the task finished executing
   • opens files for reading (input file) or writing (output file)
2. the orange OWEwoksBaseWidget class
   • persistent: stays alive as long a the workflow in the orange canvas is alive
   • opens files for visualization (Qt): either the input file or output file of the associated ewoks task

The input file and the output file may be the same file.
The input file may have external writers at any time or may have external links to files that may have external writers at any time.

Input file:

• a external process can open the file for writing at any time: before/during/after the ewoks Task execution
• the ewoks Task should never block a potential incoming writer
• the ewoks Task reads ("r") and handles external writing in different way
  • read data on a best-effort basis and absorb failures
  • read data on a best-effort basis and fail otherwise
  • read data faithfully (using blissdata/silx-retry)
  • read data faithfully, assuming it is static, and fail otherwise
  • read data faithfully, assuming it is static, and absorb failures
• the associated orange OWEwoksBaseWidget in the same process may want to read ("r") and handles external writing in different way
  • visualize the file on a best-effort basis
  • see updates when writers are changing it whenever we access/refresh
  • failures are always absorbed because we don't want to crash the Qt application or have endless pop-ups

Output file:

• the ewoks Task opens the file in append mode (either "a" or "w") and fails when another process tries to do so
• the orange OWEwoksBaseWidget opens the file in read mode ("r") to visualize the results of the ewoks task

Since ewoks Task and orange OWEwoksBaseWidget share the same process that need to open a file in the highest common mode ("a"/"w" higher than "r"). When external writers are to be assumed, read-only without locking is the only option.