Add PyPOLCA polishing and use polished assemblies for long-read binning

[Harshita-sriv]

Summary

This PR adds a PyPOLCA polishing module and integrates polished assemblies into the long-read MAG workflow.

Changes

• Added a new PyPOLCA module and conda environment.
• Added publication of PyPOLCA outputs.
• Added the run_pypolca parameter.
• Modified the workflow so that long-read binning uses ch_polished_assemblies instead of raw ch_longread_assemblies.

Validation

Tested on one Nanopore metagenomic sample.

Verified that:

• PyPOLCA successfully generated sample_polished.fasta.
• LONGREAD_BINNING_PREPARATION:MINIMAP2_ASSEMBLY_INDEX used sample_polished.fasta.
• Coverage estimation and downstream binning (MetaBAT2, MaxBin2, CONCOCT, COMEBin, MetaBinner, DAS Tool) executed after PyPOLCA completion.

This ensures MAG generation is performed using polished assemblies rather than raw Flye assemblies.

[dialvarezs]

Hi @Harshita-sriv, thanks for this!
For the meantime, can you join the nf-core org? That way you will able to run the ci without explicit approval.
https://nf-co.re/join

[dialvarezs]

In general I think it looks right, but needs to migrate to the nf-core module, and of course, pass the ci.

[Harshita-sriv]

Thanks for the review. I investigated the CI failures and found that the current implementation does not define ch_polished_assemblies when run_pypolca is not enabled, which causes the test workflows to fail. I'll address this together with the requested migration to the official nf-core PyPOLCA module, removal of debug statements, and formatting fixes

[dialvarezs]

Hi @Harshita-sriv! I think this looks fine, but it would be ideal to wire it up in some test. I'll think what would be the best option here and I'll come back to you.
For the meantime, can you update the changelog?
Thanks!

[d4straub]

Thanks for the addition, I think thats absolutely desired!
Looks good but I have some concerns with multiple samples.

[Harshita-sriv]

I noticed that the remaining lint failure is due to modifications in modules/nf-core/pypolca/run, which no longer matches the upstream module.

The changes were introduced because PyPOLCA was failing during testing, and with the modifications the pipeline completes successfully on the test dataset.

Would it be acceptable to keep these modifications to the PyPOLCA module, or would you prefer that the implementation uses the upstream module unchanged and handles the required logic elsewhere?

[d4straub]

Would it be acceptable to keep these modifications to the PyPOLCA module, or would you prefer that the implementation uses the upstream module unchanged and handles the required logic elsewhere?

Some of the changes are not acceptable: the output block and the versioning must be kept as it was originally. Additionally the line ch_versions = ch_versions.mix(PYPOLCA_RUN.out.versions) in workflows/mag.nf has to be removed.
The incoming data (contigs) should be compressed as in the original code. Even if it would be necessary to allow non-compressed input (which may be ok in same cases), copying the files when already uncompressed seems like an unnecessary burden on IO and file sizes to me (assemblies in nf-core/mag can be huge!).

Generally, small changes on nf-core modules can be done and accepted by the linting via https://nf-co.re/docs/nf-core-tools/cli/modules/patch

[Harshita-sriv]

@d4straub I restored the PyPOLCA module to the upstream nf-core implementation and moved the coassembly-specific logic into the workflow layer (workflows/mag.nf). The workflow now groups short reads appropriately when --coassemble_group is enabled before passing them to PyPOLCA.

The remaining CI issue appears to be the check_local_copy lint check for pypolca/run. Locally, the module matches the installed nf-core module definition (nf-core modules info pypolca/run), and the workflow compiles and reaches PYPOLCA_RUN in stub runs. All other CI checks pass.

Could you advise whether there is an additional module sync/patch step expected for this module?

[d4straub]

Looks good.
I think the easiest way of satisfying the linting is to re-install the module.