Many custom documents at once

[Gldkslfmsd]

Hello,

the README or help messages do not suggest how can I evaluate many custom documents at once, and get aggregated score?

E.g.

MTeval -i A.mt A.ref B.mt B.ref C.mt C.ref -f mt ref mt ref mt ref

This option is important.

[mohammad2928]

Hi,

MTeval and other *eval do not support multiple documents with different sources and references at once, because the naming template will appear and we can not separate them (which source or reference related to a document).

We can use the order format as you showed in the example, but I think it is complex.

[Gldkslfmsd]

then I suggest to change the parameters so it does support. It does't have to be the current way.

Ideas:

MTeval --translations A.mt B.mt C.mt --references A.ref B.ref C.ref

or

echo "A.mt A.ref \n B.mt B.ref \n C.mt C.ref " | MTeval --input_filenames --format mt ref

[mohammad2928]

Thanks for your suggestion, it seems ok for the MT evaluation but for SLT evaluation the number of parameters will be a lot.
So in my opinion, the first approach (using order format) is the best.
Your example:
MTeval -i A.mt A.ref B.mt B.ref C.mt C.ref -f mt ref mt ref mt ref

If you agree I will implement it in the next version.

[Gldkslfmsd]

No. Rather this:

MTeval -i A.mt A.ref B.mt B.ref C.mt C.ref -f mt ref

With 2 items in -f parameters, it means that the files come in pairs = 2 files per document. If 3, then in triples, etc.

And I would also really appreciate this:

echo "A.mt A.ref
          B.mt B.ref
          C.mt C.ref " | MTeval --input_filenames --format mt ref

Because that would be very convenient in scripts. I could do

find -name \*ref | while read x ; do echo $x ${x/.ref/.mt} ; done | MTeval --input_filenames --format ref mt

[mohammad2928]

Ok,
I will implement it as you want.

[obo]

Dominik, please check and close if resolved. Thanks, O.