Conversation
davidberenstein1957
left a comment
There was a problem hiding this comment.
Hi, looks good. Should we update the documentation somewhere?
4074374 to
63d675a
Compare
|
This PR has been inactive for 10 days and is now marked as stale. |
63d675a to
bb440ac
Compare
sharpenb
left a comment
There was a problem hiding this comment.
That looks very nice! If you can just fix the two comments before merging that would be perfect :)
There was a problem hiding this comment.
Could we say "Stateful metrics can generally be evaluated in single-model and pariwise modes. Single-mode allows to compare a mode against ground-truth data, while pairwise mode allows to compare the fidelity of model against another model. "
There was a problem hiding this comment.
My suggestion would be to return an error here instead of silently fixing the problem :)
There was a problem hiding this comment.
Makes so much sense! I also moved throwing this error to the general utils file where i check call types for all metrics, so we would throw this error not just for tochmetrics but for all of our metrics!
de73860 to
a29d9c4
Compare
a29d9c4 to
d8aa4f4
Compare
Description
This PR adds support for the arniqa image quality metric. Since ARNIQA uses an internal dataset, it now runs with the new call_type="y" (output-only).
Type of Change
How Has This Been Tested?
Checklist