Event banner
Event details
Other cognitive services have been marketed as on-par or exceeding human performance before. Unfortunately, I could not confirm this when trying things out.
How do you compare human performance with model performance? What type of human is setting the benchmark?
If you have questions or comments related to your particular use case, please reach out to us. We are evaluating the models through a combination of automated metrics and human judgement. Performance is dependent on language pair, content type, actual content you translate, so it could vary significantly depending on the case.