Audio player

Comments

114 - Behavioral Testing of NLP Models, with Marco Tulio Ribeiro

NLP Highlights

Science & Medicine

We invited Marco Tulio Ribeiro, a Senior Researcher at Microsoft, to talk about evaluating NLP models using behavioral testing, a framework borrowed from Software Engineering. Marco describes three kinds of black-box tests the check whether NLP models satisfy certain necessary conditions. While breaking the standard IID assumption, this framework presents a way to evaluate whether NLP systems are ready for real-world use. We also discuss what capabilities can be tested using this framework, how one can come up with good tests, and the need for an evolving set of behavioral tests for NLP systems. Marco’s homepage: https://homes.cs.washington.edu/~marcotcr/


More Episodes  


Listen to 120 - Evaluation of Text Generation, with Asli Celikyilmaz

120 - Evaluation of Text Generation, with Asli Celikyilmaz

Oct 2, 2020
Listen to 119 - Social NLP, with Diyi Yang

119 - Social NLP, with Diyi Yang

Sep 3, 2020
Listen to 118 - Coreference Resolution, with Marta Recasens

118 - Coreference Resolution, with Marta Recasens

Aug 26, 2020
Listen to 117 - Interpreting NLP Model Predictions, with Sameer Singh

117 - Interpreting NLP Model Predictions, with Sameer Singh

Aug 12, 2020
Listen to 116 - Grounded Language Understanding, with Yonatan Bisk

116 - Grounded Language Understanding, with Yonatan Bisk

Jul 2, 2020