@asir was asking me for resources on good research practices, and we realized there’s no central repository of information, so he suggested creating one.
I will, first dump here resources that I think communicate good research practices, and gather any suggestions. Eventually we could turn this into a informational repo, à la NLP-Progress.
- Troubling Trends in Machine Learning Scholarship (2018)
Model Evaluation, Model Selection, and Algorithm Selection in Machine Learning; equivalent to this blog series:
- Show Your Work: Improved Reporting of Experimental Results (Recently featured in Sebastian Ruder’s newsletter)
- Errudite: Scalable, Reproducible, and Testable Error Analysis (Featured in the same newsletter)
- The Hitchhikers Guide to Testing Statistical Significance in Natural Language Processing (github & ACL 2018 paper link)
- Evaluation: from precision, recall and F-measure to ROC, informedness,
markedness & correlation
- What the F-measure doesn’t measure: Features, Flaws, Fallacies and Fixes
- How to Read/Write an International Conference Paper by Graham Neubig
- How to Write a Great Research Paper by Simon Peyton-Jones