Looks like a cool project! I gotta say though, I’m still always surprised how few papers include code. I hope it continues to become more common to include code in the future.
The trend in economics is to require authors to provide their code and data. The journal I edit checks many of the replication packages to confirm that they actually produce the results in the paper. Some papers have been rejected after completing the peer review process and being conditionally accepted.
Looks like a cool project! I gotta say though, I’m still always surprised how few papers include code. I hope it continues to become more common to include code in the future.
The trend in economics is to require authors to provide their code and data. The journal I edit checks many of the replication packages to confirm that they actually produce the results in the paper. Some papers have been rejected after completing the peer review process and being conditionally accepted.
To confirm, is the code and data also made available to the readers, not just the editors?
Late to respond, but it has to be made available to the public. Here's an example for a paper selected at random: https://www.sciencedirect.com/science/article/pii/S014098832...
(I wasn't involved with that paper and don't know if it was audited, but that's how it's done.)
If you're interested in this, they have a pretty good podcast that interviews paper authors: https://blog.joss.theoj.org/2024/01/introducing-josscast
In a similar vein, there is Journal of Machine Learning Research that publishes open source software papers